Instructions to use vibhansh/Veda-8B-v1-Cognitive with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use vibhansh/Veda-8B-v1-Cognitive with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="vibhansh/Veda-8B-v1-Cognitive", filename="Veda-8B-v1-IQ4_XS.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use vibhansh/Veda-8B-v1-Cognitive with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M # Run inference directly in the terminal: llama cli -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M # Run inference directly in the terminal: llama cli -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
Use Docker
docker model run hf.co/vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use vibhansh/Veda-8B-v1-Cognitive with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "vibhansh/Veda-8B-v1-Cognitive" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "vibhansh/Veda-8B-v1-Cognitive", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
- Ollama
How to use vibhansh/Veda-8B-v1-Cognitive with Ollama:
ollama run hf.co/vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
- Unsloth Studio
How to use vibhansh/Veda-8B-v1-Cognitive with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vibhansh/Veda-8B-v1-Cognitive to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for vibhansh/Veda-8B-v1-Cognitive to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for vibhansh/Veda-8B-v1-Cognitive to start chatting
- Atomic Chat new
- Docker Model Runner
How to use vibhansh/Veda-8B-v1-Cognitive with Docker Model Runner:
docker model run hf.co/vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
- Lemonade
How to use vibhansh/Veda-8B-v1-Cognitive with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull vibhansh/Veda-8B-v1-Cognitive:Q4_K_M
Run and chat with the model
lemonade run user.Veda-8B-v1-Cognitive-Q4_K_M
List all available models
lemonade list
💠 VEDA-8B-v1-COGNITIVE
Fine-tuned to think. Not just predict.
By Vibhansh Bamnia · 18 · Independent AI Developer · Rajasthan, India
⚡ Why Veda-8B?
Most 8B models pattern-match. Veda-8B reasons.
LoRA fine-tuned on open-source reasoning datasets — built specifically to solve where standard Llama-3 fails.
| Capability | Standard Llama-3-8B | Veda-8B-v1 |
|---|---|---|
| Relational Chain Reasoning | ⚠️ Often fails | ✅ Reliable |
| Multi-hop Logic | ⚠️ Inconsistent | ✅ Strong |
| Code Hallucination | ❌ Common | ✅ Reduced |
| Chain-of-Thought | ⚠️ Partial | ✅ Complete |
🧠 See It In Action
🔗 Relational Reasoning
Prompt: A is B's father. C is A's sister. D is C's son. What is D's relation to B?
Veda-8B:
A is B's father → C is B's aunt → D is C's son
∴ D is B's cousin.
🔣 Logic Syllogism
Prompt: All Bloops are Razzies. All Razzies are Lazzies. Are all Bloops Lazzies?
Veda-8B:
Bloops ⊆ Razzies ⊆ Lazzies
∴ Yes — by transitivity, all Bloops are definitely Lazzies.
💻 Code (No Hallucination)
Prompt: Second largest unique number without sort()
Veda-8B:
def second_largest(nums):
unique = set(nums)
if len(unique) < 2:
return None
first = second = float('-inf')
for n in unique:
if n > first:
second, first = first, n
elif n > second:
second = n
return second
💎 Quantization
| Version | Size | RAM | Use |
|---|---|---|---|
| 🥇 Q4_K_M | 4.92 GB | 8 GB | Local dev, edge |
| 🥈 Q8_0 | 8.54 GB | 16 GB | Production |
| 🔬 F16 | 16 GB+ | 32 GB | Research |
💻 Quick Start
import llama_cpp
llm = llama_cpp.Llama(model_path="./Veda-8B-v1-Q4_K_M.gguf", n_ctx=8192, n_threads=4)
response = llm("A is taller than B. B is taller than C. Is A taller than C?",
max_tokens=512, temperature=0.1)
⚠️ Limitations
- Inherits Llama-3-8B base limitations
- Not for real-time / factual queries
- Verify math-heavy outputs
- Multilingual not tested
No institution. No GPU cluster. No team. Just a focused fine-tune built to make small models reason correctly.
Veda AI Labs · HuggingFace
- Downloads last month
- 236
Model tree for vibhansh/Veda-8B-v1-Cognitive
Base model
meta-llama/Meta-Llama-3-8B-Instruct