How much VRAM do you need for a 4B–6B model?
Short answer: about 3.8 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 4B–6B open model we track.
- Q4_K_M
- 3.8 GB 2.4 GB weights
- Q8_0
- 5.9 GB 4.3 GB weights
- FP16
- 10.3 GB 8.0 GB weights
Cheapest GPU that fits at Q4: Intel Arc B570 (10 GB, $219). Cheapest Mac or mini PC: Mac mini M4 16GB ($789).
Cheapest hardware for each quant
| Setting | Memory needed | Cheapest GPU | Cheapest Mac / mini PC |
|---|---|---|---|
| Q4_K_M, ~4k context | 3.8 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
| Q8_0, ~4k context | 5.9 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
| FP16, ~4k context | 10.3 GB | Intel Arc B580 12 GB usable · $249 | Mac mini M4 16GB 12 GB usable · $789 |
| Q4_K_M, 32k context | 8.0 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
Memory by context length
Each extra token of context adds to the KV cache. Numbers for Qwen3-4B, from its published config; a Q8 KV cache halves the cache part.
| Context | Q4_K_M, FP16 cache | Q4_K_M, Q8 cache | Q8_0, FP16 cache |
|---|---|---|---|
| 4k tokens | 3.8 GB | 3.8 GB | 5.9 GB |
| 8k tokens | 4.4 GB | 3.8 GB | 6.5 GB |
| 16k tokens | 5.6 GB | 4.4 GB | 7.7 GB |
| 32k tokens | 8.0 GB | 5.6 GB | 10.1 GB |
| 64k tokens | 12.9 GB | 8.0 GB | 15.0 GB |
| 128k tokens | 22.5 GB | 12.9 GB | 24.6 GB |
Hardware that runs it
Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate).FitsTight
| Hardware | Q4 need vs. usable | Q4 / Q8 / FP16 | Est. tok/s |
|---|---|---|---|
| GeForce RTX 2060 6GBGPU · $349 | 3.8 / 6.0 GB | Q4FitsQ8TightFP16Won’t fit | 83–117 |
| GeForce RTX 3050 8GBGPU · $249 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 55–78 |
| Radeon RX 7600GPU · $269 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 71–100 |
| Intel Arc A750GPU · $289 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 126–178 |
| GeForce RTX 4060GPU · $299 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 67–95 |
| GeForce RTX 5060GPU · $299 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| Radeon RX 9060 XT 8GBGPU · $299 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 79–112 |
| GeForce RTX 5060 Ti 8GBGPU · $379 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 3060 TiGPU · $399 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 2060 SuperGPU · $399 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 4060 Ti 8GBGPU · $399 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 71–100 |
| GeForce RTX 3070GPU · $499 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 2070GPU · $499 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 2070 SuperGPU · $499 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 3070 TiGPU · $599 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 150–212 |
| GeForce RTX 2080GPU · $699 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 110–156 |
| GeForce RTX 2080 SuperGPU · $699 | 3.8 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 122–173 |
| Intel Arc B570GPU · $219 | 3.8 / 10.0 GB | Q4FitsQ8FitsFP16Won’t fit | 93–132 |
| GeForce RTX 3080 10GBGPU · $699 | 3.8 / 10.0 GB | Q4FitsQ8FitsFP16Won’t fit | 187–265 |
| GeForce RTX 2080 TiGPU · $999 | 3.8 / 11.0 GB | Q4FitsQ8FitsFP16Tight | 152–215 |
| GeForce RTX 2060 12GBGPU | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 83–117 |
| Intel Arc B580GPU · $249 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 112–159 |
| GeForce RTX 3060 12GBGPU · $329 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 89–125 |
| Radeon RX 7700 XTGPU · $449 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 106–151 |
| Radeon RX 6700 XTGPU · $479 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 94–134 |
| GeForce RTX 5070GPU · $549 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 165–234 |
| Radeon RX 6750 XTGPU · $549 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 106–151 |
| GeForce RTX 4070GPU · $599 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 124–176 |
| GeForce RTX 4070 SuperGPU · $599 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 124–176 |
| Mac mini M4 16GBunified memory · $789 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 22–39 |
| GeForce RTX 3080 12GBGPU · $799 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 224–318 |
| GeForce RTX 4070 TiGPU · $799 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 124–176 |
| Mac mini M6 16GBunified memory · $899 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 28–50 |
| GeForce RTX 3080 TiGPU · $1,199 | 3.8 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 224–318 |
| Radeon RX 7600 XTGPU · $329 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 71–100 |
| Radeon RX 9060 XT 16GBGPU · $349 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 79–112 |
| Intel Arc A770 16GBGPU · $349 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 138–195 |
| GeForce RTX 5060 Ti 16GBGPU · $429 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 110–156 |
| GeForce RTX 4060 Ti 16GBGPU · $499 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 71–100 |
| Radeon RX 7800 XTGPU · $499 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 154–217 |
| Radeon RX 7900 GREGPU · $549 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 142–201 |
| Radeon RX 9070GPU · $549 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 157–223 |
| Radeon RX 6800GPU · $579 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 126–178 |
| Radeon RX 9070 XTGPU · $599 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 157–223 |
| Radeon RX 6800 XTGPU · $649 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 126–178 |
| NVIDIA RTX 2000 Ada 16GBGPU · $649 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 55–78 |
| GeForce RTX 5070 TiGPU · $749 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 220–312 |
| GeForce RTX 4070 Ti SuperGPU · $799 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 165–234 |
| GeForce RTX 4080 SuperGPU · $999 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 181–257 |
| GeForce RTX 5080GPU · $999 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 236–335 |
| Radeon RX 6900 XTGPU · $999 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 126–178 |
| NVIDIA RTX A4000GPU · $1,000 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 110–156 |
| Radeon RX 6950 XTGPU · $1,099 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 142–201 |
| GeForce RTX 4080GPU · $1,199 | 3.8 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 176–250 |
| Mac mini M6 24GBunified memory · $1,299 | 3.8 / 18.0 GB | Q4FitsQ8FitsFP16Fits | 31–56 |
| Radeon RX 7900 XTGPU · $899 | 3.8 / 20.0 GB | Q4FitsQ8FitsFP16Fits | 197–279 |
| NVIDIA RTX 4000 Ada 20GBGPU · $1,250 | 3.8 / 20.0 GB | Q4FitsQ8FitsFP16Fits | 89–125 |
| NVIDIA Tesla P40 24GBGPU | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 85–121 |
| Radeon RX 7900 XTXGPU · $999 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 236–335 |
| GeForce RTX 3090GPU · $1,499 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 230–326 |
| GeForce RTX 4090GPU · $1,599 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 248–351 |
| Mac mini M6 32GBunified memory · $1,799 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 31–56 |
| GeForce RTX 3090 TiGPU · $1,999 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 248–351 |
| NVIDIA RTX A5000GPU · $2,250 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 189–268 |
| MacBook Pro M6 32GBunified memory · $2,399 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 31–56 |
| NVIDIA Titan RTXGPU · $2,499 | 3.8 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 165–234 |
| NVIDIA Tesla V100 32GB (PCIe)GPU | 3.8 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 221–314 |
| AMD Radeon Instinct MI50 32GBGPU | 3.8 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 252–357 |
| AMD Radeon Instinct MI60 32GBGPU | 3.8 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 252–357 |
| Intel Arc Pro B70 32GBGPU · $949 | 3.8 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 150–212 |
| GeForce RTX 5090GPU · $1,999 | 3.8 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 441–625 |
| Mac mini M5 Pro 64GBunified memory · $3,199 | 3.8 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 57–101 |
| NVIDIA RTX A6000GPU · $4,649 | 3.8 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 189–268 |
| NVIDIA DGX Spark 64GBunified memory · $4,999 | 3.8 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 50–90 |
| NVIDIA RTX 6000 Ada 48GBGPU · $6,799 | 3.8 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 236–335 |
| Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449 | 3.8 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 47–84 |
| MacBook Pro M4 Max 128GBunified memory · $4,999 | 3.8 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 101–179 |
| MacBook Pro M5 Max 128GBunified memory · $6,699 | 3.8 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 113–201 |
| NVIDIA DGX Spark 128GBunified memory · $6,950 | 3.8 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 50–90 |
| NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565 | 3.8 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 441–625 |
| Mac Studio M5 Ultra 256GBunified memory · $9,499 | 3.8 / 192.0 GB | Q4FitsQ8FitsFP16Fits | 221–394 |
| Mac Studio M5 Ultra 512GBunified memory · $22,299 | 3.8 / 384.0 GB | Q4FitsQ8FitsFP16Fits | 221–394 |
4B–6B models we track
| Model | Q4_K_M | Q8_0 | FP16 |
|---|---|---|---|
| kanana-2-3b-instruct3.5B params · kakaocorp | 3.4 GB | 5.3 GB | 9.1 GB |
| granite-4.2-3b3.7B params · ibm-granite | 3.6 GB | 5.5 GB | 9.4 GB |
| kanana-1.5-v-3b-instruct3.7B params · kakaocorp | 3.6 GB | 5.5 GB | 9.4 GB |
| HyperCLOVAX-SEED-Vision-Instruct-3B3.7B params · naver-hyperclovax | 3.6 GB | 5.5 GB | 9.6 GB |
| CapRL-3B3.8B params · internlm | 3.6 GB | 5.6 GB | 9.6 GB |
| Phi-tiny-MoE-instruct3.8B params · microsoft · MoE | 3.6 GB | 5.6 GB | 9.6 GB |
| HY-Embodied-0.53.8B params · tencent | 3.6 GB | 5.6 GB | 9.7 GB |
| Phi-3.5-mini-instruct3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| MediPhi-Instruct3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| MediPhi3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| MediPhi-Clinical3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| MediPhi-PubMed3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| Nemotron-Labs-Diffusion-3B3.8B params · nvidia | 3.7 GB | 5.7 GB | 9.8 GB |
| Phi-4-mini-instruct3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| Phi-4-mini-reasoning3.8B params · microsoft | 3.7 GB | 5.7 GB | 9.8 GB |
| Phi-4-mini-flash-reasoning3.9B params · microsoft | 3.7 GB | 5.7 GB | 9.9 GB |
| Nanbeige4.1-3B3.9B params · Nanbeige | 3.7 GB | 5.8 GB | 10.0 GB |
| Nanbeige4-3B-Thinking-25113.9B params · Nanbeige | 3.7 GB | 5.8 GB | 10.0 GB |
| NVIDIA-Nemotron-3-Nano-4B-BF164.0B params · nvidia | 3.8 GB | 5.9 GB | 10.1 GB |
| Nemotron Labs Dory 1.0 4B4.0B params · nvidia | 3.8 GB | 5.9 GB | 10.1 GB |
| granite-vision-4.1-4b4.0B params · ibm-granite | 3.8 GB | 5.9 GB | 10.2 GB |
| granite-4.0-3b-vision4.0B params · ibm-granite | 3.8 GB | 5.9 GB | 10.2 GB |
| Mellum-4b-base4.0B params · JetBrains | 3.8 GB | 5.9 GB | 10.2 GB |
| Qwen3-4B4.0B params · Qwen | 3.8 GB | 5.9 GB | 10.3 GB |
| Qwen3-4B-Instruct-25074.0B params · Qwen | 3.8 GB | 5.9 GB | 10.3 GB |
| Qwen3-4B-Thinking-25074.0B params · Qwen | 3.8 GB | 5.9 GB | 10.3 GB |
| MiniCPM-V-44.1B params · openbmb | 3.8 GB | 6.0 GB | 10.3 GB |
| POINTS-Reader4.1B params · tencent | 3.9 GB | 6.0 GB | 10.5 GB |
| Spark-X2.6-4B4.1B params · XHToken | 3.9 GB | 6.0 GB | 10.5 GB |
| Spark-X2.5-4B4.1B params · XHToken | 3.9 GB | 6.0 GB | 10.5 GB |
| Phi-3.5-vision-instruct4.1B params · microsoft | 3.9 GB | 6.1 GB | 10.5 GB |
| Nanbeige4.2-3B4.2B params · Nanbeige | 3.9 GB | 6.1 GB | 10.6 GB |
| gemma-3-4b-it4.3B params · google | 4.0 GB | 6.3 GB | 10.9 GB |
| medgemma-4b-it4.3B params · google | 4.0 GB | 6.3 GB | 10.9 GB |
| medgemma-1.5-4b-it4.3B params · google | 4.0 GB | 6.3 GB | 10.9 GB |
| gemma-3-4b-pt4.3B params · google | 4.0 GB | 6.3 GB | 10.9 GB |
| glm-edge-4b-chat4.3B params · zai-org | 4.0 GB | 6.3 GB | 11.0 GB |
| Qwen3-VL-4B-Instruct4.4B params · Qwen | 4.1 GB | 6.4 GB | 11.2 GB |
| CapRL-Qwen3VL-4B4.4B params · internlm | 4.1 GB | 6.4 GB | 11.2 GB |
| Llama-3.1-Minitron-4B-Width-Base4.5B params · nvidia | 4.1 GB | 6.5 GB | 11.4 GB |
| AFM-4.5B4.6B params · arcee-ai | 4.2 GB | 6.6 GB | 11.6 GB |
| Qwen3.5-4B4.7B params · Qwen | 4.2 GB | 6.7 GB | 11.7 GB |
| S5-nano4.7B params · maritaca-ai | 4.2 GB | 6.7 GB | 11.7 GB |
| Mage-VL4.7B params · microsoft | 4.3 GB | 6.8 GB | 11.9 GB |
| Mage-VL-Cowork4.7B params · microsoft | 4.3 GB | 6.8 GB | 11.9 GB |
| WeVisOne-VL-4B4.8B params · tencent | 4.4 GB | 6.9 GB | 12.1 GB |
| Apriel-5B-Instruct4.8B params · ServiceNow-AI | 4.4 GB | 6.9 GB | 12.1 GB |
| Molmo2-ER4.9B params · allenai | 4.4 GB | 6.9 GB | 12.2 GB |
| Gemma 4 E2B5.1B params · google | 4.6 GB | 7.3 GB | 12.8 GB |
| Youtu-VL-4B-Instruct5.3B params · tencent | 4.7 GB | 7.5 GB | 13.3 GB |
| gemma-3n-E2B-it5.4B params · google | 4.8 GB | 7.6 GB | 13.5 GB |
| Trinity-Nano-Preview6.1B params · arcee-ai · MoE | 5.3 GB | 8.5 GB | 15.1 GB |
| Trinity-Nano-Base6.1B params · arcee-ai · MoE | 5.3 GB | 8.5 GB | 15.1 GB |
| YuE-s1-7B-anneal-en-cot6.2B params · m-a-p | 5.3 GB | 8.6 GB | 15.3 GB |
Quick answers
- How much VRAM do I need for a 4B–6B model?
- About 3.8 GB at Q4_K_M, 5.9 GB at Q8_0 and 10.3 GB at FP16, at a ~4k context. That is for Qwen3-4B (4.0B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
- How much VRAM does a 4B–6B model need for 32k context?
- About 8.0 GB at Q4_K_M with an FP16 KV cache, or 5.6 GB with a Q8 cache, based on Qwen3-4B's published config. Models with different attention layouts need more or less.
- What is the cheapest GPU for a 4B–6B model?
- Of the GPUs we track, the Intel Arc B570 (10 GB, $219 list price) is the cheapest that runs it fully on the GPU at Q4.
- Can a Mac run a 4B–6B model?
- Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M4 16GB (16 GB, about 12.0 GB usable, $789).
- Does a MoE model of the same size need less VRAM?
- No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.
Other sizes: 1B–2B · 2B–3B · 7B–8B · 12B–14B · 16B–21B · 24B–27B · 30B–32B · 40B–50B · 70B · 120B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant