How much VRAM do you need for a 2B–3B model?
Short answer: about 3.2 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 2B–3B open model we track.
- Q4_K_M
- 3.2 GB 1.9 GB weights
- Q8_0
- 4.9 GB 3.4 GB weights
- FP16
- 8.4 GB 6.4 GB weights
Cheapest GPU that fits at Q4: Intel Arc B570 (10 GB, $219). Cheapest Mac or mini PC: Mac mini M4 16GB ($789).
Cheapest hardware for each quant
| Setting | Memory needed | Cheapest GPU | Cheapest Mac / mini PC |
|---|---|---|---|
| Q4_K_M, ~4k context | 3.2 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
| Q8_0, ~4k context | 4.9 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
| FP16, ~4k context | 8.4 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
| Q4_K_M, 32k context | 6.5 GB | Intel Arc B570 10 GB usable · $219 | Mac mini M4 16GB 12 GB usable · $789 |
Memory by context length
Each extra token of context adds to the KV cache. Numbers for Llama-3.2-3B-Instruct, from its published config; a Q8 KV cache halves the cache part.
| Context | Q4_K_M, FP16 cache | Q4_K_M, Q8 cache | Q8_0, FP16 cache |
|---|---|---|---|
| 4k tokens | 3.2 GB | 3.2 GB | 4.9 GB |
| 8k tokens | 3.7 GB | 3.2 GB | 5.4 GB |
| 16k tokens | 4.6 GB | 3.7 GB | 6.3 GB |
| 32k tokens | 6.5 GB | 4.6 GB | 8.2 GB |
| 64k tokens | 10.3 GB | 6.5 GB | 12.0 GB |
| 128k tokens | 17.8 GB | 10.3 GB | 19.5 GB |
Hardware that runs it
Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate).FitsTight
| Hardware | Q4 need vs. usable | Q4 / Q8 / FP16 | Est. tok/s |
|---|---|---|---|
| GeForce RTX 2060 6GBGPU · $349 | 3.2 / 6.0 GB | Q4FitsQ8FitsFP16Won’t fit | 104–147 |
| GeForce RTX 3050 8GBGPU · $249 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 69–98 |
| Radeon RX 7600GPU · $269 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 89–126 |
| Intel Arc A750GPU · $289 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 158–223 |
| GeForce RTX 4060GPU · $299 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 84–119 |
| GeForce RTX 5060GPU · $299 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| Radeon RX 9060 XT 8GBGPU · $299 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 99–140 |
| GeForce RTX 5060 Ti 8GBGPU · $379 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 3060 TiGPU · $399 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 2060 SuperGPU · $399 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 4060 Ti 8GBGPU · $399 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 89–126 |
| GeForce RTX 3070GPU · $499 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 2070GPU · $499 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 2070 SuperGPU · $499 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 3070 TiGPU · $599 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 187–265 |
| GeForce RTX 2080GPU · $699 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 138–196 |
| GeForce RTX 2080 SuperGPU · $699 | 3.2 / 8.0 GB | Q4FitsQ8FitsFP16Won’t fit | 153–216 |
| Intel Arc B570GPU · $219 | 3.2 / 10.0 GB | Q4FitsQ8FitsFP16Fits | 117–166 |
| GeForce RTX 3080 10GBGPU · $699 | 3.2 / 10.0 GB | Q4FitsQ8FitsFP16Fits | 234–332 |
| GeForce RTX 2080 TiGPU · $999 | 3.2 / 11.0 GB | Q4FitsQ8FitsFP16Fits | 190–269 |
| GeForce RTX 2060 12GBGPU | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 104–147 |
| Intel Arc B580GPU · $249 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 140–199 |
| GeForce RTX 3060 12GBGPU · $329 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 111–157 |
| Radeon RX 7700 XTGPU · $449 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 133–189 |
| Radeon RX 6700 XTGPU · $479 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 118–168 |
| GeForce RTX 5070GPU · $549 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 207–293 |
| Radeon RX 6750 XTGPU · $549 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 133–189 |
| GeForce RTX 4070GPU · $599 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 155–220 |
| GeForce RTX 4070 SuperGPU · $599 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 155–220 |
| Mac mini M4 16GBunified memory · $789 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 28–49 |
| GeForce RTX 3080 12GBGPU · $799 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 281–398 |
| GeForce RTX 4070 TiGPU · $799 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 155–220 |
| Mac mini M6 16GBunified memory · $899 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 35–63 |
| GeForce RTX 3080 TiGPU · $1,199 | 3.2 / 12.0 GB | Q4FitsQ8FitsFP16Fits | 281–398 |
| Radeon RX 7600 XTGPU · $329 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 89–126 |
| Radeon RX 9060 XT 16GBGPU · $349 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 99–140 |
| Intel Arc A770 16GBGPU · $349 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 173–244 |
| GeForce RTX 5060 Ti 16GBGPU · $429 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 138–196 |
| GeForce RTX 4060 Ti 16GBGPU · $499 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 89–126 |
| Radeon RX 7800 XTGPU · $499 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 192–272 |
| Radeon RX 7900 GREGPU · $549 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 177–251 |
| Radeon RX 9070GPU · $549 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 197–279 |
| Radeon RX 6800GPU · $579 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 158–223 |
| Radeon RX 9070 XTGPU · $599 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 197–279 |
| Radeon RX 6800 XTGPU · $649 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 158–223 |
| NVIDIA RTX 2000 Ada 16GBGPU · $649 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 69–98 |
| GeForce RTX 5070 TiGPU · $749 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 276–391 |
| GeForce RTX 4070 Ti SuperGPU · $799 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 207–293 |
| GeForce RTX 4080 SuperGPU · $999 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 227–321 |
| GeForce RTX 5080GPU · $999 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 296–419 |
| Radeon RX 6900 XTGPU · $999 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 158–223 |
| NVIDIA RTX A4000GPU · $1,000 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 138–196 |
| Radeon RX 6950 XTGPU · $1,099 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 177–251 |
| GeForce RTX 4080GPU · $1,199 | 3.2 / 16.0 GB | Q4FitsQ8FitsFP16Fits | 221–313 |
| Mac mini M6 24GBunified memory · $1,299 | 3.2 / 18.0 GB | Q4FitsQ8FitsFP16Fits | 39–70 |
| Radeon RX 7900 XTGPU · $899 | 3.2 / 20.0 GB | Q4FitsQ8FitsFP16Fits | 246–349 |
| NVIDIA RTX 4000 Ada 20GBGPU · $1,250 | 3.2 / 20.0 GB | Q4FitsQ8FitsFP16Fits | 111–157 |
| NVIDIA Tesla P40 24GBGPU | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 107–151 |
| Radeon RX 7900 XTXGPU · $999 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 296–419 |
| GeForce RTX 3090GPU · $1,499 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 288–408 |
| GeForce RTX 4090GPU · $1,599 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 311–440 |
| Mac mini M6 32GBunified memory · $1,799 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 39–70 |
| GeForce RTX 3090 TiGPU · $1,999 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 311–440 |
| NVIDIA RTX A5000GPU · $2,250 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 237–335 |
| MacBook Pro M6 32GBunified memory · $2,399 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 39–70 |
| NVIDIA Titan RTXGPU · $2,499 | 3.2 / 24.0 GB | Q4FitsQ8FitsFP16Fits | 207–293 |
| NVIDIA Tesla V100 32GB (PCIe)GPU | 3.2 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 277–393 |
| AMD Radeon Instinct MI50 32GBGPU | 3.2 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 315–447 |
| AMD Radeon Instinct MI60 32GBGPU | 3.2 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 315–447 |
| Intel Arc Pro B70 32GBGPU · $949 | 3.2 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 187–265 |
| GeForce RTX 5090GPU · $1,999 | 3.2 / 32.0 GB | Q4FitsQ8FitsFP16Fits | 552–782 |
| Mac mini M5 Pro 64GBunified memory · $3,199 | 3.2 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 71–126 |
| NVIDIA RTX A6000GPU · $4,649 | 3.2 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 237–335 |
| NVIDIA DGX Spark 64GBunified memory · $4,999 | 3.2 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 63–112 |
| NVIDIA RTX 6000 Ada 48GBGPU · $6,799 | 3.2 / 48.0 GB | Q4FitsQ8FitsFP16Fits | 296–419 |
| Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449 | 3.2 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 59–105 |
| MacBook Pro M4 Max 128GBunified memory · $4,999 | 3.2 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 126–224 |
| MacBook Pro M5 Max 128GBunified memory · $6,699 | 3.2 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 142–252 |
| NVIDIA DGX Spark 128GBunified memory · $6,950 | 3.2 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 63–112 |
| NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565 | 3.2 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 552–782 |
| Mac Studio M5 Ultra 256GBunified memory · $9,499 | 3.2 / 192.0 GB | Q4FitsQ8FitsFP16Fits | 277–493 |
| Mac Studio M5 Ultra 512GBunified memory · $22,299 | 3.2 / 384.0 GB | Q4FitsQ8FitsFP16Fits | 277–493 |
2B–3B models we track
| Model | Q4_K_M | Q8_0 | FP16 |
|---|---|---|---|
| Qwen3-1.7B2.0B params · Qwen | 2.4 GB | 3.5 GB | 5.7 GB |
| Hy-MT2-1.8B2.0B params · tencent | 2.4 GB | 3.5 GB | 5.7 GB |
| kanana-nano-2.1b-instruct2.1B params · kakaocorp | 2.5 GB | 3.6 GB | 5.8 GB |
| Qwen3-VL-2B-Instruct2.1B params · Qwen | 2.5 GB | 3.6 GB | 5.9 GB |
| granite-swash-2b2.1B params · ibm-granite | 2.5 GB | 3.6 GB | 5.9 GB |
| LFM3-Omni-2B2.2B params · LiquidAI | 2.5 GB | 3.6 GB | 6.0 GB |
| Qwen2-VL-2B-Instruct2.2B params · Qwen | 2.5 GB | 3.7 GB | 6.1 GB |
| SmolVLM-Base2.2B params · HuggingFaceTB | 2.6 GB | 3.7 GB | 6.2 GB |
| SmolVLM2-2.2B-Instruct2.2B params · HuggingFaceTB | 2.6 GB | 3.7 GB | 6.2 GB |
| salamandra-2b2.3B params · BSC-LT | 2.6 GB | 3.8 GB | 6.2 GB |
| salamandra-2b-instruct2.3B params · BSC-LT | 2.6 GB | 3.8 GB | 6.2 GB |
| Qwen3.5-2B2.3B params · Qwen | 2.6 GB | 3.8 GB | 6.2 GB |
| kanana-1.5-2.1b-instruct-25052.3B params · kakaocorp | 2.6 GB | 3.8 GB | 6.3 GB |
| EXAONE-3.5-2.4B-Instruct2.4B params · LGAI-EXAONE | 2.7 GB | 3.9 GB | 6.5 GB |
| EXAONE-Deep-2.4B2.4B params · LGAI-EXAONE | 2.7 GB | 3.9 GB | 6.5 GB |
| bitnet-b1.58-2B-4T-bf162.4B params · microsoft | 2.7 GB | 3.9 GB | 6.5 GB |
| UI-Venus-1.5-2B2.4B params · inclusionAI | 2.7 GB | 4.0 GB | 6.6 GB |
| WeVisOne-VL-2B2.4B params · tencent | 2.7 GB | 4.0 GB | 6.6 GB |
| UI-TARS-2B-SFT2.4B params · ByteDance-Seed | 2.7 GB | 4.0 GB | 6.6 GB |
| North-Micro-Vision-Instruct2.5B params · CohereLabs | 2.7 GB | 4.0 GB | 6.7 GB |
| sarvam-1-v0.52.5B params · sarvamai | 2.7 GB | 4.1 GB | 6.8 GB |
| MiniCPM5-2B2.5B params · openbmb | 2.8 GB | 4.1 GB | 6.8 GB |
| MiniCPM5-2B-SFT2.5B params · openbmb | 2.8 GB | 4.1 GB | 6.8 GB |
| MiniCPM5-2B-Midtrain2.5B params · openbmb | 2.8 GB | 4.1 GB | 6.8 GB |
| granite-3.1-2b-instruct2.5B params · ibm-granite | 2.8 GB | 4.1 GB | 6.8 GB |
| granite-3.2-2b-instruct2.5B params · ibm-granite | 2.8 GB | 4.1 GB | 6.8 GB |
| granite-3.3-2b-instruct2.5B params · ibm-granite | 2.8 GB | 4.1 GB | 6.8 GB |
| LFM2-2.6B2.6B params · LiquidAI | 2.8 GB | 4.1 GB | 6.9 GB |
| LFM2-2.6B-Exp2.6B params · LiquidAI | 2.8 GB | 4.1 GB | 6.9 GB |
| LFM2-2.6B-Transcript2.6B params · LiquidAI | 2.8 GB | 4.1 GB | 6.9 GB |
| Motif-2.6B2.6B params · Motif-Technologies | 2.8 GB | 4.2 GB | 7.0 GB |
| EuroMoE-2.6B-A0.6B-Instruct-Preview2.6B params · utter-project · MoE | 2.8 GB | 4.2 GB | 7.0 GB |
| gemma-2-2b-it2.6B params · google | 2.8 GB | 4.2 GB | 7.0 GB |
| gemma-2-2b2.6B params · google | 2.8 GB | 4.2 GB | 7.0 GB |
| granite-3.0-2b-instruct2.6B params · ibm-granite | 2.8 GB | 4.2 GB | 7.1 GB |
| Zamba2-2.7B2.7B params · Zyphra | 2.9 GB | 4.3 GB | 7.1 GB |
| Zamba2-2.7B-instruct2.7B params · Zyphra | 2.9 GB | 4.3 GB | 7.1 GB |
| LFM2.5-2.6B2.7B params · LiquidAI | 2.9 GB | 4.3 GB | 7.2 GB |
| granite-vision-3.2-2b3.0B params · ibm-granite | 3.1 GB | 4.6 GB | 7.8 GB |
| granite-vision-3.1-2b-preview3.0B params · ibm-granite | 3.1 GB | 4.6 GB | 7.8 GB |
| granite-vision-3.3-2b-chart2csv-preview3.0B params · ibm-granite | 3.1 GB | 4.6 GB | 7.8 GB |
| granite-swash-3b-a600m3.0B params · ibm-granite · MoE | 3.1 GB | 4.7 GB | 7.9 GB |
| AI21-Jamba2-3B3.0B params · ai21labs | 3.1 GB | 4.7 GB | 8.0 GB |
| SmolLM3-3B3.1B params · HuggingFaceTB | 3.1 GB | 4.8 GB | 8.1 GB |
| Qwen2.5-3B-Instruct3.1B params · Qwen | 3.2 GB | 4.8 GB | 8.1 GB |
| Qwen2.5-Coder-3B-Instruct3.1B params · Qwen | 3.2 GB | 4.8 GB | 8.1 GB |
| Qwen2.5-3B3.1B params · Qwen | 3.2 GB | 4.8 GB | 8.1 GB |
| Qwen2.5-Coder-3B3.1B params · Qwen | 3.2 GB | 4.8 GB | 8.1 GB |
| d1-3B3.1B params · LiquidAI | 3.2 GB | 4.8 GB | 8.2 GB |
| LFM2.5-VL-3B3.1B params · LiquidAI | 3.2 GB | 4.8 GB | 8.2 GB |
| Falcon-H1-3B-Instruct3.1B params · tiiuae | 3.2 GB | 4.8 GB | 8.2 GB |
| granite-4.0-h-micro3.2B params · ibm-granite | 3.2 GB | 4.9 GB | 8.3 GB |
| AI21-Jamba-Reasoning-3B3.2B params · ai21labs | 3.2 GB | 4.9 GB | 8.4 GB |
| Llama-3.2-3B-Instruct3.2B params · meta-llama | 3.2 GB | 4.9 GB | 8.4 GB |
| Llama-3.2-3B3.2B params · meta-llama | 3.2 GB | 4.9 GB | 8.4 GB |
| Hermes-3-Llama-3.2-3B3.2B params · NousResearch | 3.2 GB | 4.9 GB | 8.4 GB |
| DeepHermes-3-Llama-3-3B-Preview3.2B params · NousResearch | 3.2 GB | 4.9 GB | 8.4 GB |
| Falcon3-3B-Instruct3.2B params · tiiuae | 3.3 GB | 4.9 GB | 8.4 GB |
| granite-3.1-3b-a800m-instruct3.3B params · ibm-granite · MoE | 3.3 GB | 5.0 GB | 8.6 GB |
| granite-4.1-3b3.4B params · ibm-granite | 3.4 GB | 5.2 GB | 8.8 GB |
| granite-4.0-micro3.4B params · ibm-granite | 3.4 GB | 5.2 GB | 8.8 GB |
Quick answers
- How much VRAM do I need for a 2B–3B model?
- About 3.2 GB at Q4_K_M, 4.9 GB at Q8_0 and 8.4 GB at FP16, at a ~4k context. That is for Llama-3.2-3B-Instruct (3.2B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
- How much VRAM does a 2B–3B model need for 32k context?
- About 6.5 GB at Q4_K_M with an FP16 KV cache, or 4.6 GB with a Q8 cache, based on Llama-3.2-3B-Instruct's published config. Models with different attention layouts need more or less.
- What is the cheapest GPU for a 2B–3B model?
- Of the GPUs we track, the Intel Arc B570 (10 GB, $219 list price) is the cheapest that runs it fully on the GPU at Q4.
- Can a Mac run a 2B–3B model?
- Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M4 16GB (16 GB, about 12.0 GB usable, $789).
- Does a MoE model of the same size need less VRAM?
- No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.
Other sizes: 1B–2B · 4B–6B · 7B–8B · 12B–14B · 16B–21B · 24B–27B · 30B–32B · 40B–50B · 70B · 120B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant