How much VRAM do you need to run an LLM?
It comes down to parameter count and quantization. Pick a size for the full breakdown: every quant, long-context needs, and the cheapest hardware that fits.
| Size | Q4_K_M | Q8_0 | FP16 | Cheapest GPU at Q4 | Cheapest Mac / mini PC at Q4 |
|---|---|---|---|---|---|
| 1B–2B modelse.g. Llama-3.2-1B-Instruct | 1.9 GB | 2.5 GB | 3.8 GB | Intel Arc B570$219 | Mac mini M4 16GB$789 |
| 2B–3B modelse.g. Llama-3.2-3B-Instruct | 3.2 GB | 4.9 GB | 8.4 GB | Intel Arc B570$219 | Mac mini M4 16GB$789 |
| 4B–6B modelse.g. Qwen3-4B | 3.8 GB | 5.9 GB | 10.3 GB | Intel Arc B570$219 | Mac mini M4 16GB$789 |
| 7B–8B modelse.g. Qwen3-8B | 6.7 GB | 11.0 GB | 19.8 GB | Intel Arc B570$219 | Mac mini M4 16GB$789 |
| 12B–14B modelse.g. Qwen3-14B | 11.3 GB | 19.0 GB | 35.0 GB | Radeon RX 7600 XT$329 | Mac mini M6 24GB$1,299 |
| 16B–21B modelse.g. gpt-oss-20b | 15.6 GB | 26.6 GB | 49.1 GB | Radeon RX 7900 XT$899 | Mac mini M6 24GB$1,299 |
| 24B–27B modelse.g. Qwen3.8-27B | 20.4 GB | 34.9 GB | 64.9 GB | Intel Arc Pro B70 32GB$949 | Mac mini M6 32GB$1,799 |
| 30B–32B modelse.g. Qwen3-32B | 23.8 GB | 41.0 GB | 76.4 GB | Intel Arc Pro B70 32GB$949 | Mac mini M5 Pro 64GB$3,199 |
| 40B–50B modelse.g. Llama-3_3-Nemotron-Super-49B-v1_5 | 35.8 GB | 61.9 GB | 115.7 GB | NVIDIA RTX A6000$4,649 | Mac mini M5 Pro 64GB$3,199 |
| 70B modelse.g. Llama-3.3-70B-Instruct | 50.2 GB | 87.2 GB | 163.3 GB | NVIDIA RTX Pro 6000 Blackwell 96GB$8,565 | Framework Desktop (Ryzen AI Max+ 395) 128GB$3,449 |
| 120B modelse.g. gpt-oss-120b | 82.5 GB | 143.8 GB | 269.7 GB | NVIDIA RTX Pro 6000 Blackwell 96GB$8,565 | Framework Desktop (Ryzen AI Max+ 395) 128GB$3,449 |
| 200B–300B modelse.g. Qwen3-235B-A22B | 164.9 GB | 288.3 GB | 541.7 GB | Needs 2+ GPUs | Mac Studio M5 Ultra 256GB$9,499 |
| 300B–480B modelse.g. Qwen3.5-397B-A17B | 282.2 GB | 493.9 GB | 928.8 GB | Needs 2+ GPUs | Mac Studio M5 Ultra 512GB$22,299 |
| 500B–800B modelse.g. DeepSeek-R1 | 478.2 GB | 837.4 GB | 1575.3 GB | Needs 2+ GPUs | None we track |
| 1T+ modelse.g. Kimi-K2-Instruct | 716.6 GB | 1255.1 GB | 2361.7 GB | Needs 2+ GPUs | None we track |
Quick answers
- How much VRAM do I need to run an LLM?
- At Q4_K_M, plan on roughly 0.7 GB per billion parameters plus about 1 GB at a ~4k context: 1B–2B ~1.9 GB, 2B–3B ~3.2 GB, 4B–6B ~3.8 GB, 7B–8B ~6.7 GB, 12B–14B ~11.3 GB, 16B–21B ~15.6 GB, 24B–27B ~20.4 GB, 30B–32B ~23.8 GB, 40B–50B ~35.8 GB, 70B ~50.2 GB, 120B ~82.5 GB, 200B–300B ~164.9 GB, 300B–480B ~282.2 GB, 500B–800B ~478.2 GB, 1T+ ~716.6 GB. Q8 needs about 1.75× that and FP16 about 3.3×.
- Is 16 GB of VRAM enough for local LLMs?
- Yes for models up to about 14B at Q4_K_M: a 14B model needs about 11.3 GB at a ~4k context, which leaves room for longer context. gpt-oss-20b needs about 15.6 GB at Q4_K_M by our formula, so it is tight on a 16 GB card, with little room for context; a 20 GB card runs it comfortably. 24B–27B dense models need about 20.4 GB at Q4_K_M, so 24 GB or more.
- Is 24 GB of VRAM enough for a 70B model?
- Not at Q4. A 70B model needs about 50.2 GB at Q4_K_M. You need two 24 GB cards, a 128 GB unified-memory machine, or a much smaller quant.
Ollama VRAM requirements by tag · GGUF VRAM by quant (llama.cpp, LM Studio) · Buying guides · Best LLMs by VRAM size · Best GPU for local LLMs