How much VRAM do you need for a 24B–27B model?
Short answer: about 20.4 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 24B–27B open model we track.
- Q4_K_M
- 20.4 GB 16.8 GB weights
- Q8_0
- 34.9 GB 29.5 GB weights
- FP16
- 64.9 GB 55.6 GB weights
Cheapest GPU that fits at Q4: Intel Arc Pro B70 32GB (32 GB, $949). Cheapest Mac or mini PC: Mac mini M6 32GB ($1,799).
Cheapest hardware for each quant
| Setting | Memory needed | Cheapest GPU | Cheapest Mac / mini PC |
|---|---|---|---|
| Q4_K_M, ~4k context | 20.4 GB | Intel Arc Pro B70 32GB 32 GB usable · $949 | Mac mini M6 32GB 24 GB usable · $1,799 |
| Q8_0, ~4k context | 34.9 GB | NVIDIA RTX A6000 48 GB usable · $4,649 | Mac mini M5 Pro 64GB 48 GB usable · $3,199 |
| FP16, ~4k context | 64.9 GB | NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565 | Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449 |
| Q4_K_M, 32k context | 22.2 GB | Intel Arc Pro B70 32GB 32 GB usable · $949 | Mac mini M5 Pro 64GB 48 GB usable · $3,199 |
Memory by context length
Each extra token of context adds to the KV cache. Numbers for Qwen3.8-27B, from its published config; a Q8 KV cache halves the cache part.
| Context | Q4_K_M, FP16 cache | Q4_K_M, Q8 cache | Q8_0, FP16 cache |
|---|---|---|---|
| 4k tokens | 20.4 GB | 20.4 GB | 34.9 GB |
| 8k tokens | 20.6 GB | 20.4 GB | 35.2 GB |
| 16k tokens | 21.2 GB | 20.6 GB | 35.8 GB |
| 32k tokens | 22.2 GB | 21.2 GB | 36.8 GB |
| 64k tokens | 24.4 GB | 22.2 GB | 39.0 GB |
| 128k tokens | 28.7 GB | 24.4 GB | 43.3 GB |
Hardware that runs it
Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate).FitsTight
| Hardware | Q4 need vs. usable | Q4 / Q8 / FP16 | Est. tok/s |
|---|---|---|---|
| NVIDIA Tesla P40 24GBGPU | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 12–17 |
| Radeon RX 7900 XTXGPU · $999 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 34–48 |
| GeForce RTX 3090GPU · $1,499 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 33–47 |
| GeForce RTX 4090GPU · $1,599 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 36–51 |
| Mac mini M6 32GBunified memory · $1,799 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 5–8 |
| GeForce RTX 3090 TiGPU · $1,999 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 36–51 |
| NVIDIA RTX A5000GPU · $2,250 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 27–39 |
| MacBook Pro M6 32GBunified memory · $2,399 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 5–8 |
| NVIDIA Titan RTXGPU · $2,499 | 20.4 / 24.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 24–34 |
| NVIDIA Tesla V100 32GB (PCIe)GPU | 20.4 / 32.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 32–45 |
| AMD Radeon Instinct MI50 32GBGPU | 20.4 / 32.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 36–52 |
| AMD Radeon Instinct MI60 32GBGPU | 20.4 / 32.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 36–52 |
| Intel Arc Pro B70 32GBGPU · $949 | 20.4 / 32.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 22–31 |
| GeForce RTX 5090GPU · $1,999 | 20.4 / 32.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 64–90 |
| Mac mini M5 Pro 64GBunified memory · $3,199 | 20.4 / 48.0 GB | Q4FitsQ8FitsFP16Won’t fit | 8–15 |
| NVIDIA RTX A6000GPU · $4,649 | 20.4 / 48.0 GB | Q4FitsQ8FitsFP16Won’t fit | 27–39 |
| NVIDIA DGX Spark 64GBunified memory · $4,999 | 20.4 / 48.0 GB | Q4FitsQ8FitsFP16Won’t fit | 7–13 |
| NVIDIA RTX 6000 Ada 48GBGPU · $6,799 | 20.4 / 48.0 GB | Q4FitsQ8FitsFP16Won’t fit | 34–48 |
| Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449 | 20.4 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 7–12 |
| MacBook Pro M4 Max 128GBunified memory · $4,999 | 20.4 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 15–26 |
| MacBook Pro M5 Max 128GBunified memory · $6,699 | 20.4 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 16–29 |
| NVIDIA DGX Spark 128GBunified memory · $6,950 | 20.4 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 7–13 |
| NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565 | 20.4 / 96.0 GB | Q4FitsQ8FitsFP16Fits | 64–90 |
| Mac Studio M5 Ultra 256GBunified memory · $9,499 | 20.4 / 192.0 GB | Q4FitsQ8FitsFP16Fits | 32–57 |
| Mac Studio M5 Ultra 512GBunified memory · $22,299 | 20.4 / 384.0 GB | Q4FitsQ8FitsFP16Fits | 32–57 |
24B–27B models we track
| Model | Q4_K_M | Q8_0 | FP16 |
|---|---|---|---|
| solar-pro-preview-instruct22.1B params · upstage | 16.4 GB | 28.1 GB | 51.9 GB |
| EuroLLM-22B-Instruct-251222.6B params · utter-project | 16.8 GB | 28.7 GB | 53.1 GB |
| EuroLLM-22B-251222.6B params · utter-project | 16.8 GB | 28.7 GB | 53.1 GB |
| Mistral-Small-24B-Instruct-250123.6B params · mistralai | 17.4 GB | 29.8 GB | 55.2 GB |
| LFM2-24B-A2B23.8B params · LiquidAI · MoE | 17.6 GB | 30.1 GB | 55.8 GB |
| LFM2.5-24B-A2B24.0B params · LiquidAI · MoE | 17.7 GB | 30.3 GB | 56.1 GB |
| Mistral-Small-3.2-24B-Instruct-250624.0B params · mistralai | 17.7 GB | 30.3 GB | 56.2 GB |
| Mistral-Small-3.1-24B-Instruct-250324.0B params · mistralai | 17.7 GB | 30.3 GB | 56.2 GB |
| Voxtral-Small-24B-250724.3B params · mistralai | 17.9 GB | 30.6 GB | 56.8 GB |
| Gemma 4 26B A4B25.8B params · google · MoE | 19.0 GB | 32.5 GB | 60.4 GB |
| gemma-4-26B-A4B25.8B params · google · MoE | 19.0 GB | 32.5 GB | 60.4 GB |
| diffusiongemma-26B-A4B-it25.8B params · google · MoE | 19.0 GB | 32.6 GB | 60.4 GB |
| Trinity-Mini26.1B params · arcee-ai · MoE | 19.2 GB | 32.9 GB | 61.1 GB |
| Hemmingway-1.526.9B params · HemmingwayAI | 19.8 GB | 33.9 GB | 62.9 GB |
| medgemma-27b-text-it27.0B params · google | 19.8 GB | 34.0 GB | 63.1 GB |
| Qwen3.8-27B-Longbook27-v227.1B params · summerAIO | 19.9 GB | 34.1 GB | 63.3 GB |
| gemma-2-27b-it27.2B params · google | 20.0 GB | 34.3 GB | 63.6 GB |
| gemma-2-27b27.2B params · google | 20.0 GB | 34.3 GB | 63.6 GB |
| Arete-27B27.4B params · NousResearch | 20.1 GB | 34.4 GB | 63.9 GB |
| gemma-3-27b-it27.4B params · google | 20.1 GB | 34.5 GB | 64.1 GB |
| Qwen3.8-27B27.8B params · Qwen | 20.4 GB | 34.9 GB | 64.9 GB |
| Tess-4.5-27B27.8B params · migtissera | 20.4 GB | 34.9 GB | 64.9 GB |
| Qwen3.6-27B27.8B params · Qwen | 20.4 GB | 34.9 GB | 64.9 GB |
| Qwen3.5-27B27.8B params · Qwen | 20.4 GB | 34.9 GB | 64.9 GB |
| Raon-1 mini preview28.8B params · KRAFTON · MoE | 21.1 GB | 36.2 GB | 67.2 GB |
| granite-4.1-30b28.9B params · ibm-granite | 21.1 GB | 36.3 GB | 67.4 GB |
| granite-4.2-30b29.3B params · ibm-granite | 21.4 GB | 36.8 GB | 68.3 GB |
| ERNIE-4.5-VL-28B-A3B-PT29.4B params · baidu · MoE | 21.5 GB | 36.9 GB | 68.6 GB |
Quick answers
- How much VRAM do I need for a 24B–27B model?
- About 20.4 GB at Q4_K_M, 34.9 GB at Q8_0 and 64.9 GB at FP16, at a ~4k context. That is for Qwen3.8-27B (27.8B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
- How much VRAM does a 24B–27B model need for 32k context?
- About 22.2 GB at Q4_K_M with an FP16 KV cache, or 21.2 GB with a Q8 cache, based on Qwen3.8-27B's published config. Models with different attention layouts need more or less.
- What is the cheapest GPU for a 24B–27B model?
- Of the GPUs we track, the Intel Arc Pro B70 32GB (32 GB, $949 list price) is the cheapest that runs it fully on the GPU at Q4.
- Can a Mac run a 24B–27B model?
- Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M6 32GB (32 GB, about 24.0 GB usable, $1,799).
- Does a MoE model of the same size need less VRAM?
- No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.
Other sizes: 1B–2B · 2B–3B · 4B–6B · 7B–8B · 12B–14B · 16B–21B · 30B–32B · 40B–50B · 70B · 120B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant