GGUF VRAM requirements: how much memory each quant needs

How much VRAM a GGUF model needs in llama.cpp and LM Studio, quant by quant. For 26 popular models we list the real file size of each common K-quant (Q3_K_M, Q4_K_M, Q5_K_M, Q6_K and Q8_0), estimate the total memory at ~4k and 32k context, and show the best quant for your VRAM.

Models

26 from Llama, Qwen, Gemma, Mistral, Phi, DeepSeek, gpt-oss

Quants

5 Q3_K_M to Q8_0

File sizes

2026-11-14 from Hugging Face

Short answer: a GGUF needs about its file size plus 15% plus 1 GB at a ~4k context. At Q4_K_M that's ~6.7 GB for an 8B model, ~11.3 GB for a 14B, ~23.7 GB for a 32B and ~49.9 GB for a 70B. Q5_K_M adds about 16%, Q6_K about 34% and Q8_0 about 71%.

How we estimate it

Same formula as the rest of the site (how we calculate), applied to real GGUF files instead of a bits-per-weight guess. These are estimates, not measurements. A model "fits" when it uses 90% or less of the memory. File sizes are from bartowski's GGUF repos on Hugging Face (split files added together), retrieved 2026-11-14. Other uploaders' files differ by a few percent.

GGUF memory by quant

Bold: estimated memory at ~4k context. Small: GGUF file size. Last column: Q4_K_M at 32k context.

"—" means that quant isn't in the repo, or (32k column) we don't have the model's attention config yet. Gemma and some Qwen and Mistral models are multimodal; the vision projector (mmproj) is a separate file that adds memory when you send images.

Best GGUF quant for your VRAM

The highest of these quants that fits fully on the GPU with ~4k context (90% rule). Unified-memory Macs: use the usable memory, not the total (a 32 GB Mac has about 24 GB usable for the GPU by default).

8 GB VRAM

Largest that fits: Qwen3-8B at Q4_K_M (~6.8 GB), DeepSeek-R1-0528-Qwen3-8B at Q4_K_M (~6.8 GB), Llama-3.1-8B-Instruct at Q4_K_M (~6.7 GB). See best LLMs for 8 GB VRAM.

12 GB VRAM

Largest that fits: DeepSeek-R1-Distill-Qwen-14B at Q3_K_M (~9.4 GB), Qwen3-14B at Q3_K_M (~9.4 GB), phi-4 at Q3_K_M (~9.5 GB). See best LLMs for 12 GB VRAM.

16 GB VRAM

Largest that fits: Mistral-Small-3.2-24B-Instruct-2506 at Q3_K_M (~14.2 GB), DeepSeek-R1-Distill-Qwen-14B at Q5_K_M (~13.1 GB), Qwen3-14B at Q5_K_M (~13.1 GB). See best LLMs for 16 GB VRAM.

24 GB VRAM

Largest that fits: DeepSeek-R1-Distill-Qwen-32B at Q3_K_M (~19.3 GB), Qwen2.5-Coder-32B-Instruct at Q3_K_M (~19.3 GB), Qwen3-32B at Q3_K_M (~19.4 GB). See best LLMs for 24 GB VRAM.

32 GB VRAM

Largest that fits: DeepSeek-R1-Distill-Qwen-32B at Q5_K_M (~27.7 GB), Qwen2.5-Coder-32B-Instruct at Q5_K_M (~27.7 GB), Qwen3-32B at Q5_K_M (~27.7 GB). See best LLMs for 32 GB VRAM.

Cheapest hardware for each GGUF

Cheapest current device we track that fits at ~4k context. GPUs at dated list prices (MSRP); street prices may be higher.

llama.cpp settings that change memory

From the llama-server docs:

• -ngl all puts every layer on the GPU. The default, auto, offloads what fits.
• -c 8192 sets the context. The default, 0, uses the model's full trained context, and --fit (on by default) then shrinks it to fit your free memory.
• -ctk q8_0 -ctv q8_0 stores the KV cache at 8-bit, about half the default f16. See context length and VRAM.
• -fa on forces Flash Attention (default auto).
• --n-cpu-moe N keeps the expert weights of the first N layers on the CPU, a good way to run a MoE model like Qwen3-30B-A3B on a smaller GPU.

LM Studio loads the same GGUF files with a llama.cpp engine, so the numbers above apply. Set GPU offload to the maximum and pick a context length when you load the model. Using Ollama instead? See Ollama VRAM requirements for each default tag.

How much VRAM by model size · Q4 vs Q8 vs FP16 · Ollama VRAM requirements · VRAM calculator · Context length and VRAM · MoE vs dense · Best GPU for local LLMs · Best LLMs by VRAM size