# GGUF VRAM requirements: Q4_K_M, Q5_K_M, Q6_K, Q8_0 sizes (2026) > How much VRAM each GGUF quant needs in llama.cpp and LM Studio: real Q3_K_M–Q8_0 file sizes for 26 models (Llama, Qwen3, Gemma, gpt-oss), est. memory and best quant for your GPU. Source: https://whatcanirunai.vercel.app/gguf-vram-requirements/ # GGUF VRAM requirements: how much memory each quant needs How much VRAM a GGUF model needs in llama.cpp and LM Studio, quant by quant. For 26 popular models we list the real file size of each common K-quant (Q3\_K\_M, Q4\_K\_M, Q5\_K\_M, Q6\_K and Q8\_0), estimate the total memory at \~4k and 32k context, and show the best quant for your VRAM. Models 26 from Llama, Qwen, Gemma, Mistral, Phi, DeepSeek, gpt-oss Quants 5 Q3\_K\_M to Q8\_0 File sizes 2026-11-14 from Hugging Face Short answer: a GGUF needs about its file size plus 15% plus 1 GB at a \~4k context. At Q4\_K\_M that's \~6.7 GB for an 8B model, \~11.3 GB for a 14B, \~23.7 GB for a 32B and \~49.9 GB for a 70B. Q5\_K\_M adds about 16%, Q6\_K about 34% and Q8\_0 about 71%. ## How we estimate it memory ≈ GGUF file size × 1.15 + 1 GB (≈4k context) · 32k adds the KV cache from the model's published config (f16 cache) Same formula as the rest of the site ([how we calculate](https://whatcanirunai.vercel.app/method/)), applied to real GGUF files instead of a bits-per-weight guess. These are estimates, not measurements. A model "fits" when it uses 90% or less of the memory. File sizes are from [bartowski's GGUF repos](https://huggingface.co/bartowski) on Hugging Face (split files added together), retrieved 2026-11-14\. Other uploaders' files differ by a few percent. ## GGUF memory by quant Bold: estimated memory at \~4k context. Small: GGUF file size. Last column: Q4\_K\_M at 32k context. __Estimated memory and GGUF file size for each model at Q3\_K\_M, Q4\_K\_M, Q5\_K\_M, Q6\_K and Q8\_0, plus Q4\_K\_M at 32k context__ | Model | Q3\_K\_M | Q4\_K\_M | Q5\_K\_M | Q6\_K | Q8\_0 | Q4\_K\_M, 32k ctx | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ----------------------- | ----------------------- | ----------------------- | ----------------------- | ----------------- | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)3.2B params · [GGUF files](https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF) | — | **3.3 GB**2.0 GB file | **3.7 GB**2.3 GB file | **4.0 GB**2.6 GB file | **4.9 GB**3.4 GB file | 6.6 GB | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)4.0B params · [GGUF files](https://huggingface.co/bartowski/Qwen%5FQwen3-4B-GGUF) | **3.4 GB**2.1 GB file | **3.9 GB**2.5 GB file | **4.3 GB**2.9 GB file | **4.8 GB**3.3 GB file | **5.9 GB**4.3 GB file | 8.1 GB | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)4.3B params · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-3-4b-it-GGUF) | **3.4 GB**2.1 GB file | **3.9 GB**2.5 GB file | **4.3 GB**2.8 GB file | **4.7 GB**3.2 GB file | **5.7 GB**4.1 GB file | — | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/)8.0B params · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-4-E4B-it-GGUF) | **6.6 GB**4.9 GB file | **7.2 GB**5.4 GB file | **7.7 GB**5.8 GB file | **8.3 GB**6.3 GB file | **10.2 GB**8.0 GB file | — | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)8.0B params · [GGUF files](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF) | **5.6 GB**4.0 GB file | **6.7 GB**4.9 GB file | **7.6 GB**5.7 GB file | **8.6 GB**6.6 GB file | **10.8 GB**8.5 GB file | 10.4 GB | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)8.2B params · [GGUF files](https://huggingface.co/bartowski/Qwen%5FQwen3-8B-GGUF) | **5.7 GB**4.1 GB file | **6.8 GB**5.0 GB file | **7.7 GB**5.8 GB file | **8.7 GB**6.7 GB file | **11.0 GB**8.7 GB file | 11.0 GB | | [DeepSeek-R1-0528-Qwen3-8B](https://whatcanirunai.vercel.app/models/deepseek-r1-0528-qwen3-8b/)8.2B params · [GGUF files](https://huggingface.co/bartowski/deepseek-ai%5FDeepSeek-R1-0528-Qwen3-8B-GGUF) | **5.7 GB**4.1 GB file | **6.8 GB**5.0 GB file | **7.7 GB**5.8 GB file | **8.7 GB**6.7 GB file | **11.0 GB**8.7 GB file | 11.0 GB | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/)12.2B params · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-3-12b-it-GGUF) | **7.9 GB**6.0 GB file | **9.4 GB**7.3 GB file | **10.7 GB**8.4 GB file | **12.1 GB**9.7 GB file | **15.4 GB**12.5 GB file | — | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/)12.0B params · [GGUF files](https://huggingface.co/bartowski/gemma-4-12B-it-GGUF) | **8.2 GB**6.3 GB file | **9.8 GB**7.7 GB file | **11.1 GB**8.8 GB file | **12.8 GB**10.2 GB file | **16.1 GB**13.1 GB file | — | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)12.2B params · [GGUF files](https://huggingface.co/bartowski/Mistral-Nemo-Instruct-2407-GGUF) | **8.0 GB**6.1 GB file | **9.6 GB**7.5 GB file | **11.0 GB**8.7 GB file | **12.6 GB**10.1 GB file | **16.0 GB**13.0 GB file | 14.3 GB | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)14.8B params · [GGUF files](https://huggingface.co/bartowski/Qwen%5FQwen3-14B-GGUF) | **9.4 GB**7.3 GB file | **11.3 GB**9.0 GB file | **13.1 GB**10.5 GB file | **14.9 GB**12.1 GB file | **19.1 GB**15.7 GB file | 16.0 GB | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/)14.7B params · [GGUF files](https://huggingface.co/bartowski/phi-4-GGUF) | **9.5 GB**7.4 GB file | **11.4 GB**9.1 GB file | **13.2 GB**10.6 GB file | **14.8 GB**12.0 GB file | **18.9 GB**15.6 GB file | 17.3 GB | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/)14.8B params · [GGUF files](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF) | **9.4 GB**7.3 GB file | **11.3 GB**9.0 GB file | **13.1 GB**10.5 GB file | **14.9 GB**12.1 GB file | **19.1 GB**15.7 GB file | 17.0 GB | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)20.9B params · MoE · [GGUF files](https://huggingface.co/bartowski/openai%5Fgpt-oss-20b-GGUF) | **14.3 GB**11.6 GB file | **14.4 GB**11.7 GB file | **14.5 GB**11.7 GB file | **14.8 GB**12.0 GB file | **14.9 GB**12.1 GB file | 15.1 GB | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)24.0B params · [GGUF files](https://huggingface.co/bartowski/mistralai%5FMistral-Small-3.2-24B-Instruct-2506-GGUF) | **14.2 GB**11.5 GB file | **17.5 GB**14.3 GB file | **20.3 GB**16.8 GB file | **23.3 GB**19.4 GB file | **29.8 GB**25.1 GB file | 22.2 GB | | [Gemma 4 26B A4B](https://whatcanirunai.vercel.app/models/gemma-4-26b-a4b-it/)25.8B params · MoE · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-4-26B-A4B-it-GGUF) | **16.0 GB**13.0 GB file | **20.6 GB**17.0 GB file | **23.2 GB**19.3 GB file | **27.3 GB**22.9 GB file | **32.4 GB**27.3 GB file | — | | [gemma-3-27b-it](https://whatcanirunai.vercel.app/models/gemma-3-27b-it/)27.4B params · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-3-27b-it-GGUF) | **16.5 GB**13.4 GB file | **20.0 GB**16.6 GB file | **23.2 GB**19.3 GB file | **26.5 GB**22.2 GB file | **34.0 GB**28.7 GB file | — | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)27.8B params · [GGUF files](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF) | **16.4 GB**13.4 GB file | **21.1 GB**17.4 GB file | **25.1 GB**20.9 GB file | **28.4 GB**23.9 GB file | **34.5 GB**29.1 GB file | 22.9 GB | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)30.5B params · MoE · [GGUF files](https://huggingface.co/bartowski/Qwen%5FQwen3-30B-A3B-GGUF) | **17.2 GB**14.1 GB file | **22.4 GB**18.6 GB file | **26.0 GB**21.7 GB file | **29.9 GB**25.1 GB file | **38.4 GB**32.5 GB file | 25.2 GB | | [Gemma 4 31B](https://whatcanirunai.vercel.app/models/gemma-4-31b-it/)31.3B params · [GGUF files](https://huggingface.co/bartowski/google%5Fgemma-4-31B-it-GGUF) | **19.3 GB**15.9 GB file | **23.5 GB**19.6 GB file | **27.0 GB**22.6 GB file | **31.7 GB**26.7 GB file | **39.1 GB**33.1 GB file | — | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)32.8B params · [GGUF files](https://huggingface.co/bartowski/Qwen%5FQwen3-32B-GGUF) | **19.4 GB**16.0 GB file | **23.7 GB**19.8 GB file | **27.7 GB**23.2 GB file | **31.9 GB**26.9 GB file | **41.0 GB**34.8 GB file | 31.2 GB | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/)32.8B params · [GGUF files](https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF) | **19.3 GB**15.9 GB file | **23.8 GB**19.9 GB file | **27.7 GB**23.3 GB file | **31.9 GB**26.9 GB file | **41.0 GB**34.8 GB file | 31.3 GB | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/)32.8B params · [GGUF files](https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF) | **19.3 GB**15.9 GB file | **23.8 GB**19.9 GB file | **27.7 GB**23.3 GB file | **31.9 GB**26.9 GB file | **41.0 GB**34.8 GB file | 31.3 GB | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)70.6B params · [GGUF files](https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF) | **40.4 GB**34.3 GB file | **49.9 GB**42.5 GB file | **58.4 GB**50.0 GB file | **67.6 GB**57.9 GB file | **87.2 GB**75.0 GB file | 59.3 GB | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/)72.7B params · [GGUF files](https://huggingface.co/bartowski/Qwen2.5-72B-Instruct-GGUF) | **44.4 GB**37.7 GB file | **55.5 GB**47.4 GB file | **63.6 GB**54.5 GB file | **75.0 GB**64.3 GB file | **89.8 GB**77.3 GB file | 64.9 GB | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/)116.8B params · MoE · [GGUF files](https://huggingface.co/bartowski/openai%5Fgpt-oss-120b-GGUF) | **73.1 GB**62.7 GB file | **73.3 GB**62.8 GB file | — | **73.8 GB**63.3 GB file | **73.9 GB**63.4 GB file | 74.3 GB | "—" means that quant isn't in the repo, or (32k column) we don't have the model's attention config yet. Gemma and some Qwen and Mistral models are multimodal; the vision projector (mmproj) is a separate file that adds memory when you send images. ## Best GGUF quant for your VRAM The highest of these quants that fits fully on the GPU with \~4k context (90% rule). Unified-memory Macs: use the usable memory, not the total (a 32 GB Mac has about 24 GB usable for the GPU by default). __Highest GGUF quant that fits each VRAM size__ | Model | 8 GB | 12 GB | 16 GB | 24 GB | 32 GB | 48 GB | 64 GB | 96 GB | | ----------------------------------------------------------------------------------- | --------- | --------- | --------- | --------- | --------- | --------- | --------- | --------- | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/) | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/) | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/) | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/) | Q3\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/) | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/) | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [DeepSeek-R1-0528-Qwen3-8B](https://whatcanirunai.vercel.app/models/deepseek-r1-0528-qwen3-8b/) | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/) | — | Q5\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/) | — | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/) | — | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/) | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/) | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/) | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/) | — | — | Q3\_K\_M | Q5\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Gemma 4 26B A4B](https://whatcanirunai.vercel.app/models/gemma-4-26b-a4b-it/) | — | — | — | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [gemma-3-27b-it](https://whatcanirunai.vercel.app/models/gemma-3-27b-it/) | — | — | — | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/) | — | — | — | Q4\_K\_M | Q6\_K | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/) | — | — | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Gemma 4 31B](https://whatcanirunai.vercel.app/models/gemma-4-31b-it/) | — | — | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/) | — | — | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/) | — | — | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/) | — | — | — | Q3\_K\_M | Q5\_K\_M | **Q8\_0** | **Q8\_0** | **Q8\_0** | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/) | — | — | — | — | — | Q3\_K\_M | Q4\_K\_M | Q6\_K | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/) | — | — | — | — | — | — | Q4\_K\_M | Q6\_K | ### 8 GB VRAM Largest that fits: Qwen3-8B at Q4\_K\_M (\~6.8 GB), DeepSeek-R1-0528-Qwen3-8B at Q4\_K\_M (\~6.8 GB), Llama-3.1-8B-Instruct at Q4\_K\_M (\~6.7 GB). See [best LLMs for 8 GB VRAM](https://whatcanirunai.vercel.app/best-llm-for/8gb-vram/). ### 12 GB VRAM Largest that fits: DeepSeek-R1-Distill-Qwen-14B at Q3\_K\_M (\~9.4 GB), Qwen3-14B at Q3\_K\_M (\~9.4 GB), phi-4 at Q3\_K\_M (\~9.5 GB). See [best LLMs for 12 GB VRAM](https://whatcanirunai.vercel.app/best-llm-for/12gb-vram/). ### 16 GB VRAM Largest that fits: Mistral-Small-3.2-24B-Instruct-2506 at Q3\_K\_M (\~14.2 GB), DeepSeek-R1-Distill-Qwen-14B at Q5\_K\_M (\~13.1 GB), Qwen3-14B at Q5\_K\_M (\~13.1 GB). See [best LLMs for 16 GB VRAM](https://whatcanirunai.vercel.app/best-llm-for/16gb-vram/). ### 24 GB VRAM Largest that fits: DeepSeek-R1-Distill-Qwen-32B at Q3\_K\_M (\~19.3 GB), Qwen2.5-Coder-32B-Instruct at Q3\_K\_M (\~19.3 GB), Qwen3-32B at Q3\_K\_M (\~19.4 GB). See [best LLMs for 24 GB VRAM](https://whatcanirunai.vercel.app/best-llm-for/24gb-vram/). ### 32 GB VRAM Largest that fits: DeepSeek-R1-Distill-Qwen-32B at Q5\_K\_M (\~27.7 GB), Qwen2.5-Coder-32B-Instruct at Q5\_K\_M (\~27.7 GB), Qwen3-32B at Q5\_K\_M (\~27.7 GB). See [best LLMs for 32 GB VRAM](https://whatcanirunai.vercel.app/best-llm-for/32gb-vram/). ## Cheapest hardware for each GGUF Cheapest current device we track that fits at \~4k context. GPUs at dated list prices (MSRP); street prices may be higher. __Cheapest GPU at Q4\_K\_M and Q6\_K and cheapest Mac or mini PC at Q4\_K\_M__ | Model | Cheapest GPU, Q4\_K\_M | Cheapest GPU, Q6\_K | Cheapest Mac / mini PC, Q4\_K\_M | | ----------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)Q4\_K\_M \~3.3 GB · Q6\_K \~4.0 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)Q4\_K\_M \~3.9 GB · Q6\_K \~4.8 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)Q4\_K\_M \~3.9 GB · Q6\_K \~4.7 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/)Q4\_K\_M \~7.2 GB · Q6\_K \~8.3 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)Q4\_K\_M \~6.7 GB · Q6\_K \~8.6 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)Q4\_K\_M \~6.8 GB · Q6\_K \~8.7 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [DeepSeek-R1-0528-Qwen3-8B](https://whatcanirunai.vercel.app/models/deepseek-r1-0528-qwen3-8b/)Q4\_K\_M \~6.8 GB · Q6\_K \~8.7 GB | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)$219 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/)Q4\_K\_M \~9.4 GB · Q6\_K \~12.1 GB | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)$249 list | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/)Q4\_K\_M \~9.8 GB · Q6\_K \~12.8 GB | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)$249 list | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)Q4\_K\_M \~9.6 GB · Q6\_K \~12.6 GB | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)$249 list | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 list | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)Q4\_K\_M \~11.3 GB · Q6\_K \~14.9 GB | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 list | [Mac mini M6 24GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-24/)$1,299 list | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/)Q4\_K\_M \~11.4 GB · Q6\_K \~14.8 GB | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 list | [Mac mini M6 24GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-24/)$1,299 list | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/)Q4\_K\_M \~11.3 GB · Q6\_K \~14.9 GB | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 list | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 list | [Mac mini M6 24GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-24/)$1,299 list | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)Q4\_K\_M \~17.5 GB · Q6\_K \~23.3 GB | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 list | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 list | | [Gemma 4 26B A4B](https://whatcanirunai.vercel.app/models/gemma-4-26b-a4b-it/)Q4\_K\_M \~20.6 GB · Q6\_K \~27.3 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 list | | [gemma-3-27b-it](https://whatcanirunai.vercel.app/models/gemma-3-27b-it/)Q4\_K\_M \~20.0 GB · Q6\_K \~26.5 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 list | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)Q4\_K\_M \~21.1 GB · Q6\_K \~28.4 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 list | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)Q4\_K\_M \~22.4 GB · Q6\_K \~29.9 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 list | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 list | | [Gemma 4 31B](https://whatcanirunai.vercel.app/models/gemma-4-31b-it/)Q4\_K\_M \~23.5 GB · Q6\_K \~31.7 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 list | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 list | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)Q4\_K\_M \~23.7 GB · Q6\_K \~31.9 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 list | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 list | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/)Q4\_K\_M \~23.8 GB · Q6\_K \~31.9 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 list | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 list | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/)Q4\_K\_M \~23.8 GB · Q6\_K \~31.9 GB | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 list | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 list | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 list | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)Q4\_K\_M \~49.9 GB · Q6\_K \~67.6 GB | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 list | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 list | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 list | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/)Q4\_K\_M \~55.5 GB · Q6\_K \~75.0 GB | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 list | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 list | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 list | ## llama.cpp settings that change memory From the [llama-server docs](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md): * `-ngl all` puts every layer on the GPU. The default, `auto`, offloads what fits. * `-c 8192` sets the context. The default, 0, uses the model's full trained context, and `--fit` (on by default) then shrinks it to fit your free memory. * `-ctk q8_0 -ctv q8_0` stores the KV cache at 8-bit, about half the default f16\. See [context length and VRAM](https://whatcanirunai.vercel.app/context-length/). * `-fa on` forces Flash Attention (default `auto`). * `--n-cpu-moe N` keeps the expert weights of the first N layers on the CPU, a good way to run a MoE model like Qwen3-30B-A3B on a smaller GPU. llama-server -m Qwen\_Qwen3-8B-Q4\_K\_M.gguf -ngl all -c 8192 -ctk q8\_0 -ctv q8\_0 LM Studio loads the same GGUF files with a llama.cpp engine, so the numbers above apply. Set GPU offload to the maximum and pick a context length when you load the model. Using Ollama instead? See [Ollama VRAM requirements](https://whatcanirunai.vercel.app/ollama-vram-requirements/) for each default tag. ## Quick answers How much VRAM does a Q4\_K\_M GGUF need? About the GGUF file size plus 15%, plus about 1 GB for the runtime at a \~4k context. By that estimate Llama-3.1-8B-Instruct needs \~6.7 GB, Qwen3-14B needs \~11.3 GB, Qwen3-32B needs \~23.7 GB, Llama-3.3-70B-Instruct needs \~49.9 GB at Q4\_K\_M. Longer context adds KV cache on top. Q4\_K\_M vs Q5\_K\_M vs Q6\_K vs Q8\_0: how much bigger is each? Across 23 models in our table (gpt-oss excluded), Q5\_K\_M files are about 16% bigger than Q4\_K\_M, Q6\_K about 34% bigger and Q8\_0 about 71% bigger. Q3\_K\_M is about 19% smaller. VRAM needs grow by about the same share. Which GGUF quant should I download? Pick the biggest quant that still leaves about 10% of your VRAM free for context. Q4\_K\_M is the usual default. Bartowski's model cards rate Q5\_K\_M "high quality" and Q6\_K "very high quality, near perfect", and call Q8\_0 "generally unneeded". Q3\_K\_M is a lower-quality fallback when nothing else fits. The "best quant for your VRAM" table above does this for you. Can I run a 14B GGUF on 12 GB of VRAM? At Q4\_K\_M, just barely: Qwen3-14B needs about 11.3 GB, which is tight on a 12 GB card at \~4k context. Q5\_K\_M (\~13.1 GB) is too big, and Q6\_K (\~14.9 GB) needs a 16 GB card. Which quant of a 32B model fits a 24 GB GPU? Qwen3-32B needs about 23.7 GB at Q4\_K\_M, so Q4\_K\_M is tight on 24 GB at \~4k context. Q5\_K\_M needs \~27.7 GB (too big), and Q6\_K (\~31.9 GB) needs 32 GB or more. Why are all gpt-oss GGUF files about the same size? gpt-oss ships with its expert weights in MXFP4, a 4-bit format, and the GGUF repos keep them that way. So Q3\_K\_M to Q8\_0 only change the small non-expert part: gpt-oss-20b files run from 11.6 to 12.1 GB (about 14.3–14.9 GB of memory by our estimate), and gpt-oss-120b about 63 GB. Pick the largest one. What happens in llama.cpp if the GGUF is bigger than my VRAM? llama.cpp keeps the layers that don't fit in system RAM and runs them on the CPU. It still works, but generation slows down a lot. By default llama-server also fits the context to your free memory (the --fit option). For MoE models, --n-cpu-moe N moves only the expert weights of N layers to the CPU, which costs less speed than moving whole layers. How do I cut memory use in llama.cpp or LM Studio? Use a smaller quant, set a shorter context (-c 8192), and quantize the KV cache with -ctk q8\_0 -ctv q8\_0, which halves the cache memory of the default f16\. In LM Studio the same settings are the context length, GPU offload and KV cache quantization options when you load a model. Does LM Studio need the same VRAM as llama.cpp? About the same. LM Studio runs GGUF files with a llama.cpp engine, so the file sizes and estimates here apply. [How much VRAM by model size](https://whatcanirunai.vercel.app/how-much-vram/) · [Q4 vs Q8 vs FP16](https://whatcanirunai.vercel.app/quantization/) · [Ollama VRAM requirements](https://whatcanirunai.vercel.app/ollama-vram-requirements/) · [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/) · [Context length and VRAM](https://whatcanirunai.vercel.app/context-length/) · [MoE vs dense](https://whatcanirunai.vercel.app/moe-vs-dense/) · [Best GPU for local LLMs](https://whatcanirunai.vercel.app/best-gpu-for-local-llm/) · [Best LLMs by VRAM size](https://whatcanirunai.vercel.app/best-llm-for/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/