# LLM context length and VRAM: KV cache by model (2026) > How much VRAM do 8k, 32k and 128k context need? KV cache per token for 535 open models, total memory at Q4, and the cheapest GPU and Mac per context length. Source: https://whatcanirunai.vercel.app/context-length/ # How much VRAM does context length need? Long chats, big codebases and long documents need memory for the KV cache, on top of the model weights. This page shows the cache size per token and at 8k, 32k and 128k tokens for 535 open models, total memory at Q4\_K\_M, and the cheapest current GPU and Mac or mini PC that fits popular models at 32k and 128k. Formula 2 × layers × KV heads × head size × 2 bytes per token, FP16 Q8\_0 cache \~53% of FP16 Q4\_0 cache \~28% of FP16 Models modeled 535 Short answer: an 8B model adds about 4.8 GB for 32k tokens; a 70B dense model adds 10.7 GB. Models that cache only some layers (sliding-window or hybrid attention) stay far leaner. Quantizing the cache to Q8\_0 roughly halves it. ## KV cache for popular models FP16 cache. Totals = Q4\_K\_M weights plus overhead ([our formula](https://whatcanirunai.vercel.app/method/#context)), which already covers \~4k tokens, plus the extra cache. Estimates from each model's published config.json. __KV cache size and total memory by context length for popular models__ | Model | Cache per token | Cache at 8k | Cache at 32k | Cache at 128k | Total Q4, 32k | Total Q4, 128k | | --------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ----------- | ------------ | ------------- | ------------- | -------------- | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/#context)8.2B params · 6.7 GB at Q4 with \~4k | 144 KiB | 1.2 GB | 4.8 GB | 19.3 GB | 10.9 GB | 25.4 GB | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/#context)8.0B params · 6.6 GB at Q4 with \~4k | 128 KiB | 1.1 GB | 4.3 GB | 17.2 GB | 10.4 GB | 23.2 GB | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/#context)14.8B params · 11.3 GB at Q4 with \~4k | 160 KiB | 1.3 GB | 5.4 GB | 21.5 GB | 16.0 GB | 32.1 GB | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/#context)20.9B params · 15.6 GB at Q4 with \~4k | 24 KiB\+ sliding window | 0.2 GB | 0.8 GB | 3.2 GB | 16.3 GB | 18.7 GB | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/#context)24.0B params · 17.7 GB at Q4 with \~4k | 160 KiB | 1.3 GB | 5.4 GB | 21.5 GB | 22.4 GB | 38.5 GB | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/#context)30.5B params · 22.3 GB at Q4 with \~4k | 96 KiB | 0.8 GB | 3.2 GB | 12.9 GB | 25.1 GB | 34.8 GB | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/#context)27.8B params · 20.4 GB at Q4 with \~4k | 64 KiB | 0.5 GB | 2.1 GB | 8.6 GB | 22.2 GB | 28.7 GB | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/#context)32.8B params · 23.8 GB at Q4 with \~4k | 256 KiB | 2.1 GB | 8.6 GB | 34.4 GB | 31.4 GB | 57.1 GB | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/#context)33.1B params · 24.1 GB at Q4 with \~4k | 208 KiB | 1.7 GB | 7.0 GB | 27.9 GB | 30.2 GB | 51.1 GB | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/#context)70.6B params · 50.2 GB at Q4 with \~4k | 320 KiB | 2.7 GB | 10.7 GB | 42.9 GB | 59.6 GB | 91.8 GB | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/#context)116.8B params · 82.5 GB at Q4 with \~4k | 36 KiB\+ sliding window | 0.3 GB | 1.2 GB | 4.8 GB | 83.5 GB | 87.1 GB | ## Cheapest hardware for 32k context The cheapest current device (at its dated price) that fits each model at Q4\_K\_M with 32k tokens and an FP16 cache, with headroom to spare. A Q8 cache can drop you a tier. __Cheapest GPU and Mac or mini PC for each model at 32k context__ | Model | Cheapest GPU | Cheapest Mac / mini PC | | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)needs \~10.9 GB at Q4, 32k | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)$329 · 16 GB usable | [Mac mini M6 24GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-24/)$1,299 · 18 GB usable | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)needs \~10.4 GB at Q4, 32k | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)$249 · 12 GB usable | [Mac mini M6 16GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-16/)$899 · 12 GB usable | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)needs \~16.0 GB at Q4, 32k | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 · 20 GB usable | [Mac mini M6 24GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-24/)$1,299 · 18 GB usable | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)needs \~16.3 GB at Q4, 32k | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)$899 · 20 GB usable | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 · 24 GB usable | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)needs \~22.4 GB at Q4, 32k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)needs \~25.1 GB at Q4, 32k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)needs \~22.2 GB at Q4, 32k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)needs \~31.4 GB at Q4, 32k | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 · 48 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/)needs \~30.2 GB at Q4, 32k | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 · 48 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)needs \~59.6 GB at Q4, 32k | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 · 96 GB usable | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 · 96 GB usable | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/)needs \~83.5 GB at Q4, 32k | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 · 96 GB usable | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 · 96 GB usable | ## Cheapest hardware for 128k context The cheapest current device (at its dated price) that fits each model at Q4\_K\_M with 128k tokens and an FP16 cache, with headroom to spare. A Q8 cache can drop you a tier. __Cheapest GPU and Mac or mini PC for each model at 128k context__ | Model | Cheapest GPU | Cheapest Mac / mini PC | | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)needs \~25.4 GB at Q4, 128k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)needs \~23.2 GB at Q4, 128k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)needs \~32.1 GB at Q4, 128k | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 · 48 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)needs \~18.7 GB at Q4, 128k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M6 32GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m6-32/)$1,799 · 24 GB usable | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)needs \~38.5 GB at Q4, 128k | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 · 48 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)needs \~34.8 GB at Q4, 128k | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)$6,799 · 48 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)needs \~28.7 GB at Q4, 128k | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)$949 · 32 GB usable | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)$3,199 · 48 GB usable | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)needs \~57.1 GB at Q4, 128k | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 · 96 GB usable | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 · 96 GB usable | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/)needs \~51.1 GB at Q4, 128k | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)$8,565 · 96 GB usable | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)$3,449 · 96 GB usable | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)needs \~91.8 GB at Q4, 128k | no single GPU we track; see [multi-GPU](https://whatcanirunai.vercel.app/multi-gpu-for-local-llm/) | [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-256/)$9,499 · 192 GB usable | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/)needs \~87.1 GB at Q4, 128k | no single GPU we track; see [multi-GPU](https://whatcanirunai.vercel.app/multi-gpu-for-local-llm/) | [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-256/)$9,499 · 192 GB usable | ## Every model, leanest cache first All 535 models with a modeled cache, sorted by cache size at 128k. Sliding-window layers count only their window; linear/Mamba layers count as fixed state. __KV cache size for every modeled model at 32k and 128k context__ | Model | Per token (FP16) | 32k (FP16) | 128k (FP16) | 128k (Q8 cache) | | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------- | ----------- | --------------- | | [NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16](https://whatcanirunai.vercel.app/models/nvidia-nemotron-3.5-lightning-30b-a3b-bf16/#context)31.6B params · 6 of 52 layers keep a growing cache | 6 KiB | 0.2 GB | 0.8 GB | 0.4 GB | | [Qwen2.5-0.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-0.5b-instruct/#context)0.5B params · all 24 layers cache | 12 KiB | 0.4 GB | 1.6 GB | 0.9 GB | | [Qwen3.5-0.8B](https://whatcanirunai.vercel.app/models/qwen3.5-0.8b/#context)0.9B params · 6 of 24 layers keep a growing cache | 12 KiB | 0.4 GB | 1.6 GB | 0.9 GB | | [LFM3-1.7B](https://whatcanirunai.vercel.app/models/lfm3-1.7b/#context)1.7B params · 6 of 24 layers keep a growing cache | 12 KiB | 0.4 GB | 1.6 GB | 0.9 GB | | [Muse-Glimmer-30B](https://whatcanirunai.vercel.app/models/muse-glimmer-30b/#context)29.8B params · 13 of 52 layers keep a growing cache | 13 KiB | 0.5 GB | 1.8 GB | 1.0 GB | | [Mellum2.1-12B-A2.5B-Thinking](https://whatcanirunai.vercel.app/models/mellum2.1-12b-a2.5b-thinking/#context)12.1B params · 7 of 28 layers keep a growing cache | 14 KiB | 0.5 GB | 1.9 GB | 1.0 GB | | [LFM2.5-24B-A2B](https://whatcanirunai.vercel.app/models/lfm2.5-24b-a2b/#context)24.0B params · 10 of 40 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Ornith-1.5-35B-A3B](https://whatcanirunai.vercel.app/models/ornith-1.5-35b-a3b/#context)36.0B params · 10 of 40 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Sage-A1.5](https://whatcanirunai.vercel.app/models/sage-a1.5/#context)36.0B params · 10 of 40 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Qwen3.6-35B-A3B](https://whatcanirunai.vercel.app/models/qwen3.6-35b-a3b/#context)36.0B params · 10 of 40 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Qwen3.5-35B-A3B](https://whatcanirunai.vercel.app/models/qwen3.5-35b-a3b/#context)36.0B params · 10 of 40 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Kolibri-1](https://whatcanirunai.vercel.app/models/kolibri-1/#context)78.1B params · 10 of 50 layers keep a growing cache | 20 KiB | 0.7 GB | 2.7 GB | 1.4 GB | | [Ornith-1.5-122B](https://whatcanirunai.vercel.app/models/ornith-1.5-122b/#context)125.1B params · 12 of 48 layers keep a growing cache | 24 KiB | 0.8 GB | 3.2 GB | 1.7 GB | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/#context)20.9B params · 12 of 24 layers keep a growing cache | 24 KiB | 0.8 GB | 3.2 GB | 1.7 GB | | [Qwen2.5-1.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-1.5b-instruct/#context)1.5B params · all 28 layers cache | 28 KiB | 0.9 GB | 3.8 GB | 2.0 GB | | [Recursive Flint](https://whatcanirunai.vercel.app/models/flint/#context)39.8B params · 14 of 49 layers keep a growing cache | 28 KiB | 1.0 GB | 3.8 GB | 2.0 GB | | [Llama-3.2-1B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-1b-instruct/#context)1.2B params · all 16 layers cache | 32 KiB | 1.1 GB | 4.3 GB | 2.3 GB | | [Ornith-1.5-9B](https://whatcanirunai.vercel.app/models/ornith-1.5-9b/#context)9.7B params · 8 of 32 layers keep a growing cache | 32 KiB | 1.1 GB | 4.3 GB | 2.3 GB | | [Qwen3.5-9B](https://whatcanirunai.vercel.app/models/qwen3.5-9b/#context)9.7B params · 8 of 32 layers keep a growing cache | 32 KiB | 1.1 GB | 4.3 GB | 2.3 GB | | [Qwen3.5-4B](https://whatcanirunai.vercel.app/models/qwen3.5-4b/#context)4.7B params · 8 of 32 layers keep a growing cache | 32 KiB | 1.1 GB | 4.3 GB | 2.3 GB | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/#context)116.8B params · 18 of 36 layers keep a growing cache | 36 KiB | 1.2 GB | 4.8 GB | 2.6 GB | | [Spark-X2.6-4B](https://whatcanirunai.vercel.app/models/spark-x2.6-4b/#context)4.1B params · 9 of 36 layers keep a growing cache | 36 KiB | 1.3 GB | 4.9 GB | 2.6 GB | | [Beam-1.0-501B-A23B](https://whatcanirunai.vercel.app/models/beam-1.0-501b-a23b/#context)501.3B params · 9 of 52 layers keep a growing cache | 36 KiB | 1.3 GB | 4.9 GB | 2.6 GB | | [MiniCPM5-2B](https://whatcanirunai.vercel.app/models/minicpm5-2b/#context)2.5B params · all 42 layers cache | 42 KiB | 1.4 GB | 5.6 GB | 3.0 GB | | [Qwen2.5-Coder-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-7b-instruct/#context)7.6B params · all 28 layers cache | 56 KiB | 1.9 GB | 7.5 GB | 4.0 GB | | [DeepSeek-R1-Distill-Qwen-7B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-7b/#context)7.6B params · all 28 layers cache | 56 KiB | 1.9 GB | 7.5 GB | 4.0 GB | | [Qwen2.5-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-7b-instruct/#context)7.6B params · all 28 layers cache | 56 KiB | 1.9 GB | 7.5 GB | 4.0 GB | | [Qwen2.5-VL-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-vl-7b-instruct/#context)8.3B params · all 28 layers cache | 56 KiB | 1.9 GB | 7.5 GB | 4.0 GB | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/#context)27.8B params · 16 of 64 layers keep a growing cache | 64 KiB | 2.1 GB | 8.6 GB | 4.6 GB | | [Tess-4.5-27B](https://whatcanirunai.vercel.app/models/tess-4.5-27b/#context)27.8B params · 16 of 64 layers keep a growing cache | 64 KiB | 2.1 GB | 8.6 GB | 4.6 GB | | [Qwen3.6-27B](https://whatcanirunai.vercel.app/models/qwen3.6-27b/#context)27.8B params · 16 of 64 layers keep a growing cache | 64 KiB | 2.1 GB | 8.6 GB | 4.6 GB | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/#context)30.5B params · all 48 layers cache | 96 KiB | 3.2 GB | 12.9 GB | 6.8 GB | | [Qwen3-Coder-30B-A3B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-coder-30b-a3b-instruct/#context)30.5B params · all 48 layers cache | 96 KiB | 3.2 GB | 12.9 GB | 6.8 GB | | [Hy-Memory-30B-A3B](https://whatcanirunai.vercel.app/models/hy-memory-30b-a3b/#context)30.1B params · all 48 layers cache | 96 KiB | 3.2 GB | 12.9 GB | 6.8 GB | | [Qwen3-0.6B](https://whatcanirunai.vercel.app/models/qwen3-0.6b/#context)0.8B params · all 28 layers cache | 112 KiB | 3.8 GB | 15.0 GB | 8.0 GB | | [Qwen3-1.7B](https://whatcanirunai.vercel.app/models/qwen3-1.7b/#context)2.0B params · all 28 layers cache | 112 KiB | 3.8 GB | 15.0 GB | 8.0 GB | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/#context)3.2B params · all 28 layers cache | 112 KiB | 3.8 GB | 15.0 GB | 8.0 GB | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/#context)8.0B params · all 32 layers cache | 128 KiB | 4.3 GB | 17.2 GB | 9.1 GB | | [Mistral-7B-Instruct-v0.3](https://whatcanirunai.vercel.app/models/mistral-7b-instruct-v0.3/#context)7.2B params · all 32 layers cache | 128 KiB | 4.3 GB | 17.2 GB | 9.1 GB | | [Phi-4-mini-instruct](https://whatcanirunai.vercel.app/models/phi-4-mini-instruct/#context)3.8B params · all 32 layers cache | 128 KiB | 4.3 GB | 17.2 GB | 9.1 GB | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/#context)4.0B params · all 36 layers cache | 144 KiB | 4.8 GB | 19.3 GB | 10.2 GB | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/#context)8.2B params · all 36 layers cache | 144 KiB | 4.8 GB | 19.3 GB | 10.2 GB | | [Qwen3-VL-8B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-vl-8b-instruct/#context)8.8B params · all 36 layers cache | 144 KiB | 4.8 GB | 19.3 GB | 10.2 GB | | [Qwen3-4B-Instruct-2507](https://whatcanirunai.vercel.app/models/qwen3-4b-instruct-2507/#context)4.0B params · all 36 layers cache | 144 KiB | 4.8 GB | 19.3 GB | 10.2 GB | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/#context)14.8B params · all 40 layers cache | 160 KiB | 5.4 GB | 21.5 GB | 11.4 GB | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/#context)12.2B params · all 40 layers cache | 160 KiB | 5.4 GB | 21.5 GB | 11.4 GB | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/#context)24.0B params · all 40 layers cache | 160 KiB | 5.4 GB | 21.5 GB | 11.4 GB | | [granite-3.3-8b-instruct](https://whatcanirunai.vercel.app/models/granite-3.3-8b-instruct/#context)8.2B params · all 40 layers cache | 160 KiB | 5.4 GB | 21.5 GB | 11.4 GB | | [Qwen2.5-Coder-14B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-14b-instruct/#context)14.8B params · all 48 layers cache | 192 KiB | 6.4 GB | 25.8 GB | 13.7 GB | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/#context)14.8B params · all 48 layers cache | 192 KiB | 6.4 GB | 25.8 GB | 13.7 GB | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/#context)14.7B params · all 40 layers cache | 200 KiB | 6.7 GB | 26.8 GB | 14.2 GB | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/#context)33.1B params · all 52 layers cache | 208 KiB | 7.0 GB | 27.9 GB | 14.8 GB | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/#context)32.8B params · all 64 layers cache | 256 KiB | 8.6 GB | 34.4 GB | 18.2 GB | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/#context)32.8B params · all 64 layers cache | 256 KiB | 8.6 GB | 34.4 GB | 18.2 GB | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/#context)32.8B params · all 64 layers cache | 256 KiB | 8.6 GB | 34.4 GB | 18.2 GB | | [Inkling](https://whatcanirunai.vercel.app/models/inkling/#context)952.4B params · all 66 layers cache | 264 KiB | 8.9 GB | 35.4 GB | 18.8 GB | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/#context)72.7B params · all 80 layers cache | 320 KiB | 10.7 GB | 42.9 GB | 22.8 GB | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/#context)70.6B params · all 80 layers cache | 320 KiB | 10.7 GB | 42.9 GB | 22.8 GB | | [DeepSeek-R1-Distill-Llama-70B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-llama-70b/#context)70.6B params · all 80 layers cache | 320 KiB | 10.7 GB | 42.9 GB | 22.8 GB | | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/#context)13.7B params · all 40 layers cache | 800 KiB | 26.8 GB | 107.4 GB | 56.9 GB | Showing the 60 most popular of 535 models with a modeled cache. [See all 535 in the model directory →](https://whatcanirunai.vercel.app/models/?kv=1) Try any memory size, context and cache type in the [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/?ctx=32768). Each model page has its own "Memory by context length" table. ## Quick answers How much VRAM does context length use? Every token in the context window stores a key and a value per attention layer (the KV cache). For Qwen3-8B that's about 144 KiB per token at FP16, so 32k tokens add \~4.8 GB and 128k add \~19.3 GB on top of the weights. Llama-3.3-70B-Instruct needs \~42.9 GB of cache at 128k. How much VRAM do I need for 32k context? It depends on the model. Qwen3-32B needs about 31.4 GB at Q4\_K\_M with an FP16 cache at 32k (vs 23.8 GB at \~4k); Qwen3.8-27B, which caches only 16 of 64 layers, needs 22.2 GB. See the table above for more models. Does a quantized KV cache help? Yes. llama.cpp and Ollama can store the cache at Q8\_0 (about 53% of FP16) or Q4\_0 (about 28%). A Q8 cache roughly halves the cache part with little quality loss; Q4 cache saves more but can hurt long-context accuracy. The weights' quantization (Q4\_K\_M etc.) is separate. Which models use the least memory at long context? Models with sliding-window or linear-attention layers only keep a growing cache on some layers. Of the 535 models we model, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 needs the least cache at 128k (\~0.8 GB) and OLMo-2-1124-13B-Instruct the most (\~107.4 GB). gpt-oss-20b uses a 128-token sliding window on half its layers, so its 128k cache is just \~3.2 GB. Is a model's maximum context the same as what I can run? No. The model card states the longest context it was trained for; whether you can use it depends on memory. We don't track each model's maximum, so check the model card before relying on long contexts. Why are some models missing? 120 models we track use attention layouts our formula doesn't model yet (for example multi-head latent attention, or Gemma 4's separate global layers), or their config is gated. Their pages say "cache not modeled". [How much VRAM do you need?](https://whatcanirunai.vercel.app/how-much-vram/) · [Q4 vs Q8 vs FP16](https://whatcanirunai.vercel.app/quantization/) · [Tokens per second](https://whatcanirunai.vercel.app/tokens-per-second/) · [Best GPU for local LLMs](https://whatcanirunai.vercel.app/best-gpu-for-local-llm/) · [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/