# How much VRAM do you need for a 40B–50B model? > A 40B–50B LLM needs about 35.8 GB of VRAM at Q4, 61.9 GB at Q8 and 115.7 GB at FP16. Context-length table and the cheapest GPU and Mac that fit. Source: https://whatcanirunai.vercel.app/how-much-vram/49b/ # How much VRAM do you need for a 40B–50B model? Short answer: about **35.8 GB** at Q4\_K\_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 40B–50B open model we track. Q4\_K\_M 35.8 GB 30.2 GB weights Q8\_0 61.9 GB 53.0 GB weights FP16 115.7 GB 99.7 GB weights Worked out for [Llama-3\_3-Nemotron-Super-49B-v1\_5](https://whatcanirunai.vercel.app/models/llama-3-3-nemotron-super-49b-v1-5/) (49.9B parameters, from the Hugging Face API). Formula: weights × 1.15 + 1 GB at a \~4k context ([how we calculate](https://whatcanirunai.vercel.app/method/)). Cheapest GPU that fits at Q4: **NVIDIA RTX A6000** (48 GB, $4,649). Cheapest Mac or mini PC: **Mac mini M5 Pro 64GB** ($3,199). ## Cheapest hardware for each quant Only devices with a sourced price. GPUs at list price (MSRP); street prices may be higher. __Cheapest GPU and Mac or mini PC for a 40B–50B model at each quant__ | Setting | Memory needed | Cheapest GPU | Cheapest Mac / mini PC | | ---------------------- | ------------- | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | | Q4\_K\_M, \~4k context | 35.8 GB | [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/) 48 GB usable · $4,649 | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/) 48 GB usable · $3,199 | | Q8\_0, \~4k context | 61.9 GB | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/) 96 GB usable · $8,565 | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/) 96 GB usable · $3,449 | | FP16, \~4k context | 115.7 GB | None we track | [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-256/) 192 GB usable · $9,499 | | Q4\_K\_M, 32k context | 110.9 GB | None we track | [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-256/) 192 GB usable · $9,499 | ## Memory by context length Each extra token of context adds to the KV cache. Numbers for Llama-3\_3-Nemotron-Super-49B-v1\_5, from its published config; a Q8 KV cache halves the cache part. __Memory needed for a 40B–50B model by context length__ | Context | Q4\_K\_M, FP16 cache | Q4\_K\_M, Q8 cache | Q8\_0, FP16 cache | | ----------- | -------------------- | ------------------ | ----------------- | | 4k tokens | 35.8 GB | 35.8 GB | 61.9 GB | | 8k tokens | 46.5 GB | 35.8 GB | 72.7 GB | | 16k tokens | 68.0 GB | 46.5 GB | 94.1 GB | | 32k tokens | 110.9 GB | 68.0 GB | 137.1 GB | | 64k tokens | 196.8 GB | 110.9 GB | 223.0 GB | | 128k tokens | 368.6 GB | 196.8 GB | 394.8 GB | ## Hardware that runs it Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 ([how we estimate](https://whatcanirunai.vercel.app/method/#speed)).FitsTight __Tracked hardware that runs a 40B–50B model at Q4__ | Hardware | Q4 need vs. usable | Q4 / Q8 / FP16 | Est. tok/s | | ------------------------------------------------------------------------------------------------------ | ------------------- | ------------------------------ | ---------- | | [Mac mini M5 Pro 64GB](https://whatcanirunai.vercel.app/hardware/mac-mini-m5-pro-64/)unified memory · $3,199 | **35.8** / 48.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 5–8 | | [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)GPU · $4,649 | **35.8** / 48.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 15–22 | | [NVIDIA DGX Spark 64GB](https://whatcanirunai.vercel.app/hardware/dgx-spark-64/)unified memory · $4,999 | **35.8** / 48.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 4–7 | | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)GPU · $6,799 | **35.8** / 48.0 GB | Q4FitsQ8Won’t fitFP16Won’t fit | 19–27 | | [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/hardware/framework-desktop-128/)unified memory · $3,449 | **35.8** / 96.0 GB | Q4FitsQ8FitsFP16Won’t fit | 4–7 | | [MacBook Pro M4 Max 128GB](https://whatcanirunai.vercel.app/hardware/macbook-pro-m4-max-128/)unified memory · $4,999 | **35.8** / 96.0 GB | Q4FitsQ8FitsFP16Won’t fit | 8–14 | | [MacBook Pro M5 Max 128GB](https://whatcanirunai.vercel.app/hardware/macbook-pro-m5-max-128/)unified memory · $6,699 | **35.8** / 96.0 GB | Q4FitsQ8FitsFP16Won’t fit | 9–16 | | [NVIDIA DGX Spark 128GB](https://whatcanirunai.vercel.app/hardware/dgx-spark-128/)unified memory · $6,950 | **35.8** / 96.0 GB | Q4FitsQ8FitsFP16Won’t fit | 4–7 | | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)GPU · $8,565 | **35.8** / 96.0 GB | Q4FitsQ8FitsFP16Won’t fit | 36–50 | | [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-256/)unified memory · $9,499 | **35.8** / 192.0 GB | Q4FitsQ8FitsFP16Fits | 18–32 | | [Mac Studio M5 Ultra 512GB](https://whatcanirunai.vercel.app/hardware/mac-studio-m5-ultra-512/)unified memory · $22,299 | **35.8** / 384.0 GB | Q4FitsQ8FitsFP16Fits | 18–32 | ## 40B–50B models we track __Memory needed by each 40B–50B model we track__ | Model | Q4\_K\_M | Q8\_0 | FP16 | | ------------------------------------------------------------------------------------------------------- | -------- | ------- | -------- | | [K2-Horizon-MoVA-36B-A4B](https://whatcanirunai.vercel.app/models/k2-horizon-mova-36b-a4b/)37.4B params · IFM · MoE | 27.1 GB | 46.8 GB | 87.1 GB | | [Recursive Flint](https://whatcanirunai.vercel.app/models/flint/)39.8B params · RecursiveSI · MoE | 28.7 GB | 49.6 GB | 92.5 GB | | [IQuest-Coder-V1-40B-Instruct](https://whatcanirunai.vercel.app/models/iquest-coder-v1-40b-instruct/)39.8B params · IQuestLab | 28.7 GB | 49.6 GB | 92.5 GB | | [IQuest-Coder-V1-40B-Loop-Instruct](https://whatcanirunai.vercel.app/models/iquest-coder-v1-40b-loop-instruct/)39.8B params · IQuestLab | 28.7 GB | 49.6 GB | 92.5 GB | | [ALIA-40b](https://whatcanirunai.vercel.app/models/alia-40b/)40.4B params · BSC-LT | 29.2 GB | 50.4 GB | 94.0 GB | | [Phi-3.5-MoE-instruct](https://whatcanirunai.vercel.app/models/phi-3.5-moe-instruct/)41.9B params · microsoft · MoE | 30.2 GB | 52.2 GB | 97.3 GB | | [GRIN-MoE](https://whatcanirunai.vercel.app/models/grin-moe/)41.9B params · microsoft · MoE | 30.2 GB | 52.2 GB | 97.3 GB | | [Llama-3\_3-Nemotron-Super-49B-v1\_5](https://whatcanirunai.vercel.app/models/llama-3-3-nemotron-super-49b-v1-5/)49.9B params · nvidia | 35.8 GB | 61.9 GB | 115.7 GB | | [Llama-3\_3-Nemotron-Super-49B-v1](https://whatcanirunai.vercel.app/models/llama-3-3-nemotron-super-49b-v1/)49.9B params · nvidia | 35.8 GB | 61.9 GB | 115.7 GB | | [Llama-3\_1-Nemotron-51B-Instruct](https://whatcanirunai.vercel.app/models/llama-3-1-nemotron-51b-instruct/)51.5B params · nvidia | 36.9 GB | 63.9 GB | 119.5 GB | | [AI21-Jamba2-Mini](https://whatcanirunai.vercel.app/models/ai21-jamba2-mini/)51.6B params · ai21labs · MoE | 37.0 GB | 64.0 GB | 119.6 GB | Have a different amount of memory? Try the [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/) or see [best LLMs by VRAM size](https://whatcanirunai.vercel.app/best-llm-for/). ## Quick answers How much VRAM do I need for a 40B–50B model? About 35.8 GB at Q4\_K\_M, 61.9 GB at Q8\_0 and 115.7 GB at FP16, at a \~4k context. That is for Llama-3\_3-Nemotron-Super-49B-v1\_5 (49.9B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime. How much VRAM does a 40B–50B model need for 32k context? About 110.9 GB at Q4\_K\_M with an FP16 KV cache, or 68.0 GB with a Q8 cache, based on Llama-3\_3-Nemotron-Super-49B-v1\_5's published config. Models with different attention layouts need more or less. What is the cheapest GPU for a 40B–50B model? Of the GPUs we track, the NVIDIA RTX A6000 (48 GB, $4,649) is the cheapest that runs it fully on the GPU at Q4. Can a Mac run a 40B–50B model? Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M5 Pro 64GB (64 GB, about 48.0 GB usable, $3,199). Does a MoE model of the same size need less VRAM? No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token. Other sizes: [1B–2B](https://whatcanirunai.vercel.app/how-much-vram/1b/) · [2B–3B](https://whatcanirunai.vercel.app/how-much-vram/3b/) · [4B–6B](https://whatcanirunai.vercel.app/how-much-vram/4b/) · [7B–8B](https://whatcanirunai.vercel.app/how-much-vram/8b/) · [12B–14B](https://whatcanirunai.vercel.app/how-much-vram/14b/) · [16B–21B](https://whatcanirunai.vercel.app/how-much-vram/20b/) · [24B–27B](https://whatcanirunai.vercel.app/how-much-vram/27b/) · [30B–32B](https://whatcanirunai.vercel.app/how-much-vram/32b/) · [70B](https://whatcanirunai.vercel.app/how-much-vram/70b/) · [120B](https://whatcanirunai.vercel.app/how-much-vram/120b/) · [200B–300B](https://whatcanirunai.vercel.app/how-much-vram/235b/) · [300B–480B](https://whatcanirunai.vercel.app/how-much-vram/400b/) · [500B–800B](https://whatcanirunai.vercel.app/how-much-vram/671b/) · [1T+](https://whatcanirunai.vercel.app/how-much-vram/1t/) · [All sizes](https://whatcanirunai.vercel.app/how-much-vram/) · [Ollama VRAM by tag](https://whatcanirunai.vercel.app/ollama-vram-requirements/) · [GGUF sizes by quant](https://whatcanirunai.vercel.app/gguf-vram-requirements/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/