How much VRAM do I need for a 40B–50B model?

About 35.8 GB at Q4_K_M, 61.9 GB at Q8_0 and 115.7 GB at FP16, at a ~4k context. That is for Llama-3_3-Nemotron-Super-49B-v1_5 (49.9B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.

How much VRAM does a 40B–50B model need for 32k context?

About 110.9 GB at Q4_K_M with an FP16 KV cache, or 68.0 GB with a Q8 cache, based on Llama-3_3-Nemotron-Super-49B-v1_5's published config. Models with different attention layouts need more or less.

What is the cheapest GPU for a 40B–50B model?

Of the GPUs we track, the NVIDIA RTX A6000 (48 GB, $4,649) is the cheapest that runs it fully on the GPU at Q4.

Can a Mac run a 40B–50B model?

Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M5 Pro 64GB (64 GB, about 48.0 GB usable, $3,199).

Does a MoE model of the same size need less VRAM?

No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

How much VRAM do you need for a 40B–50B model?

Short answer: about 35.8 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 40B–50B open model we track.

Q4_K_M: 35.8 GB 30.2 GB weights
Q8_0: 61.9 GB 53.0 GB weights
FP16: 115.7 GB 99.7 GB weights

Worked out for Llama-3_3-Nemotron-Super-49B-v1_5 (49.9B parameters, from the Hugging Face API). Formula: weights × 1.15 + 1 GB at a ~4k context (how we calculate).

Cheapest GPU that fits at Q4: NVIDIA RTX A6000 (48 GB, $4,649). Cheapest Mac or mini PC: Mac mini M5 Pro 64GB ($3,199).

Cheapest hardware for each quant

Only devices with a sourced price. GPUs at list price (MSRP); street prices may be higher.

Cheapest GPU and Mac or mini PC for a 40B–50B model at each quant
Setting	Memory needed	Cheapest GPU	Cheapest Mac / mini PC
Q4_K_M, ~4k context	35.8 GB	NVIDIA RTX A6000 48 GB usable · $4,649	Mac mini M5 Pro 64GB 48 GB usable · $3,199
Q8_0, ~4k context	61.9 GB	NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565	Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449
FP16, ~4k context	115.7 GB	None we track	Mac Studio M5 Ultra 256GB 192 GB usable · $9,499
Q4_K_M, 32k context	110.9 GB	None we track	Mac Studio M5 Ultra 256GB 192 GB usable · $9,499

Memory by context length

Each extra token of context adds to the KV cache. Numbers for Llama-3_3-Nemotron-Super-49B-v1_5, from its published config; a Q8 KV cache halves the cache part.

Memory needed for a 40B–50B model by context length
Context	Q4_K_M, FP16 cache	Q4_K_M, Q8 cache	Q8_0, FP16 cache
4k tokens	35.8 GB	35.8 GB	61.9 GB
8k tokens	46.5 GB	35.8 GB	72.7 GB
16k tokens	68.0 GB	46.5 GB	94.1 GB
32k tokens	110.9 GB	68.0 GB	137.1 GB
64k tokens	196.8 GB	110.9 GB	223.0 GB
128k tokens	368.6 GB	196.8 GB	394.8 GB

Hardware that runs it

Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate).FitsTight

Tracked hardware that runs a 40B–50B model at Q4
Hardware	Q4 need vs. usable	Q4 / Q8 / FP16	Est. tok/s
Mac mini M5 Pro 64GBunified memory · $3,199	35.8 / 48.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	5–8
NVIDIA RTX A6000GPU · $4,649	35.8 / 48.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	15–22
NVIDIA DGX Spark 64GBunified memory · $4,999	35.8 / 48.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	4–7
NVIDIA RTX 6000 Ada 48GBGPU · $6,799	35.8 / 48.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	19–27
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449	35.8 / 96.0 GB	Q4FitsQ8FitsFP16Won’t fit	4–7
MacBook Pro M4 Max 128GBunified memory · $4,999	35.8 / 96.0 GB	Q4FitsQ8FitsFP16Won’t fit	8–14
MacBook Pro M5 Max 128GBunified memory · $6,699	35.8 / 96.0 GB	Q4FitsQ8FitsFP16Won’t fit	9–16
NVIDIA DGX Spark 128GBunified memory · $6,950	35.8 / 96.0 GB	Q4FitsQ8FitsFP16Won’t fit	4–7
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565	35.8 / 96.0 GB	Q4FitsQ8FitsFP16Won’t fit	36–50
Mac Studio M5 Ultra 256GBunified memory · $9,499	35.8 / 192.0 GB	Q4FitsQ8FitsFP16Fits	18–32
Mac Studio M5 Ultra 512GBunified memory · $22,299	35.8 / 384.0 GB	Q4FitsQ8FitsFP16Fits	18–32

40B–50B models we track

Memory needed by each 40B–50B model we track
Model	Q4_K_M	Q8_0	FP16
K2-Horizon-MoVA-36B-A4B37.4B params · IFM · MoE	27.1 GB	46.8 GB	87.1 GB
Recursive Flint39.8B params · RecursiveSI · MoE	28.7 GB	49.6 GB	92.5 GB
IQuest-Coder-V1-40B-Instruct39.8B params · IQuestLab	28.7 GB	49.6 GB	92.5 GB
IQuest-Coder-V1-40B-Loop-Instruct39.8B params · IQuestLab	28.7 GB	49.6 GB	92.5 GB
ALIA-40b40.4B params · BSC-LT	29.2 GB	50.4 GB	94.0 GB
Phi-3.5-MoE-instruct41.9B params · microsoft · MoE	30.2 GB	52.2 GB	97.3 GB
GRIN-MoE41.9B params · microsoft · MoE	30.2 GB	52.2 GB	97.3 GB
Llama-3_3-Nemotron-Super-49B-v1_549.9B params · nvidia	35.8 GB	61.9 GB	115.7 GB
Llama-3_3-Nemotron-Super-49B-v149.9B params · nvidia	35.8 GB	61.9 GB	115.7 GB
Llama-3_1-Nemotron-51B-Instruct51.5B params · nvidia	36.9 GB	63.9 GB	119.5 GB
AI21-Jamba2-Mini51.6B params · ai21labs · MoE	37.0 GB	64.0 GB	119.6 GB

Have a different amount of memory? Try the VRAM calculator or see best LLMs by VRAM size.

Quick answers

How much VRAM do I need for a 40B–50B model?: About 35.8 GB at Q4_K_M, 61.9 GB at Q8_0 and 115.7 GB at FP16, at a ~4k context. That is for Llama-3_3-Nemotron-Super-49B-v1_5 (49.9B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
How much VRAM does a 40B–50B model need for 32k context?: About 110.9 GB at Q4_K_M with an FP16 KV cache, or 68.0 GB with a Q8 cache, based on Llama-3_3-Nemotron-Super-49B-v1_5's published config. Models with different attention layouts need more or less.
What is the cheapest GPU for a 40B–50B model?: Of the GPUs we track, the NVIDIA RTX A6000 (48 GB, $4,649) is the cheapest that runs it fully on the GPU at Q4.
Can a Mac run a 40B–50B model?: Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M5 Pro 64GB (64 GB, about 48.0 GB usable, $3,199).
Does a MoE model of the same size need less VRAM?: No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

Other sizes: 1B–2B · 2B–3B · 4B–6B · 7B–8B · 12B–14B · 16B–21B · 24B–27B · 30B–32B · 70B · 120B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant