How much VRAM do I need for a 120B model?

About 82.5 GB at Q4_K_M, 143.8 GB at Q8_0 and 269.7 GB at FP16, at a ~4k context. That is for gpt-oss-120b (116.8B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.

How much VRAM does a 120B model need for 32k context?

About 83.5 GB at Q4_K_M with an FP16 KV cache, or 82.9 GB with a Q8 cache, based on gpt-oss-120b's published config. Models with different attention layouts need more or less.

What is the cheapest GPU for a 120B model?

Of the GPUs we track, the NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565 list price) is the cheapest that runs it fully on the GPU at Q4.

Can a Mac run a 120B model?

Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Framework Desktop (Ryzen AI Max+ 395) 128GB (128 GB, about 96.0 GB usable, $3,449).

Does a MoE model of the same size need less VRAM?

No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

How much VRAM do you need for a 120B model?

Short answer: about 82.5 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 120B open model we track.

Q4_K_M: 82.5 GB 70.8 GB weights
Q8_0: 143.8 GB 124.1 GB weights
FP16: 269.7 GB 233.7 GB weights

Worked out for gpt-oss-120b (116.8B parameters, from the Hugging Face API). Formula: weights × 1.15 + 1 GB at a ~4k context (how we calculate).

Cheapest GPU that fits at Q4: NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565). Cheapest Mac or mini PC: Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449).

Cheapest hardware for each quant

Only devices with a sourced price. GPUs at list price (MSRP); street prices may be higher.

Cheapest GPU and Mac or mini PC for a 120B model at each quant
Setting	Memory needed	Cheapest GPU	Cheapest Mac / mini PC
Q4_K_M, ~4k context	82.5 GB	NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565	Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449
Q8_0, ~4k context	143.8 GB	None we track	Mac Studio M5 Ultra 256GB 192 GB usable · $9,499
FP16, ~4k context	269.7 GB	None we track	Mac Studio M5 Ultra 512GB 384 GB usable · $22,299
Q4_K_M, 32k context	83.5 GB	NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565	Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449

Memory by context length

Each extra token of context adds to the KV cache. Numbers for gpt-oss-120b, from its published config; a Q8 KV cache halves the cache part.

Memory needed for a 120B model by context length
Context	Q4_K_M, FP16 cache	Q4_K_M, Q8 cache	Q8_0, FP16 cache
4k tokens	82.5 GB	82.5 GB	143.8 GB
8k tokens	82.6 GB	82.5 GB	143.9 GB
16k tokens	82.9 GB	82.6 GB	144.2 GB
32k tokens	83.5 GB	82.9 GB	144.8 GB
64k tokens	84.7 GB	83.5 GB	146.0 GB
128k tokens	87.1 GB	84.7 GB	148.4 GB

Hardware that runs it

Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate); MoE shows an upper bound only.FitsTight

Tracked hardware that runs a 120B model at Q4
Hardware	Q4 need vs. usable	Q4 / Q8 / FP16	Est. tok/s
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449	82.5 / 96.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	up to 66
MacBook Pro M4 Max 128GBunified memory · $4,999	82.5 / 96.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	up to 141
MacBook Pro M5 Max 128GBunified memory · $6,699	82.5 / 96.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	up to 159
NVIDIA DGX Spark 128GBunified memory · $6,950	82.5 / 96.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	up to 71
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565	82.5 / 96.0 GB	Q4FitsQ8Won’t fitFP16Won’t fit	up to 493
Mac Studio M5 Ultra 256GBunified memory · $9,499	82.5 / 192.0 GB	Q4FitsQ8FitsFP16Won’t fit	up to 310
Mac Studio M5 Ultra 512GBunified memory · $22,299	82.5 / 384.0 GB	Q4FitsQ8FitsFP16Fits	up to 310

120B models we track

Memory needed by each 120B model we track
Model	Q4_K_M	Q8_0	FP16
Solar-Open-100B102.7B params · upstage · MoE	72.6 GB	126.4 GB	237.1 GB
Ling-flash-2.0102.9B params · inclusionAI · MoE	72.7 GB	126.7 GB	237.6 GB
LLaDA2.1-flash102.9B params · inclusionAI · MoE	72.7 GB	126.7 GB	237.6 GB
Ring-flash-2.0102.9B params · inclusionAI · MoE	72.7 GB	126.7 GB	237.6 GB
LLaDA2.2-flash102.9B params · inclusionAI · MoE	72.7 GB	126.7 GB	237.6 GB
sarvam-105b106.0B params · sarvamai · MoE	74.9 GB	130.6 GB	244.9 GB
Ling-2.6-flash107.5B params · inclusionAI · MoE	75.9 GB	132.3 GB	248.2 GB
GLM-4.5V107.7B params · zai-org · MoE	76.1 GB	132.6 GB	248.7 GB
GLM-4.6V107.7B params · zai-org · MoE	76.1 GB	132.6 GB	248.7 GB
Llama-4-Scout-17B-16E-Instruct108.6B params · meta-llama · MoE	76.7 GB	133.7 GB	250.9 GB
Llama-4-Scout-17B-16E108.6B params · meta-llama · MoE	76.7 GB	133.7 GB	250.9 GB
GLM-4.5-Air110.5B params · zai-org · MoE	78.0 GB	136.0 GB	255.1 GB
c4ai-command-a-03-2025111.1B params · CohereLabs	78.4 GB	136.7 GB	256.4 GB
gpt-oss-120b116.8B params · openai · MoE	82.5 GB	143.8 GB	269.7 GB
Laguna-S-2.1117.6B params · poolside · MoE	83.0 GB	144.6 GB	271.4 GB
Youtu-LLM-A7B-Base118.4B params · tencent · MoE	83.5 GB	145.7 GB	273.3 GB
Mistral-Small-4-119B-2603119.4B params · mistralai · MoE	84.2 GB	146.9 GB	275.6 GB
NVIDIA-Nemotron-3-Super-120B-A12B-BF16123.6B params · nvidia · MoE	87.2 GB	152.0 GB	285.3 GB
Ling-3.0-flash-VL124.8B params · inclusionAI · MoE	88.0 GB	153.5 GB	288.2 GB
Devstral-2-123B-Instruct-2512125.0B params · mistralai	88.2 GB	153.8 GB	288.6 GB
Ornith-1.5-122B125.1B params · ornith-ai	88.2 GB	153.8 GB	288.7 GB
Qwen3.5-122B-A10B125.1B params · Qwen · MoE	88.2 GB	153.8 GB	288.7 GB
Ling-3.0-flash-Sante127.5B params · inclusionAI · MoE	89.9 GB	156.8 GB	294.2 GB
Ling-3.0-flash127.5B params · inclusionAI · MoE	89.9 GB	156.8 GB	294.2 GB
Ling-3.0-flash-Fin127.5B params · inclusionAI · MoE	89.9 GB	156.8 GB	294.2 GB

Have a different amount of memory? Try the VRAM calculator or see best LLMs by VRAM size.

Quick answers

How much VRAM do I need for a 120B model?: About 82.5 GB at Q4_K_M, 143.8 GB at Q8_0 and 269.7 GB at FP16, at a ~4k context. That is for gpt-oss-120b (116.8B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
How much VRAM does a 120B model need for 32k context?: About 83.5 GB at Q4_K_M with an FP16 KV cache, or 82.9 GB with a Q8 cache, based on gpt-oss-120b's published config. Models with different attention layouts need more or less.
What is the cheapest GPU for a 120B model?: Of the GPUs we track, the NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565 list price) is the cheapest that runs it fully on the GPU at Q4.
Can a Mac run a 120B model?: Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Framework Desktop (Ryzen AI Max+ 395) 128GB (128 GB, about 96.0 GB usable, $3,449).
Does a MoE model of the same size need less VRAM?: No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

Other sizes: 1B–2B · 2B–3B · 4B–6B · 7B–8B · 12B–14B · 16B–21B · 24B–27B · 30B–32B · 40B–50B · 70B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant