How much VRAM do I need for a 70B model?

About 50.2 GB at Q4_K_M, 87.2 GB at Q8_0 and 163.3 GB at FP16, at a ~4k context. That is for Llama-3.3-70B-Instruct (70.6B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.

How much VRAM does a 70B model need for 32k context?

About 59.6 GB at Q4_K_M with an FP16 KV cache, or 54.2 GB with a Q8 cache, based on Llama-3.3-70B-Instruct's published config. Models with different attention layouts need more or less.

What is the cheapest GPU for a 70B model?

Of the GPUs we track, the NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565 list price) is the cheapest that runs it fully on the GPU at Q4.

Can a Mac run a 70B model?

Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Framework Desktop (Ryzen AI Max+ 395) 128GB (128 GB, about 96.0 GB usable, $3,449).

Does a MoE model of the same size need less VRAM?

No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

How much VRAM do you need for a 70B model?

Short answer: about 50.2 GB at Q4_K_M, the usual quant for local use. Below: every quant, longer contexts, the cheapest hardware that fits, and every 70B open model we track.

Q4_K_M: 50.2 GB 42.8 GB weights
Q8_0: 87.2 GB 75.0 GB weights
FP16: 163.3 GB 141.1 GB weights

Worked out for Llama-3.3-70B-Instruct (70.6B parameters, from the Hugging Face API). Formula: weights × 1.15 + 1 GB at a ~4k context (how we calculate).

Cheapest GPU that fits at Q4: NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565). Cheapest Mac or mini PC: Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449).

Cheapest hardware for each quant

Only devices with a sourced price. GPUs at list price (MSRP); street prices may be higher.

Cheapest GPU and Mac or mini PC for a 70B model at each quant
Setting	Memory needed	Cheapest GPU	Cheapest Mac / mini PC
Q4_K_M, ~4k context	50.2 GB	NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565	Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449
Q8_0, ~4k context	87.2 GB	None we track	Mac Studio M5 Ultra 256GB 192 GB usable · $9,499
FP16, ~4k context	163.3 GB	None we track	Mac Studio M5 Ultra 256GB 192 GB usable · $9,499
Q4_K_M, 32k context	59.6 GB	NVIDIA RTX Pro 6000 Blackwell 96GB 96 GB usable · $8,565	Framework Desktop (Ryzen AI Max+ 395) 128GB 96 GB usable · $3,449

Memory by context length

Each extra token of context adds to the KV cache. Numbers for Llama-3.3-70B-Instruct, from its published config; a Q8 KV cache halves the cache part.

Memory needed for a 70B model by context length
Context	Q4_K_M, FP16 cache	Q4_K_M, Q8 cache	Q8_0, FP16 cache
4k tokens	50.2 GB	50.2 GB	87.2 GB
8k tokens	51.5 GB	50.2 GB	88.5 GB
16k tokens	54.2 GB	51.5 GB	91.2 GB
32k tokens	59.6 GB	54.2 GB	96.6 GB
64k tokens	70.3 GB	59.6 GB	107.3 GB
128k tokens	91.8 GB	70.3 GB	128.8 GB

Hardware that runs it

Bar: Q4 memory needed vs. usable memory. Speed is estimated decode at Q4 (how we estimate).FitsTight

Tracked hardware that runs a 70B model at Q4
Hardware	Q4 need vs. usable	Q4 / Q8 / FP16	Est. tok/s
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · $3,449	50.2 / 96.0 GB	Q4FitsQ8TightFP16Won’t fit	3–5
MacBook Pro M4 Max 128GBunified memory · $4,999	50.2 / 96.0 GB	Q4FitsQ8TightFP16Won’t fit	6–10
MacBook Pro M5 Max 128GBunified memory · $6,699	50.2 / 96.0 GB	Q4FitsQ8TightFP16Won’t fit	6–11
NVIDIA DGX Spark 128GBunified memory · $6,950	50.2 / 96.0 GB	Q4FitsQ8TightFP16Won’t fit	3–5
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · $8,565	50.2 / 96.0 GB	Q4FitsQ8TightFP16Won’t fit	25–36
Mac Studio M5 Ultra 256GBunified memory · $9,499	50.2 / 192.0 GB	Q4FitsQ8FitsFP16Fits	13–22
Mac Studio M5 Ultra 512GBunified memory · $22,299	50.2 / 384.0 GB	Q4FitsQ8FitsFP16Fits	13–22

70B models we track

Memory needed by each 70B model we track
Model	Q4_K_M	Q8_0	FP16
LongCat-Flash-Lite69.1B params · meituan-longcat	49.2 GB	85.4 GB	159.9 GB
Llama-3.3-70B-Instruct70.6B params · meta-llama	50.2 GB	87.2 GB	163.3 GB
DeepSeek-R1-Distill-Llama-70B70.6B params · deepseek-ai	50.2 GB	87.2 GB	163.3 GB
Llama-3.1-70B70.6B params · meta-llama	50.2 GB	87.2 GB	163.3 GB
Hermes-2-Theta-Llama-3-70B70.6B params · NousResearch	50.2 GB	87.2 GB	163.3 GB
Meta-Llama-3.1-70B-Instruct70.6B params · NousResearch	50.2 GB	87.2 GB	163.3 GB
Apertus-70B-Instruct-250970.6B params · swiss-ai	50.2 GB	87.3 GB	163.4 GB
Apertus-70B-250970.6B params · swiss-ai	50.2 GB	87.3 GB	163.4 GB
Qwen2.5-72B-Instruct72.7B params · Qwen	51.7 GB	89.8 GB	168.2 GB
UI-TARS-72B-DPO73.4B params · ByteDance-Seed	52.2 GB	90.7 GB	169.8 GB
Qwen2.5-VL-72B-Instruct73.4B params · Qwen	52.2 GB	90.7 GB	169.8 GB
Kolibri-178.1B params · Aleph-Alpha	55.5 GB	96.4 GB	180.6 GB
Qwen3-Coder-Next79.7B params · Qwen · MoE	56.5 GB	98.4 GB	184.3 GB

Have a different amount of memory? Try the VRAM calculator or see best LLMs by VRAM size.

Quick answers

How much VRAM do I need for a 70B model?: About 50.2 GB at Q4_K_M, 87.2 GB at Q8_0 and 163.3 GB at FP16, at a ~4k context. That is for Llama-3.3-70B-Instruct (70.6B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.
How much VRAM does a 70B model need for 32k context?: About 59.6 GB at Q4_K_M with an FP16 KV cache, or 54.2 GB with a Q8 cache, based on Llama-3.3-70B-Instruct's published config. Models with different attention layouts need more or less.
What is the cheapest GPU for a 70B model?: Of the GPUs we track, the NVIDIA RTX Pro 6000 Blackwell 96GB (96 GB, $8,565 list price) is the cheapest that runs it fully on the GPU at Q4.
Can a Mac run a 70B model?: Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Framework Desktop (Ryzen AI Max+ 395) 128GB (128 GB, about 96.0 GB usable, $3,449).
Does a MoE model of the same size need less VRAM?: No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.

Other sizes: 1B–2B · 2B–3B · 4B–6B · 7B–8B · 12B–14B · 16B–21B · 24B–27B · 30B–32B · 40B–50B · 120B · 200B–300B · 300B–480B · 500B–800B · 1T+ · All sizes · Ollama VRAM by tag · GGUF sizes by quant