How much VRAM do you need for a 1B–2B model?

Quick answers

How much VRAM do I need for a 1B–2B model?

About 1.9 GB at Q4_K_M, 2.5 GB at Q8_0 and 3.8 GB at FP16, at a ~4k context. That is for Llama-3.2-1B-Instruct (1.2B parameters): weights plus about 15% for the KV cache and 1 GB for the runtime.

How much VRAM does a 1B–2B model need for 32k context?

About 2.8 GB at Q4_K_M with an FP16 KV cache, or 2.3 GB with a Q8 cache, based on Llama-3.2-1B-Instruct's published config. Models with different attention layouts need more or less.

What is the cheapest GPU for a 1B–2B model?

Of the GPUs we track, the Intel Arc B570 (10 GB, $219 list price) is the cheapest that runs it fully on the GPU at Q4.

Can a Mac run a 1B–2B model?

Yes. The cheapest Mac or unified-memory mini PC we track that runs it at Q4 is the Mac mini M4 16GB (16 GB, about 12.0 GB usable, $789).

Does a MoE model of the same size need less VRAM?

No. A mixture-of-experts model still has to hold all of its weights in memory, so it needs about the same VRAM as a dense model with the same total parameters. It runs faster, because only a few experts are read per token.