Best GPU for running local LLMs

Quick answers

What is the best GPU for running local LLMs?

For most people it is a 24 GB card, because 24 GB fits 27B–32B models at Q4. The cheapest 24 GB card we track is the Intel Arc Pro B70 32GB ($949). On a tighter budget, the Radeon RX 7600 XT ($329) gives 16 GB. For the most speed in one card, the NVIDIA RTX Pro 6000 Blackwell 96GB has 96 GB and 1792 GB/s.

How much VRAM do I need for a local LLM?

At Q4_K_M, plan on roughly 0.7 GB per billion parameters plus about 1 GB of overhead at ~4k context. An 8B model needs ~6.7 GB, a 14B ~11.3 GB, a 32B ~23.8 GB. Longer context needs more.

What is the cheapest GPU for local AI?

The Intel Arc B580 at $249 (2026-10-22) is the cheapest 12 GB card we track. It runs 350 of our 656 models at Q4. 8 GB cards are cheaper but limit you to about 7–8B models.

Can one GPU run a 70B model?

Not at Q4 on any consumer card we track: Llama 3.3 70B needs about 50.2 GB, more than the 32 GB on an RTX 5090. Use two 24 GB cards, a 128 GB unified-memory machine, or a smaller quant with less quality.

NVIDIA, AMD or Intel for local LLMs?

NVIDIA (CUDA) has the widest app support. AMD and Intel cards work in llama.cpp-based apps through ROCm, SYCL or Vulkan, often for less money per GB of VRAM, but some tools support them later or not at all. Our fit estimates are the same for all three; speed depends mostly on memory bandwidth.

How are the speed estimates made?

Memory bandwidth divided by the model's Q4 weight size, times an efficiency band calibrated on public llama.cpp CUDA results. MoE models show an upper bound only. Real speed depends on the app, context length and settings.