How we calculate

Every verdict on this site comes from one formula. It's an estimate: real use varies with context length and runtime.

Memory a model needs

weights GB = params × bits per weight ÷ 8
needed GB = weights × 1.15 + 1 GB

Bits per weight: Q4_K_M ≈ 4.85, Q8_0 ≈ 8.5, FP16 = 16. The 15% plus 1 GB covers the KV cache at ~4k context and runtime overhead.

Verdicts

Fits needs 90% or less of usable memory.

Tight needs 90 to 100%. It loads, but keep the context short.

Won’t fit needs more than you have, so layers spill to the CPU and it gets slow.

Context length and the KV cache

The base estimate covers about 4k tokens of context. For longer contexts we add the KV cache: full-attention layers × KV heads × head size × 2 (keys and values) × 2 bytes (FP16) per token, read from each model’s config.json on Hugging Face. Sliding-window layers only cache their window, and linear-attention, Mamba and conv layers keep a small fixed state, so hybrid models grow much more slowly. A Q8_0 cache is about 53% of FP16 and Q4_0 about 28%. Models with compressed or shared attention (for example MLA) aren’t sized yet and show “cache not modeled”.

Usable memory

GPUs: all of the VRAM. Macs and unified-memory mini-PCs: about 75% of memory, a rule of thumb for what the GPU may use.

Want to plug in your own numbers? Use the VRAM calculator.

Estimated speed (tokens per second)

tok/s ≈ memory bandwidth (GB/s) ÷ model weights (GB) × efficiency

Generating each token reads every active weight once, so decode speed is capped by memory bandwidth. Efficiency bands: 60–85% for discrete GPUs, 45–80% for unified memory (Macs, Ryzen AI Max, DGX Spark). We show the range, not one number.

Calibration (Llama 2 7B Q4_0, 3.82 GB, tg128): llama.cpp CUDA scoreboard #15013 gives RTX 3090 162 tok/s (66% of 936 GB/s), RTX 4090 189 (72%), RTX 5090 290 (62%), RTX 3080 140 (70%), RTX 4070 Ti Super 132 (75%), RTX 4060 Ti 64 (85%). llama.cpp Apple Silicon scoreboard #4167 gives M1 Ultra 84 tok/s (40%), M2 Ultra 94 (45%), M4 Max 83 (58%), M5 Max 120 (75%), M6 36 (80%). Expect real numbers up to about ±25% outside the band: longer contexts, other runtimes and other quant formats all move it.

Mixture-of-experts models: when active parameters are known (from the name, e.g. 30B-A3B, or OpenAI's gpt-oss model card), we show only an upper bound ("up to") from the active weights, because routing and attention overhead isn't calibrated. Otherwise we use total parameters, which underestimates MoE speed. Multi-GPU: memory adds up, but llama.cpp's default layer split runs cards in turn, so speed is about one card's on the whole model. Prompt processing (prefill) is compute-bound and not estimated.

Power and fit checks

PSU ≥ (GPU board power × count + CPU TDP + 150 W) × 1.3

GPU board power, length and slot width come from NVIDIA Founders Edition and AMD reference spec pages; partner cards are often longer and hungrier. Case limits and PSU connectors come from Corsair product pages. Running cost is load watts × your electricity rate (the $0.18/kWh default is an example, not a sourced average). Max context is not shown yet: it needs per-model KV-cache data we haven't sourced.