How many tokens per second will a local LLM run at?

Decode speed is set mostly by memory bandwidth: every new token reads the whole model (or, for MoE models, the active experts) from memory. Our estimate is bandwidth ÷ Q4 weight size × an efficiency band of 60–85% for GPUs and 45–80% for unified memory, calibrated on public llama.cpp results. For example, Qwen3-8B (~5.0 GB at Q4) runs at an estimated 217–307 tok/s on the GeForce RTX 5090.

What is the fastest GPU for a 32B model?

Of the devices we track, the GeForce RTX 5090 (1792 GB/s) is fastest for Qwen3-32B at Q4: an estimated 54–77 tok/s. A 32B model needs ~23.8 GB at Q4: 24 GB cards hold it with short context only; 32 GB cards fit it with room for ~4k context.

How fast does a 70B model run locally?

Llama-3.3-70B-Instruct needs ~50.2 GB at Q4, so no single consumer GPU we track holds it. On a single device, the fastest we track is the NVIDIA RTX Pro 6000 Blackwell 96GB at an estimated 25–36 tok/s; the cheapest that fits, the Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449), manages about 3–5 tok/s. Two 24 GB GPUs also hold it, at roughly one card's speed (see our planner).

Why do MoE models like gpt-oss show "up to"?

A mixture-of-experts model reads only its active experts per token (5.1B of 116.8B for gpt-oss-120b), so it can run much faster than its total size suggests. Routing, attention and shared layers add cost we haven't calibrated, so we show only an upper bound. All of the model still has to fit in memory.

Is a Mac or a GPU faster for local LLMs?

For a model that fits in a GPU's VRAM, the GPU is usually faster per dollar because desktop cards have more bandwidth per dollar. Macs and unified-memory mini PCs win on capacity: they run models no single consumer card can hold. See our NVIDIA vs AMD vs Mac page.

Does context length change the speed?

Yes. These estimates are for short prompts. Longer contexts make each token read the KV cache too, so speed drops as the conversation grows, and prompt processing (prefill) is a separate, compute-bound step we don't estimate. See our method page.

How fast do local LLMs run? Tokens per second by GPU and Mac

Estimated decode speed in tokens per second for 8 popular open models at Q4_K_M on every current GPU, Mac and mini PC we track, with prices and speed per dollar. Memory decides whether a model fits; memory bandwidth decides how fast it runs.

Devices: 51
Models: 8
Bandwidth range: 153–1792 GB/s

Short answer: an 8B model like Qwen3-8B runs at an estimated 217–307 tok/s on the GeForce RTX 5090, Qwen3-32B at 54–77 tok/s on the same card, and Llama-3.3-70B-Instruct at 25–36 tok/s on the NVIDIA RTX Pro 6000 Blackwell 96GB. These are estimates from memory bandwidth (method), not benchmarks.

Jump to: Qwen3-8B · Qwen3-14B · gpt-oss-20b · Qwen3-30B-A3B · Qwen3.8-27B · Qwen3-32B · Llama-3.3-70B-Instruct · gpt-oss-120b · all devices by bandwidth

"Fit at Q4": fits with ~4k context, or tight (short context only). tok/s per $1,000 uses the low end of the estimate and the dated list price (MSRP); it is shown for dense models only, since MoE estimates are upper bounds. Discontinued or reseller-priced devices are left out. Longer contexts and multi-GPU splits run slower than these figures.

Qwen3-8B: tokens per second

Qwen3-8B · 8.2B parameters · ~5.0 GB of Q4_K_M weights · needs ~6.7 GB with ~4k context. 51 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at 217–307 tok/s. Cheapest that fits: Intel Arc B570 ($219) at 46–65 tok/s. Most speed per dollar: Intel Arc B580 (221.3 tok/s per $1,000).

Estimated decode speed of Qwen3-8B at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	217–307	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	108.3
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	217–307	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	25.3
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	109–193	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	11.4
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	109–193	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	4.9
GeForce RTX 4090GPU · 24 GB usable	Fits	122–173	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	76.2
GeForce RTX 3090 TiGPU · 24 GB usable	Fits	122–173	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	60.9
GeForce RTX 5080GPU · 16 GB usable	Fits	116–164	960 GB/s	$999list price (MSRP), source, 2026-10-22	116.1
Radeon RX 7900 XTXGPU · 24 GB usable	Fits	116–164	960 GB/s	$999list price (MSRP), source, 2026-10-22	116.1
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	116–164	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	17.1
GeForce RTX 3090GPU · 24 GB usable	Fits	113–160	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	75.4
GeForce RTX 3080 TiGPU · 12 GB usable	Fits	110–156	912 GB/s	$1,199list price (MSRP), source, 2026-10-22	91.9
GeForce RTX 5070 TiGPU · 16 GB usable	Fits	108–153	896 GB/s	$749list price (MSRP), source, 2026-10-28	144.5
Radeon RX 7900 XTGPU · 20 GB usable	Fits	97–137	800 GB/s	$899list price (MSRP), source, 2026-10-22	107.5
GeForce RTX 3080 10GBGPU · 10 GB usable	Fits	92–130	760 GB/s	$699list price (MSRP), source, 2026-10-22	131.4
GeForce RTX 4080 SuperGPU · 16 GB usable	Fits	89–126	736 GB/s	$999list price (MSRP), source, 2026-10-22	89.0
GeForce RTX 5070GPU · 12 GB usable	Fits	81–115	672 GB/s	$549list price (MSRP), source, 2026-10-22	147.9
GeForce RTX 4070 Ti SuperGPU · 16 GB usable	Fits	81–115	672 GB/s	$799list price (MSRP), source, 2026-10-22	101.6
Radeon RX 9070GPU · 16 GB usable	Fits	77–110	640 GB/s	$549list price (MSRP), source, 2026-10-28	140.9
Radeon RX 9070 XTGPU · 16 GB usable	Fits	77–110	640 GB/s	$599list price (MSRP), source, 2026-10-28	129.1
Radeon RX 7800 XTGPU · 16 GB usable	Fits	75–107	624 GB/s	$499list price (MSRP), source, 2026-10-22	151.1
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	73–104	608 GB/s	$949list price (MSRP), source, 2026-10-29	77.4
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	56–99	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	8.3
Radeon RX 7900 GREGPU · 16 GB usable	Fits	70–99	576 GB/s	$549list price (MSRP), source, 2026-10-28	126.8
GeForce RTX 4070GPU · 12 GB usable	Fits	61–86	504 GB/s	$599list price (MSRP), source, 2026-10-22	101.7
GeForce RTX 4070 SuperGPU · 12 GB usable	Fits	61–86	504 GB/s	$599list price (MSRP), source, 2026-10-22	101.7
Intel Arc B580GPU · 12 GB usable	Fits	55–78	456 GB/s	$249list price (MSRP), source, 2026-10-22	221.3
GeForce RTX 5060GPU · 8 GB usable	Fits	54–77	448 GB/s	$299list price (MSRP), source, 2026-10-28	181.0
GeForce RTX 5060 Ti 8GBGPU · 8 GB usable	Fits	54–77	448 GB/s	$379list price (MSRP), source, 2026-10-28	142.8
GeForce RTX 3060 TiGPU · 8 GB usable	Fits	54–77	448 GB/s	$399list price (MSRP), source, 2026-10-22	135.7
GeForce RTX 5060 Ti 16GBGPU · 16 GB usable	Fits	54–77	448 GB/s	$429list price (MSRP), source, 2026-10-28	126.2
GeForce RTX 3070GPU · 8 GB usable	Fits	54–77	448 GB/s	$499list price (MSRP), source, 2026-10-22	108.5
Radeon RX 7700 XTGPU · 12 GB usable	Fits	52–74	432 GB/s	$449list price (MSRP), source, 2026-10-28	116.3
Intel Arc B570GPU · 10 GB usable	Fits	46–65	380 GB/s	$219list price (MSRP), source, 2026-10-28	209.7
GeForce RTX 3060 12GBGPU · 12 GB usable	Fits	43–62	360 GB/s	$329list price (MSRP), source, 2026-10-22	132.2
NVIDIA RTX 4000 Ada 20GBGPU · 20 GB usable	Fits	43–62	360 GB/s	$1,250list price (MSRP), source, 2026-10-28	34.8
Radeon RX 9060 XT 8GBGPU · 8 GB usable	Fits	39–55	320 GB/s	$299list price (MSRP), source, 2026-10-28	129.3
Radeon RX 9060 XT 16GBGPU · 16 GB usable	Fits	39–55	320 GB/s	$349list price (MSRP), source, 2026-10-28	110.8
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	28–49	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	8.7
Radeon RX 7600GPU · 8 GB usable	Fits	35–49	288 GB/s	$269list price (MSRP), source, 2026-10-28	129.4
Radeon RX 7600 XTGPU · 16 GB usable	Fits	35–49	288 GB/s	$329list price (MSRP), source, 2026-10-22	105.8
GeForce RTX 4060 Ti 16GBGPU · 16 GB usable	Fits	35–49	288 GB/s	$499list price (MSRP), source, 2026-10-22	69.7
GeForce RTX 4060GPU · 8 GB usable	Fits	33–47	272 GB/s	$299list price (MSRP), source, 2026-10-22	109.9
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	25–44	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	4.9
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	25–44	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	3.6
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	23–41	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	6.7
GeForce RTX 3050 8GBGPU · 8 GB usable	Fits	27–38	224 GB/s	$249list price (MSRP), source, 2026-10-22	108.7
NVIDIA RTX 2000 Ada 16GBGPU · 16 GB usable	Fits	27–38	224 GB/s	$649list price (MSRP), source, 2026-10-28	41.7
Mac mini M6 24GBunified memory · 18 GB usable	Fits	15–27	170 GB/s	$1,299Apple list price, M6 12C CPU/12C GPU, 24GB/512GB (MHQM4LL/A), source, 2026-11-12	11.9
Mac mini M6 32GBunified memory · 24 GB usable	Fits	15–27	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	8.6
MacBook Pro M6 32GBunified memory · 24 GB usable	Fits	15–27	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	6.4
Mac mini M6 16GBunified memory · 12 GB usable	Fits	14–25	153 GB/s	$899Apple list price, M6 12C CPU/12C GPU, 16GB/256GB (MHQK4LL/A), source, 2026-11-12	15.4

Qwen3-14B: tokens per second

Qwen3-14B · 14.8B parameters · ~9.0 GB of Q4_K_M weights · needs ~11.3 GB with ~4k context. 41 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at 120–170 tok/s. Cheapest that fits: Radeon RX 7600 XT ($329) at 19–27 tok/s. Most speed per dollar: Radeon RX 7800 XT (83.8 tok/s per $1,000).

Estimated decode speed of Qwen3-14B at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	120–170	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	60.1
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	120–170	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	14.0
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	60–107	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	6.3
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	60–107	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	2.7
GeForce RTX 4090GPU · 24 GB usable	Fits	68–96	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	42.2
GeForce RTX 3090 TiGPU · 24 GB usable	Fits	68–96	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	33.8
GeForce RTX 5080GPU · 16 GB usable	Fits	64–91	960 GB/s	$999list price (MSRP), source, 2026-10-22	64.4
Radeon RX 7900 XTXGPU · 24 GB usable	Fits	64–91	960 GB/s	$999list price (MSRP), source, 2026-10-22	64.4
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	64–91	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	9.5
GeForce RTX 3090GPU · 24 GB usable	Fits	63–89	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	41.8
GeForce RTX 3080 TiGPU · 12 GB usable	Tight	61–87	912 GB/s	$1,199list price (MSRP), source, 2026-10-22	51.0
GeForce RTX 5070 TiGPU · 16 GB usable	Fits	60–85	896 GB/s	$749list price (MSRP), source, 2026-10-28	80.2
Radeon RX 7900 XTGPU · 20 GB usable	Fits	54–76	800 GB/s	$899list price (MSRP), source, 2026-10-22	59.6
GeForce RTX 4080 SuperGPU · 16 GB usable	Fits	49–70	736 GB/s	$999list price (MSRP), source, 2026-10-22	49.4
GeForce RTX 5070GPU · 12 GB usable	Tight	45–64	672 GB/s	$549list price (MSRP), source, 2026-10-22	82.0
GeForce RTX 4070 Ti SuperGPU · 16 GB usable	Fits	45–64	672 GB/s	$799list price (MSRP), source, 2026-10-22	56.4
Radeon RX 9070GPU · 16 GB usable	Fits	43–61	640 GB/s	$549list price (MSRP), source, 2026-10-28	78.1
Radeon RX 9070 XTGPU · 16 GB usable	Fits	43–61	640 GB/s	$599list price (MSRP), source, 2026-10-28	71.6
Radeon RX 7800 XTGPU · 16 GB usable	Fits	42–59	624 GB/s	$499list price (MSRP), source, 2026-10-22	83.8
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	41–58	608 GB/s	$949list price (MSRP), source, 2026-10-29	42.9
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	31–55	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	4.6
Radeon RX 7900 GREGPU · 16 GB usable	Fits	39–55	576 GB/s	$549list price (MSRP), source, 2026-10-28	70.3
GeForce RTX 4070GPU · 12 GB usable	Tight	34–48	504 GB/s	$599list price (MSRP), source, 2026-10-22	56.4
GeForce RTX 4070 SuperGPU · 12 GB usable	Tight	34–48	504 GB/s	$599list price (MSRP), source, 2026-10-22	56.4
Intel Arc B580GPU · 12 GB usable	Tight	31–43	456 GB/s	$249list price (MSRP), source, 2026-10-22	122.7
GeForce RTX 5060 Ti 16GBGPU · 16 GB usable	Fits	30–43	448 GB/s	$429list price (MSRP), source, 2026-10-28	70.0
Radeon RX 7700 XTGPU · 12 GB usable	Tight	29–41	432 GB/s	$449list price (MSRP), source, 2026-10-28	64.5
GeForce RTX 3060 12GBGPU · 12 GB usable	Tight	24–34	360 GB/s	$329list price (MSRP), source, 2026-10-22	73.3
NVIDIA RTX 4000 Ada 20GBGPU · 20 GB usable	Fits	24–34	360 GB/s	$1,250list price (MSRP), source, 2026-10-28	19.3
Radeon RX 9060 XT 16GBGPU · 16 GB usable	Fits	21–30	320 GB/s	$349list price (MSRP), source, 2026-10-28	61.4
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	15–27	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	4.8
Radeon RX 7600 XTGPU · 16 GB usable	Fits	19–27	288 GB/s	$329list price (MSRP), source, 2026-10-22	58.7
GeForce RTX 4060 Ti 16GBGPU · 16 GB usable	Fits	19–27	288 GB/s	$499list price (MSRP), source, 2026-10-22	38.7
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	14–24	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	2.7
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	14–24	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	2.0
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	13–23	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	3.7
NVIDIA RTX 2000 Ada 16GBGPU · 16 GB usable	Fits	15–21	224 GB/s	$649list price (MSRP), source, 2026-10-28	23.1
Mac mini M6 24GBunified memory · 18 GB usable	Fits	9–15	170 GB/s	$1,299Apple list price, M6 12C CPU/12C GPU, 24GB/512GB (MHQM4LL/A), source, 2026-11-12	6.6
Mac mini M6 32GBunified memory · 24 GB usable	Fits	9–15	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	4.7
MacBook Pro M6 32GBunified memory · 24 GB usable	Fits	9–15	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	3.6
Mac mini M6 16GBunified memory · 12 GB usable	Tight	8–14	153 GB/s	$899Apple list price, M6 12C CPU/12C GPU, 16GB/256GB (MHQK4LL/A), source, 2026-11-12	8.6

gpt-oss-20b: tokens per second

gpt-oss-20b · 20.9B parameters, MoE (3.6B active per token) · ~12.7 GB of Q4_K_M weights · needs ~15.6 GB with ~4k context. 33 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at up to 698 tok/s. Cheapest that fits: Radeon RX 7900 XT ($899) at up to 312 tok/s.

Estimated decode speed of gpt-oss-20b at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	up to 698	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	—
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	up to 698	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	—
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	up to 440	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	—
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	up to 440	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	—
GeForce RTX 4090GPU · 24 GB usable	Fits	up to 393	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	—
GeForce RTX 3090 TiGPU · 24 GB usable	Fits	up to 393	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	—
GeForce RTX 5080GPU · 16 GB usable	Tight	up to 374	960 GB/s	$999list price (MSRP), source, 2026-10-22	—
Radeon RX 7900 XTXGPU · 24 GB usable	Fits	up to 374	960 GB/s	$999list price (MSRP), source, 2026-10-22	—
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	up to 374	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	—
GeForce RTX 3090GPU · 24 GB usable	Fits	up to 365	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	—
GeForce RTX 5070 TiGPU · 16 GB usable	Tight	up to 349	896 GB/s	$749list price (MSRP), source, 2026-10-28	—
Radeon RX 7900 XTGPU · 20 GB usable	Fits	up to 312	800 GB/s	$899list price (MSRP), source, 2026-10-22	—
GeForce RTX 4080 SuperGPU · 16 GB usable	Tight	up to 287	736 GB/s	$999list price (MSRP), source, 2026-10-22	—
GeForce RTX 4070 Ti SuperGPU · 16 GB usable	Tight	up to 262	672 GB/s	$799list price (MSRP), source, 2026-10-22	—
Radeon RX 9070GPU · 16 GB usable	Tight	up to 249	640 GB/s	$549list price (MSRP), source, 2026-10-28	—
Radeon RX 9070 XTGPU · 16 GB usable	Tight	up to 249	640 GB/s	$599list price (MSRP), source, 2026-10-28	—
Radeon RX 7800 XTGPU · 16 GB usable	Tight	up to 243	624 GB/s	$499list price (MSRP), source, 2026-10-22	—
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	up to 237	608 GB/s	$949list price (MSRP), source, 2026-10-29	—
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	up to 225	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	—
Radeon RX 7900 GREGPU · 16 GB usable	Tight	up to 224	576 GB/s	$549list price (MSRP), source, 2026-10-28	—
GeForce RTX 5060 Ti 16GBGPU · 16 GB usable	Tight	up to 174	448 GB/s	$429list price (MSRP), source, 2026-10-28	—
NVIDIA RTX 4000 Ada 20GBGPU · 20 GB usable	Fits	up to 140	360 GB/s	$1,250list price (MSRP), source, 2026-10-28	—
Radeon RX 9060 XT 16GBGPU · 16 GB usable	Tight	up to 125	320 GB/s	$349list price (MSRP), source, 2026-10-28	—
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	up to 113	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	—
Radeon RX 7600 XTGPU · 16 GB usable	Tight	up to 112	288 GB/s	$329list price (MSRP), source, 2026-10-22	—
GeForce RTX 4060 Ti 16GBGPU · 16 GB usable	Tight	up to 112	288 GB/s	$499list price (MSRP), source, 2026-10-22	—
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	up to 100	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	—
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	up to 100	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	—
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	up to 94	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	—
NVIDIA RTX 2000 Ada 16GBGPU · 16 GB usable	Tight	up to 87	224 GB/s	$649list price (MSRP), source, 2026-10-28	—
Mac mini M6 24GBunified memory · 18 GB usable	Fits	up to 62	170 GB/s	$1,299Apple list price, M6 12C CPU/12C GPU, 24GB/512GB (MHQM4LL/A), source, 2026-11-12	—
Mac mini M6 32GBunified memory · 24 GB usable	Fits	up to 62	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	—
MacBook Pro M6 32GBunified memory · 24 GB usable	Fits	up to 62	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	—

Qwen3-30B-A3B: tokens per second

Qwen3-30B-A3B · 30.5B parameters, MoE (3.0B active per token) · ~18.5 GB of Q4_K_M weights · needs ~22.3 GB with ~4k context. 17 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at up to 837 tok/s. Cheapest that fits: Intel Arc Pro B70 32GB ($949) at up to 284 tok/s.

Estimated decode speed of Qwen3-30B-A3B at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	up to 837	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	—
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	up to 837	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	—
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	up to 528	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	—
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	up to 528	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	—
GeForce RTX 4090GPU · 24 GB usable	Tight	up to 471	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	—
GeForce RTX 3090 TiGPU · 24 GB usable	Tight	up to 471	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	—
Radeon RX 7900 XTXGPU · 24 GB usable	Tight	up to 449	960 GB/s	$999list price (MSRP), source, 2026-10-22	—
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	up to 449	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	—
GeForce RTX 3090GPU · 24 GB usable	Tight	up to 437	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	—
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	up to 284	608 GB/s	$949list price (MSRP), source, 2026-10-29	—
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	up to 270	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	—
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	up to 135	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	—
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	up to 120	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	—
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	up to 120	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	—
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	up to 113	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	—
Mac mini M6 32GBunified memory · 24 GB usable	Tight	up to 75	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	—
MacBook Pro M6 32GBunified memory · 24 GB usable	Tight	up to 75	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	—

Qwen3.8-27B: tokens per second

Qwen3.8-27B · 27.8B parameters · ~16.8 GB of Q4_K_M weights · needs ~20.4 GB with ~4k context. 17 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at 64–90 tok/s. Cheapest that fits: Intel Arc Pro B70 32GB ($949) at 22–31 tok/s. Most speed per dollar: Radeon RX 7900 XTX (34.2 tok/s per $1,000).

Estimated decode speed of Qwen3.8-27B at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	64–90	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	31.9
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	64–90	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	7.5
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	32–57	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	3.4
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	32–57	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	1.4
GeForce RTX 4090GPU · 24 GB usable	Fits	36–51	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	22.5
GeForce RTX 3090 TiGPU · 24 GB usable	Fits	36–51	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	18.0
Radeon RX 7900 XTXGPU · 24 GB usable	Fits	34–48	960 GB/s	$999list price (MSRP), source, 2026-10-22	34.2
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	34–48	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	5.0
GeForce RTX 3090GPU · 24 GB usable	Fits	33–47	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	22.2
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	22–31	608 GB/s	$949list price (MSRP), source, 2026-10-29	22.8
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	16–29	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	2.4
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	8–15	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	2.6
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	7–13	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	1.5
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	7–13	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	1.0
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	7–12	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	2.0
Mac mini M6 32GBunified memory · 24 GB usable	Fits	5–8	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	2.5
MacBook Pro M6 32GBunified memory · 24 GB usable	Fits	5–8	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	1.9

Qwen3-32B: tokens per second

Qwen3-32B · 32.8B parameters · ~19.9 GB of Q4_K_M weights · needs ~23.8 GB with ~4k context. 17 of 51 current devices run it at Q4.

Fastest: GeForce RTX 5090 at 54–77 tok/s. Cheapest that fits: Intel Arc Pro B70 32GB ($949) at 18–26 tok/s. It is also the most speed per dollar (27.1 tok/s per $1,000).

Estimated decode speed of Qwen3-32B at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
GeForce RTX 5090GPU · 32 GB usable	Fits	54–77	1792 GB/s	$1,999list price (MSRP), source, 2026-10-22	27.1
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	54–77	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	6.3
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	27–48	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	2.9
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	27–48	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	1.2
GeForce RTX 4090GPU · 24 GB usable	Tight	30–43	1008 GB/s	$1,599list price (MSRP), source, 2026-10-22	19.0
GeForce RTX 3090 TiGPU · 24 GB usable	Tight	30–43	1008 GB/s	$1,999list price (MSRP), source, 2026-10-22	15.2
Radeon RX 7900 XTXGPU · 24 GB usable	Tight	29–41	960 GB/s	$999list price (MSRP), source, 2026-10-22	29.0
NVIDIA RTX 6000 Ada 48GBGPU · 48 GB usable	Fits	29–41	960 GB/s	$6,799list price (MSRP), source, 2026-10-28	4.3
GeForce RTX 3090GPU · 24 GB usable	Tight	28–40	936 GB/s	$1,499list price (MSRP), source, 2026-10-22	18.9
Intel Arc Pro B70 32GBGPU · 32 GB usable	Fits	18–26	608 GB/s	$949list price (MSRP), source, 2026-10-29	19.4
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	14–25	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	2.1
Mac mini M5 Pro 64GBunified memory · 48 GB usable	Fits	7–12	307 GB/s	$3,199Apple list price, M5 Pro 18C CPU/20C GPU, 64GB/1TB (MHQU4LL/A), source, 2026-11-12	2.2
NVIDIA DGX Spark 64GBunified memory · 48 GB usable	Fits	6–11	273 GB/s	$4,999NVIDIA OEM-partner starting price (Acer/ASUS/Dell/Gigabyte/HP/MSI; not sold on NVIDIA store), source, 2026-11-05	1.2
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	6–11	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	0.9
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	6–10	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	1.7
Mac mini M6 32GBunified memory · 24 GB usable	Tight	4–7	170 GB/s	$1,799Apple list price, M6 12C CPU/12C GPU, 32GB/1TB (MHQV4LL/A), source, 2026-11-12	2.1
MacBook Pro M6 32GBunified memory · 24 GB usable	Tight	4–7	170 GB/s	$2,399Apple list price, 14-inch M6 12C CPU/12C GPU, 32GB/1TB (MHWV4LL/A; available starting Nov 6, 2026), source, 2026-11-12	1.6

Llama-3.3-70B-Instruct: tokens per second

Llama-3.3-70B-Instruct · 70.6B parameters · ~42.8 GB of Q4_K_M weights · needs ~50.2 GB with ~4k context. 6 of 51 current devices run it at Q4.

Fastest: NVIDIA RTX Pro 6000 Blackwell 96GB at 25–36 tok/s. Cheapest that fits: Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449) at 3–5 tok/s. It is also the most speed per dollar (2.9 tok/s per $1,000).

Estimated decode speed of Llama-3.3-70B-Instruct at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	25–36	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	2.9
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	13–22	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	1.3
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	13–22	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	0.6
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	6–11	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	1.0
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	3–5	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	0.4
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	3–5	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	0.8

gpt-oss-120b: tokens per second

gpt-oss-120b · 116.8B parameters, MoE (5.1B active per token) · ~70.8 GB of Q4_K_M weights · needs ~82.5 GB with ~4k context. 6 of 51 current devices run it at Q4.

Fastest: NVIDIA RTX Pro 6000 Blackwell 96GB at up to 493 tok/s. Cheapest that fits: Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449) at up to 66 tok/s.

Estimated decode speed of gpt-oss-120b at Q4 on each device that runs it
Device	Fit at Q4	Est. tok/s	Bandwidth	Price	tok/s per $1,000
NVIDIA RTX Pro 6000 Blackwell 96GBGPU · 96 GB usable	Fits	up to 493	1792 GB/s	$8,565list price (MSRP), source, 2026-10-28	—
Mac Studio M5 Ultra 256GBunified memory · 192 GB usable	Fits	up to 310	1200 GB/s	$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB, source, 2026-11-12	—
Mac Studio M5 Ultra 512GBunified memory · 384 GB usable	Fits	up to 310	1200 GB/s	$22,299Apple list price, M5 Ultra 36C CPU/80C GPU, 512GB/1TB, source, 2026-11-12	—
MacBook Pro M5 Max 128GBunified memory · 96 GB usable	Fits	up to 159	614 GB/s	$6,699Apple list price, 14-inch M5 Max 18C CPU/40C GPU, 128GB/2TB, source, 2026-11-12	—
NVIDIA DGX Spark 128GBunified memory · 96 GB usable	Fits	up to 71	273 GB/s	$6,950NVIDIA list price (not re-verified 2026-11-05: NVIDIA marketplace blocked/empty response; last confirmed $6,950 on marketplace listing; widely reported sold out in the US), source, 2026-10-22	—
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory · 96 GB usable	Fits	up to 66	256 GB/s	$3,449Framework list price, DIY Edition 128GB (pre-order, ships within 5 business days), source, 2026-11-05	—

All devices by memory bandwidth

Bandwidth is the main speed limit for local LLM decoding. The Qwen3-8B column is a reference point (~5.0 GB of weights at Q4); bigger models scale down roughly in proportion to their size.

Devices ranked by memory bandwidth
Device	Bandwidth	Usable memory	Qwen3-8B tok/s	Price
GeForce RTX 5090GPU	1792 GB/s	32 GB	217–307	$1,999
NVIDIA RTX Pro 6000 Blackwell 96GBGPU	1792 GB/s	96 GB	217–307	$8,565
Mac Studio M5 Ultra 256GBunified memory	1200 GB/s	192 GB	109–193	$9,499
Mac Studio M5 Ultra 512GBunified memory	1200 GB/s	384 GB	109–193	$22,299
GeForce RTX 3090 TiGPU	1008 GB/s	24 GB	122–173	$1,999
GeForce RTX 4090GPU	1008 GB/s	24 GB	122–173	$1,599
GeForce RTX 5080GPU	960 GB/s	16 GB	116–164	$999
Radeon RX 7900 XTXGPU	960 GB/s	24 GB	116–164	$999
NVIDIA RTX 6000 Ada 48GBGPU	960 GB/s	48 GB	116–164	$6,799
GeForce RTX 3090GPU	936 GB/s	24 GB	113–160	$1,499
GeForce RTX 3080 TiGPU	912 GB/s	12 GB	110–156	$1,199
GeForce RTX 5070 TiGPU	896 GB/s	16 GB	108–153	$749
Radeon RX 7900 XTGPU	800 GB/s	20 GB	97–137	$899
GeForce RTX 3080 10GBGPU	760 GB/s	10 GB	92–130	$699
GeForce RTX 4080 SuperGPU	736 GB/s	16 GB	89–126	$999
GeForce RTX 4070 Ti SuperGPU	672 GB/s	16 GB	81–115	$799
GeForce RTX 5070GPU	672 GB/s	12 GB	81–115	$549
Radeon RX 9070GPU	640 GB/s	16 GB	77–110	$549
Radeon RX 9070 XTGPU	640 GB/s	16 GB	77–110	$599
Radeon RX 7800 XTGPU	624 GB/s	16 GB	75–107	$499
MacBook Pro M5 Max 128GBunified memory	614 GB/s	96 GB	56–99	$6,699
Intel Arc Pro B70 32GBGPU	608 GB/s	32 GB	73–104	$949
Radeon RX 7900 GREGPU	576 GB/s	16 GB	70–99	$549
GeForce RTX 4070GPU	504 GB/s	12 GB	61–86	$599
GeForce RTX 4070 SuperGPU	504 GB/s	12 GB	61–86	$599
Intel Arc B580GPU	456 GB/s	12 GB	55–78	$249
GeForce RTX 3060 TiGPU	448 GB/s	8 GB	54–77	$399
GeForce RTX 3070GPU	448 GB/s	8 GB	54–77	$499
GeForce RTX 5060GPU	448 GB/s	8 GB	54–77	$299
GeForce RTX 5060 Ti 8GBGPU	448 GB/s	8 GB	54–77	$379
GeForce RTX 5060 Ti 16GBGPU	448 GB/s	16 GB	54–77	$429
Radeon RX 7700 XTGPU	432 GB/s	12 GB	52–74	$449
Intel Arc B570GPU	380 GB/s	10 GB	46–65	$219
GeForce RTX 3060 12GBGPU	360 GB/s	12 GB	43–62	$329
NVIDIA RTX 4000 Ada 20GBGPU	360 GB/s	20 GB	43–62	$1,250
Radeon RX 9060 XT 8GBGPU	320 GB/s	8 GB	39–55	$299
Radeon RX 9060 XT 16GBGPU	320 GB/s	16 GB	39–55	$349
Mac mini M5 Pro 64GBunified memory	307 GB/s	48 GB	28–49	$3,199
GeForce RTX 4060 Ti 16GBGPU	288 GB/s	16 GB	35–49	$499
Radeon RX 7600 XTGPU	288 GB/s	16 GB	35–49	$329
Radeon RX 7600GPU	288 GB/s	8 GB	35–49	$269
NVIDIA DGX Spark 64GBunified memory	273 GB/s	48 GB	25–44	$4,999
NVIDIA DGX Spark 128GBunified memory	273 GB/s	96 GB	25–44	$6,950
GeForce RTX 4060GPU	272 GB/s	8 GB	33–47	$299
Framework Desktop (Ryzen AI Max+ 395) 128GBunified memory	256 GB/s	96 GB	23–41	$3,449
GeForce RTX 3050 8GBGPU	224 GB/s	8 GB	27–38	$249
NVIDIA RTX 2000 Ada 16GBGPU	224 GB/s	16 GB	27–38	$649
Mac mini M6 24GBunified memory	170 GB/s	18 GB	15–27	$1,299
Mac mini M6 32GBunified memory	170 GB/s	24 GB	15–27	$1,799
MacBook Pro M6 32GBunified memory	170 GB/s	24 GB	15–27	$2,399
Mac mini M6 16GBunified memory	153 GB/s	12 GB	14–25	$899

Have a card we don't list? The VRAM calculator checks fit for any memory size, and the planner estimates tok/s for multi-GPU builds.

Quick answers

How many tokens per second will a local LLM run at?: Decode speed is set mostly by memory bandwidth: every new token reads the whole model (or, for MoE models, the active experts) from memory. Our estimate is bandwidth ÷ Q4 weight size × an efficiency band of 60–85% for GPUs and 45–80% for unified memory, calibrated on public llama.cpp results. For example, Qwen3-8B (~5.0 GB at Q4) runs at an estimated 217–307 tok/s on the GeForce RTX 5090.
What is the fastest GPU for a 32B model?: Of the devices we track, the GeForce RTX 5090 (1792 GB/s) is fastest for Qwen3-32B at Q4: an estimated 54–77 tok/s. A 32B model needs ~23.8 GB at Q4: 24 GB cards hold it with short context only; 32 GB cards fit it with room for ~4k context.
How fast does a 70B model run locally?: Llama-3.3-70B-Instruct needs ~50.2 GB at Q4, so no single consumer GPU we track holds it. On a single device, the fastest we track is the NVIDIA RTX Pro 6000 Blackwell 96GB at an estimated 25–36 tok/s; the cheapest that fits, the Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449), manages about 3–5 tok/s. Two 24 GB GPUs also hold it, at roughly one card's speed (see our planner).
Why do MoE models like gpt-oss show "up to"?: A mixture-of-experts model reads only its active experts per token (5.1B of 116.8B for gpt-oss-120b), so it can run much faster than its total size suggests. Routing, attention and shared layers add cost we haven't calibrated, so we show only an upper bound. All of the model still has to fit in memory.
Is a Mac or a GPU faster for local LLMs?: For a model that fits in a GPU's VRAM, the GPU is usually faster per dollar because desktop cards have more bandwidth per dollar. Macs and unified-memory mini PCs win on capacity: they run models no single consumer card can hold. See our NVIDIA vs AMD vs Mac page.
Does context length change the speed?: Yes. These estimates are for short prompts. Longer contexts make each token read the KV cache too, so speed drops as the conversation grows, and prompt processing (prefill) is a separate, compute-bound step we don't estimate. See our method page.

Best GPU for local LLMs · Best Mac · NVIDIA vs AMD vs Mac · How much VRAM do you need? · GPU vs GPU