Compare AI PC builds side by side

Put up to three builds next to each other: total price, usable memory, $ per GB, bandwidth, estimated tokens per second, $ per tok/s, power, running cost and fit warnings, plus every model's fit. Pick recipes, or paste a share link from the planner.

Builds

Build Aor paste a planner link

Build Bor paste a planner link

Build Cor paste a planner link

WorkloadModel Quant Electricity $/kWh

Cheapest: C (RTX 5090 single-GPU build (32 GB)) at $4,533. Most memory: A (Dual used RTX 3090 build (48 GB) for 32B to 70B models) with 48.0 GB usable. Fastest on Qwen3-32B at Q4: C (RTX 5090 single-GPU build (32 GB)) (54–77 tok/s, estimate). Best $ per tok/s: C (RTX 5090 single-GPU build (32 GB)).

Head to head

Build comparison, best value in each row marked Best
Metric	A: Dual used RTX 3090 build (48 GB) for 32B to 70B models	B: Dual RX 7900 XTX build (48 GB VRAM)	C: RTX 5090 single-GPU build (32 GB)
Total price	$7,216	$5,134	$4,533 Best
Usable memory	48.0 GB Best	48.0 GB Best	32.0 GB
$ per usable GB	$150	$107 Best	$142
Memory bandwidth	936 GB/s per card	960 GB/s per card	1,792 GB/s Best
Qwen3-32B at Q4	Fits	Fits	Fits
Est. tok/s (Qwen3-32B)	28–40	29–41	54–77 Best
$ per tok/s	$211	$147	$69 Best
Models that fit at Q4	511 Best	511 Best	504 +3 tight
Biggest model that fits at Q4	AI21-Jamba2-Mini Best	AI21-Jamba2-Mini Best	IQuest-Coder-V1-40B-Loop-Instruct
Load power	1020 W	1030 W	790 W Best
Cost per hour at $0.18/kWh	$0.18	$0.19	$0.14 Best
PSU needed / headroom	1350 W headroom 174 W	1350 W headroom 161 W	1050 W headroom 173 W
Warnings	1	1	0 Best
GPU condition	Used	New	New

"Best" marks the leading build in each row (lowest price, power and $ figures; highest memory, bandwidth and speed). Tok/s = bandwidth ÷ weight size × an efficiency band calibrated on llama.cpp scoreboards (how we estimate speed); mixture-of-experts models show only an upper bound, so they get no $ per tok/s. Multi-GPU builds add memory, but speed is about one card's. Load power is a ceiling (GPU board power + CPU TDP + 150 W); "—" means a spec we could not source.

Parts

Parts in each build
Part	A: Dual used RTX 3090 build (48 GB) for 32B to 70B models	B: Dual RX 7900 XTX build (48 GB VRAM)	C: RTX 5090 single-GPU build (32 GB)
GPU	2× Used RTX 3090 24GB (ASUS TUF, "Like New")$4,080 · used listing	2× Radeon RX 7900 XTX$1,998 · list price (MSRP)	GeForce RTX 5090$1,999 · list price (MSRP)
CPU	AMD Ryzen 9 9950X$539 · Newegg price (sold by Newegg)	AMD Ryzen 9 9950X$539 · Newegg price (sold by Newegg)	AMD Ryzen 7 9700X$339 · Newegg price (sold by Newegg)
Motherboard	ASUS ProArt X870E-CREATOR WIFI$512 · Newegg price (sold by Newegg)	ASUS ProArt X870E-CREATOR WIFI$512 · Newegg price (sold by Newegg)	ASUS TUF GAMING B650E-PLUS WIFI$150 · Newegg price (sold by Newegg)
RAM	Corsair Vengeance 64GB (2x32) DDR5-6000 CL30$1,275 · Corsair store price	Corsair Vengeance 64GB (2x32) DDR5-6000 CL30$1,275 · Corsair store price	Corsair Vengeance 64GB (2x32) DDR5-6000 CL30$1,275 · Corsair store price
Power supply	Corsair HX1500i$320 · Corsair store price	Corsair HX1500i$320 · Corsair store price	Corsair HX1200i$280 · Corsair store price
Case	Corsair FRAME 5000D RS$160 · Corsair store price	Corsair FRAME 5000D RS$160 · Corsair store price	Corsair FRAME 5000D RS$160 · Corsair store price
Storage	Samsung 990 2TB NVMe$330 · Newegg price (sold by Newegg)	Samsung 990 2TB NVMe$330 · Newegg price (sold by Newegg)	Samsung 990 2TB NVMe$330 · Newegg price (sold by Newegg)

We may earn a commission when you buy through these links, at no extra cost to you. Prices are dated snapshots (2026-10-23 for parts; GPUs at list price (MSRP) unless marked used; used prices are one dated listing each).

Fit checks

A: Dual used RTX 3090 build (48 GB) for 32B to 70B models

PSU: 1500 W vs. 1326 W recommended ((GPU 700 W + CPU 170 W + 150 W) × 1.3). Headroom 174 W.
Power connectors: need 4× PCIe 8-pin; PSU has 2× 12V-2x6 + 5× 8-pin.
Length: 313 mm card vs. 450 mm case limit.
Expansion slots: 2 cards × 3-slot needs about 6; case has 8.
Motherboard: 2 GPUs on 2 x16-size slots (2x PCIe 5.0 x16 (CPU; x16 or x8/x8)).
Slot spacing: check two 3-slot cards physically fit side by side on this board. We don’t model slot positions.

B: Dual RX 7900 XTX build (48 GB VRAM)

PSU: 1500 W vs. 1339 W recommended ((GPU 710 W + CPU 170 W + 150 W) × 1.3). Headroom 161 W.
Power connectors: need 4× PCIe 8-pin; PSU has 2× 12V-2x6 + 5× 8-pin.
Length: 287 mm card vs. 450 mm case limit.
Expansion slots: 2 cards × 2.5-slot needs about 6; case has 8.
Motherboard: 2 GPUs on 2 x16-size slots (2x PCIe 5.0 x16 (CPU; x16 or x8/x8)).
Slot spacing: check two 2.5-slot cards physically fit side by side on this board. We don’t model slot positions.

C: RTX 5090 single-GPU build (32 GB)

PSU: 1200 W vs. 1027 W recommended ((GPU 575 W + CPU 65 W + 150 W) × 1.3). Headroom 173 W.
Power connectors: need 1× 12V-2x6 cable + 0× PCIe 8-pin; PSU has 2× 12V-2x6 + 5× 8-pin.
Length: 304 mm card vs. 450 mm case limit.
Expansion slots: 1 card × 2-slot needs about 2; case has 8.

Every model on each build

At the quant above, about 4k context. Max context isn't modeled yet (no sourced KV-cache data).

Every tracked model at Q4 on each build
Model	A: Dual used RTX 3090 build (48 GB) for 32B to 70B models	B: Dual RX 7900 XTX build (48 GB VRAM)	C: RTX 5090 single-GPU build (32 GB)
Kimi-K3.1-Pro1939.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K31939.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3.8-2.4T-A95B1706.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
LongCat-2.51241.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
LongCat-2.01238.9 GB at Q4	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V4-Pro-08131151.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V4-Pro1115.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2.5716.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2.6716.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2.7-Code716.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2-Thinking716.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2-Instruct-0905716.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kimi-K2-Instruct716.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.6-Pro-RL715.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.6-Pro-MOPD715.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.5-Pro714.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ring-2.5-1T706.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-2.5-1T706.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-1T698.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ring-1T698.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Inkling665.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Intern-S1-Pro640.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Muse-Spark-Open-816B-A42B574.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Hy4-preview544.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-5526.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-5.1526.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-5.3526.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
GLM-5.2526.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-5.3-BF16526.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
K-EXAONE-2.0-750B-A37B523.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V3.2-Exp478.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V3.2-Speciale478.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V3.2478.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V3-0324478.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-R1-0528478.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V3.1-Base478.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-R1478.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-R1-Zero478.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Step-5-Preview422.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-3.1-flash399.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-3.1-flash-1019399.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
LongCat-Flash-Chat392.7 GB at Q4	Won’t fit	Won’t fit	Won’t fit
LongCat-Flash-Thinking392.7 GB at Q4	Won’t fit	Won’t fit	Won’t fit
LongCat-Flash-Thinking-2601392.7 GB at Q4	Won’t fit	Won’t fit	Won’t fit
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16391.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Beam-1.0-501B-A23B350.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-Coder-480B-A35B-Instruct335.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M1-40k319.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M1-80k319.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M3298.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
ERNIE-4.5-VL-424B-A47B-PT296.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Intern-S2-397B282.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3.5-397B-A17B282.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Llama-4-Maverick-17B-128E-Instruct281.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Trinity-Large-Thinking278.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Trinity-Large-Preview278.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Trinity-Large-TrueBase278.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Trinity-Large-Base278.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Nex-N2.5-Pro277.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Nex-N2-Pro277.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
NorthCUA-Pro277.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-4.5250.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-4.7250.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-4.6249.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-5.3-Flash225.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Motif-3220.5 GB at Q4	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.5217.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.6-Flash-RL217.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2.6-Flash-MOPD217.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiMo-V2-Flash217.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V4-Flash-Vision-Exp213.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V4-Flash-0731213.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
ERNIE-4.5-300B-A47B-PT210.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Hy3209.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Hy3-preview209.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-plus205.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V4-Flash203.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MemT-1.0-Flash201.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
ERNIE-5.5-Flash-Base200.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Inkling-Small186.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Llama-3_1-Nemotron-Ultra-253B-v1177.7 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Solar-Open2-250B175.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Intern-S1168.8 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
K-EXAONE-236B-A23B166.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V2.5165.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-Coder-V2-Instruct165.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V2.5-1210165.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-V2-Chat-0628165.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
DeepSeek-Coder-V2-Instruct-0724165.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-VL-235B-A22B-Instruct165.3 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-235B-A22B164.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-235B-A22B-Instruct-2507164.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-235B-A22B-Thinking-2507164.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M2160.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M2.5160.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M2.7160.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
MiniMax-M2.1160.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
command-a-plus-05-2026-bf16153.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Step-3.7-Flash141.4 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Step-3.5-Flash140.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Step-3.5-Flash-Base-Midtrain138.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3.8-Flash-Next126.5 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Mixtral-8x22B-Instruct-v0.199.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-3.0-flash-Sante89.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-3.0-flash89.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-3.0-flash-Fin89.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ornith-1.5-122B88.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Qwen3.5-122B-A10B88.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Devstral-2-123B-Instruct-251288.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Ling-3.0-flash-VL88.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
NVIDIA-Nemotron-3-Super-120B-A12B-BF1687.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Mistral-Small-4-119B-260384.2 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Youtu-LLM-A7B-Base83.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Laguna-S-2.183.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
gpt-oss-120b82.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
c4ai-command-a-03-202578.4 GB at Q4	Won’t fit	Won’t fit	Won’t fit
GLM-4.5-Air78.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Llama-4-Scout-17B-16E-Instruct76.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Llama-4-Scout-17B-16E76.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-4.5V76.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
GLM-4.6V76.1 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-2.6-flash75.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
sarvam-105b74.9 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ling-flash-2.072.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
LLaDA2.1-flash72.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Ring-flash-2.072.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
LLaDA2.2-flash72.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Solar-Open-100B72.6 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-Next-80B-A3B-Instruct57.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-Next-80B-A3B-Thinking57.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
AliceAI-Foundation-80B-A3B-Base57.7 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Hunyuan-A13B-Instruct57.0 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Qwen3-Coder-Next56.5 GB at Q4 · MoE	Won’t fit	Won’t fit	Won’t fit
Kolibri-155.5 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Qwen2.5-VL-72B-Instruct52.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
UI-TARS-72B-DPO52.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Qwen2.5-72B-Instruct51.7 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Apertus-70B-Instruct-250950.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Apertus-70B-250950.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Llama-3.3-70B-Instruct50.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
DeepSeek-R1-Distill-Llama-70B50.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Llama-3.1-70B50.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Hermes-2-Theta-Llama-3-70B50.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
Meta-Llama-3.1-70B-Instruct50.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
LongCat-Flash-Lite49.2 GB at Q4	Won’t fit	Won’t fit	Won’t fit
AI21-Jamba2-Mini37.0 GB at Q4 · MoE	Fitsup to 109 tok/s	Fitsup to 112 tok/s	Won’t fit
Llama-3_1-Nemotron-51B-Instruct36.9 GB at Q4	Fits18–25 tok/s	Fits18–26 tok/s	Won’t fit
Llama-3_3-Nemotron-Super-49B-v1_535.8 GB at Q4	Fits19–26 tok/s	Fits19–27 tok/s	Won’t fit
Llama-3_3-Nemotron-Super-49B-v135.8 GB at Q4	Fits19–26 tok/s	Fits19–27 tok/s	Won’t fit
Phi-3.5-MoE-instruct30.2 GB at Q4 · MoE	Fitsup to 199 tok/s	Fitsup to 204 tok/s	Tightup to 381 tok/s
GRIN-MoE30.2 GB at Q4 · MoE	Fitsup to 199 tok/s	Fitsup to 204 tok/s	Tightup to 381 tok/s
ALIA-40b29.2 GB at Q4	Fits23–32 tok/s	Fits23–33 tok/s	Tight44–62 tok/s
IQuest-Coder-V1-40B-Loop-Instruct28.7 GB at Q4	Fits23–33 tok/s	Fits24–34 tok/s	Fits45–63 tok/s
IQuest-Coder-V1-40B-Instruct28.7 GB at Q4	Fits23–33 tok/s	Fits24–34 tok/s	Fits45–63 tok/s
Recursive Flint28.7 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 539 tok/s	Fitsup to 1006 tok/s
K2-Horizon-MoVA-36B-A4B27.1 GB at Q4 · MoE	Fitsup to 328 tok/s	Fitsup to 337 tok/s	Fitsup to 628 tok/s
Seed-OSS-36B-Instruct26.2 GB at Q4	Fits26–36 tok/s	Fits26–37 tok/s	Fits49–69 tok/s
Seed-OSS-36B-Base-woSyn26.2 GB at Q4	Fits26–36 tok/s	Fits26–37 tok/s	Fits49–69 tok/s
Intern-S2-Preview26.2 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Ornith-1.5-35B-A3B26.1 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Sage-A1.526.1 GB at Q4	Fits26–37 tok/s	Fits26–37 tok/s	Fits49–70 tok/s
Qwen3.6-35B-A3B26.1 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Qwen3.5-35B-A3B26.1 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Apex-35B-V1.026.1 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
MiniCPM-V-4.7-35B-A3B25.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
MemT-1.0-Nano25.5 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Agents-A125.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Nex-N2.5-mini25.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Nex-N2-mini25.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Axion-1.0-35B25.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
NorthCUA-Mini25.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Clover-35B-A3B-Preview25.2 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen-AgentWorld-35B-A3B25.2 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
KAT-Coder-V2.5-Dev25.2 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
EXAONE-4.5-33B24.9 GB at Q4	Fits27–38 tok/s	Fits28–39 tok/s	Fits52–73 tok/s
Falcon-H1-34B-Instruct24.5 GB at Q4	Fits28–39 tok/s	Fits28–40 tok/s	Fits53–75 tok/s
Qwen2.5-VL-32B-Instruct24.3 GB at Q4	Fits28–39 tok/s	Fits28–40 tok/s	Fits53–75 tok/s
Laguna-XS.224.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Qwen3-VL-32B-Instruct24.3 GB at Q4	Fits28–39 tok/s	Fits28–40 tok/s	Fits53–75 tok/s
HyperCLOVAX-SEED-Think-32B24.2 GB at Q4	Fits28–39 tok/s	Fits29–40 tok/s	Fits53–75 tok/s
FlexOlmo-7x7B-1T24.2 GB at Q4	Fits28–39 tok/s	Fits29–40 tok/s	Fits53–76 tok/s
GLM-Z1-Rumination-32B-041424.1 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–76 tok/s
Nanbeige4.3-33B24.1 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–76 tok/s
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1624.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen2.5-Coder-32B-Instruct23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
DeepSeek-R1-Distill-Qwen-32B23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
Qwen2.5-32B-Instruct23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
QwQ-32B23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
QwQ-32B-Preview23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
Qwen2.5-32B23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
Qwen2.5-Coder-32B23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
Qwen3-32B23.8 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
GLM-Z1-32B-041423.7 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
GLM-4-32B-041423.7 GB at Q4	Fits28–40 tok/s	Fits29–41 tok/s	Fits54–77 tok/s
OLMo-2-0325-32B-Instruct23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
OLMo-2-0325-32B23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
Olmo-3-32B-Think23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
Olmo-3-1125-32B23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
Olmo-3.1-32B-Instruct23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
Olmo-3.1-32B-Think23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
Olmo-3.1-32B-Instruct-DPO23.5 GB at Q4	Fits29–41 tok/s	Fits29–42 tok/s	Fits55–78 tok/s
granite-4.0-h-small23.5 GB at Q4 · MoE	Fitsup to 146 tok/s	Fitsup to 150 tok/s	Fitsup to 279 tok/s
sarvam-30b23.4 GB at Q4 · MoE	Fitsup to 547 tok/s	Fitsup to 561 tok/s	Fitsup to 1047 tok/s
EXAONE-4.0-32B23.3 GB at Q4	Fits29–41 tok/s	Fits30–42 tok/s	Fits55–79 tok/s
EXAONE-4.0.1-32B23.3 GB at Q4	Fits29–41 tok/s	Fits30–42 tok/s	Fits55–79 tok/s
EXAONE-3.5-32B-Instruct23.3 GB at Q4	Fits29–41 tok/s	Fits30–42 tok/s	Fits55–79 tok/s
EXAONE-Deep-32B23.3 GB at Q4	Fits29–41 tok/s	Fits30–42 tok/s	Fits55–79 tok/s
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1623.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1623.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Nemotron-Cascade-2-30B-A3B23.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Gemma 4 31B22.8 GB at Q4	Fits30–42 tok/s	Fits30–43 tok/s	Fits57–80 tok/s
gemma-4-31B22.8 GB at Q4	Fits30–42 tok/s	Fits30–43 tok/s	Fits57–80 tok/s
GLM-4.7-Flash22.8 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
Xing4.0-29B-A4B22.8 GB at Q4 · MoE	Fitsup to 328 tok/s	Fitsup to 337 tok/s	Fitsup to 628 tok/s
Xing4.0-29B-Intent22.8 GB at Q4 · MoE	Fitsup to 328 tok/s	Fitsup to 337 tok/s	Fitsup to 628 tok/s
Qwen3-VL-30B-A3B-Instruct22.7 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen3-VL-30B-A3B-Thinking22.7 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
UI-Venus-1.5-30B-A3B22.7 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
kanana-2-30b-a3b-instruct-260122.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
kanana-2-30b-a3b-instruct22.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
kanana-2-30b-a3b-thinking-260122.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Qwen3-30B-A3B22.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen3-Coder-30B-A3B-Instruct22.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen3-30B-A3B-Instruct-250722.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Qwen3-30B-A3B-Thinking-250722.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Tongyi-DeepResearch-30B-A3B22.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
Rubicon-Preview22.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 448 tok/s	Fitsup to 837 tok/s
North-Mini-Code-1.022.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Hy-Memory-30B-A3B22.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
Hy-MT2-30B-A3B22.0 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
granite-switch-4.1-30b21.9 GB at Q4	Fits31–44 tok/s	Fits32–45 tok/s	Fits59–84 tok/s
Muse-Glimmer-30B21.8 GB at Q4	Fits31–44 tok/s	Fits32–45 tok/s	Fits60–84 tok/s
ERNIE-4.5-VL-28B-A3B-Thinking21.7 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
ERNIE-4.5-VL-28B-A3B-PT21.5 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
granite-4.2-30b21.4 GB at Q4	Fits32–45 tok/s	Fits32–46 tok/s	Fits61–86 tok/s
granite-4.1-30b21.1 GB at Q4	Fits32–45 tok/s	Fits33–47 tok/s	Fits61–87 tok/s
Raon-1 mini preview21.1 GB at Q4 · MoE	Fitsup to 305 tok/s	Fitsup to 313 tok/s	Fitsup to 584 tok/s
Qwen3.8-27B20.4 GB at Q4	Fits33–47 tok/s	Fits34–48 tok/s	Fits64–90 tok/s
Tess-4.5-27B20.4 GB at Q4	Fits33–47 tok/s	Fits34–48 tok/s	Fits64–90 tok/s
Qwen3.6-27B20.4 GB at Q4	Fits33–47 tok/s	Fits34–48 tok/s	Fits64–90 tok/s
Qwen3.5-27B20.4 GB at Q4	Fits33–47 tok/s	Fits34–48 tok/s	Fits64–90 tok/s
gemma-3-27b-it20.1 GB at Q4	Fits34–48 tok/s	Fits35–49 tok/s	Fits65–92 tok/s
Arete-27B20.1 GB at Q4	Fits34–48 tok/s	Fits35–49 tok/s	Fits65–92 tok/s
gemma-2-27b-it20.0 GB at Q4	Fits34–48 tok/s	Fits35–49 tok/s	Fits65–92 tok/s
gemma-2-27b20.0 GB at Q4	Fits34–48 tok/s	Fits35–49 tok/s	Fits65–92 tok/s
Qwen3.8-27B-Longbook27-v219.9 GB at Q4	Fits34–48 tok/s	Fits35–50 tok/s	Fits66–93 tok/s
medgemma-27b-text-it19.8 GB at Q4	Fits34–49 tok/s	Fits35–50 tok/s	Fits66–93 tok/s
Hemmingway-1.519.8 GB at Q4	Fits34–49 tok/s	Fits35–50 tok/s	Fits66–93 tok/s
Trinity-Mini19.2 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
diffusiongemma-26B-A4B-it19.0 GB at Q4 · MoE	Fitsup to 328 tok/s	Fitsup to 336 tok/s	Fitsup to 628 tok/s
Gemma 4 26B A4B19.0 GB at Q4 · MoE	Fitsup to 345 tok/s	Fitsup to 354 tok/s	Fitsup to 661 tok/s
gemma-4-26B-A4B19.0 GB at Q4 · MoE	Fitsup to 328 tok/s	Fitsup to 337 tok/s	Fitsup to 628 tok/s
Voxtral-Small-24B-250717.9 GB at Q4	Fits38–54 tok/s	Fits39–55 tok/s	Fits73–104 tok/s
Mistral-Small-3.2-24B-Instruct-250617.7 GB at Q4	Fits39–55 tok/s	Fits40–56 tok/s	Fits74–105 tok/s
Mistral-Small-3.1-24B-Instruct-250317.7 GB at Q4	Fits39–55 tok/s	Fits40–56 tok/s	Fits74–105 tok/s
LFM2.5-24B-A2B17.7 GB at Q4 · MoE	Fitsup to 656 tok/s	Fitsup to 673 tok/s	Fitsup to 1257 tok/s
LFM2-24B-A2B17.6 GB at Q4 · MoE	Fitsup to 656 tok/s	Fitsup to 673 tok/s	Fitsup to 1256 tok/s
Mistral-Small-24B-Instruct-250117.4 GB at Q4	Fits39–56 tok/s	Fits40–57 tok/s	Fits75–107 tok/s
EuroLLM-22B-Instruct-251216.8 GB at Q4	Fits41–58 tok/s	Fits42–59 tok/s	Fits78–111 tok/s
EuroLLM-22B-251216.8 GB at Q4	Fits41–58 tok/s	Fits42–59 tok/s	Fits78–111 tok/s
solar-pro-preview-instruct16.4 GB at Q4	Fits42–59 tok/s	Fits43–61 tok/s	Fits80–113 tok/s
ERNIE-4.5-21B-A3B-PT16.3 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
ERNIE-4.5-21B-A3B-Thinking16.2 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
gpt-oss-20b15.6 GB at Q4 · MoE	Fitsup to 365 tok/s	Fitsup to 374 tok/s	Fitsup to 698 tok/s
reka-flash-315.6 GB at Q4	Fits44–63 tok/s	Fits45–64 tok/s	Fits85–120 tok/s
maple-preview15.1 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1345 tok/s	Fitsup to 2511 tok/s
internlm2_5-20b-chat14.8 GB at Q4	Fits47–66 tok/s	Fits48–68 tok/s	Fits89–127 tok/s
Groot-19B-A3B-26102314.5 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
Ling-Coder-lite12.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ring-lite-distill-preview12.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ling-lite12.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ling-lite-1.5-250712.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ling-lite-1.512.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ring-lite12.7 GB at Q4 · MoE	Fitsup to 477 tok/s	Fitsup to 489 tok/s	Fitsup to 913 tok/s
Ring-mini-linear-2.012.5 GB at Q4 · MoE	Fitsup to 820 tok/s	Fitsup to 841 tok/s	Fitsup to 1571 tok/s
Kimi-VL-A3B-Instruct12.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Kimi-VL-A3B-Thinking12.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Kimi-VL-A3B-Thinking-250612.4 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
LLaDA2.1-mini12.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
LLaDA2.0-mini12.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
Ling-mini-2.012.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
Ring-mini-2.012.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
LLaDA2.2-mini12.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
LLaDA2.0-mini-preview12.3 GB at Q4 · MoE	Fitsup to 938 tok/s	Fitsup to 962 tok/s	Fitsup to 1795 tok/s
Moonlight-16B-A3B-Instruct12.1 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
Moonlight-16B-A3B12.1 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 837 tok/s
Instella-MoE-16B-A3B-Think12.1 GB at Q4 · MoE	Fitsup to 437 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
DeepSeek-Coder-V2-Lite-Instruct12.0 GB at Q4 · MoE	Fitsup to 547 tok/s	Fitsup to 561 tok/s	Fitsup to 1047 tok/s
kanana-1.5-15.7b-a3b-instruct11.9 GB at Q4 · MoE	Fitsup to 438 tok/s	Fitsup to 449 tok/s	Fitsup to 838 tok/s
Phi-4-reasoning-vision-15B11.5 GB at Q4	Fits61–87 tok/s	Fits63–89 tok/s	Fits117–166 tok/s
Apriel-Nemotron-15b-Thinker11.4 GB at Q4	Fits62–88 tok/s	Fits63–90 tok/s	Fits118–168 tok/s
Apriel-1.5-15b-Thinker11.4 GB at Q4	Fits62–88 tok/s	Fits64–91 tok/s	Fits119–169 tok/s
Apriel-1.6-15b-Thinker11.4 GB at Q4	Fits62–88 tok/s	Fits64–91 tok/s	Fits119–169 tok/s
Carbon-X (DNA)11.3 GB at Q4	Fits62–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen2.5-Coder-14B-Instruct11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
DeepSeek-R1-Distill-Qwen-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen2.5-14B-Instruct11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen2.5-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen2.5-14B-Instruct-1M11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen2.5-Coder-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
Qwen3-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
JanusCoder-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
HyperCLOVAX-SEED-Think-14B11.3 GB at Q4	Fits63–89 tok/s	Fits64–91 tok/s	Fits120–170 tok/s
phi-411.2 GB at Q4	Fits63–90 tok/s	Fits65–92 tok/s	Fits121–171 tok/s
Phi-4-reasoning11.2 GB at Q4	Fits63–90 tok/s	Fits65–92 tok/s	Fits121–171 tok/s
Phi-4-reasoning-plus11.2 GB at Q4	Fits63–90 tok/s	Fits65–92 tok/s	Fits121–171 tok/s
OLMo-2-1124-13B-Instruct10.6 GB at Q4	Fits68–96 tok/s	Fits69–98 tok/s	Fits129–183 tok/s
Mistral-Nemo-Instruct-24079.5 GB at Q4	Fits76–107 tok/s	Fits78–110 tok/s	Fits145–205 tok/s
gemma-3-12b-it9.5 GB at Q4	Fits76–108 tok/s	Fits78–110 tok/s	Fits146–206 tok/s
gemma-3-12b-pt9.5 GB at Q4	Fits76–108 tok/s	Fits78–110 tok/s	Fits146–206 tok/s
Mellum2.1-12B-A2.5B-Thinking9.5 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 538 tok/s	Fitsup to 1005 tok/s
Mellum2-12B-A2.5B-Thinking9.5 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 538 tok/s	Fitsup to 1005 tok/s
Mellum2-12B-A2.5B-Instruct9.5 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 538 tok/s	Fitsup to 1005 tok/s
Mellum2-12B-A2.5B-Thinking-SFT9.5 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 538 tok/s	Fitsup to 1005 tok/s
Mellum2.1-12B-A2.5B-Thinking-MTP9.5 GB at Q4 · MoE	Fitsup to 525 tok/s	Fitsup to 538 tok/s	Fitsup to 1005 tok/s
Gemma 4 12B9.3 GB at Q4	Fits77–110 tok/s	Fits79–113 tok/s	Fits148–210 tok/s
HyperCLOVAX-SEED-Omni-8B8.5 GB at Q4	Fits86–122 tok/s	Fits88–125 tok/s	Fits165–234 tok/s
Llama-3.2-11B-Vision-Instruct8.4 GB at Q4	Fits87–123 tok/s	Fits89–126 tok/s	Fits166–235 tok/s
Llama-3.2-11B-Vision8.4 GB at Q4	Fits87–123 tok/s	Fits89–126 tok/s	Fits167–236 tok/s
Falcon3-10B-Instruct8.2 GB at Q4	Fits90–127 tok/s	Fits92–131 tok/s	Fits172–244 tok/s
GLM-4.1V-9B-Thinking8.2 GB at Q4	Fits90–128 tok/s	Fits92–131 tok/s	Fits172–244 tok/s
GLM-4.6V-Flash8.2 GB at Q4	Fits90–128 tok/s	Fits92–131 tok/s	Fits172–244 tok/s
GLM-4.1V-9B-Base8.2 GB at Q4	Fits90–128 tok/s	Fits92–131 tok/s	Fits172–244 tok/s
Glyph8.2 GB at Q4	Fits90–128 tok/s	Fits92–131 tok/s	Fits172–244 tok/s
INTELLECT-1-Instruct8.1 GB at Q4	Fits91–129 tok/s	Fits93–132 tok/s	Fits174–246 tok/s
INTELLECT-18.1 GB at Q4	Fits91–129 tok/s	Fits93–132 tok/s	Fits174–246 tok/s
ZAYA1-VL-8B7.8 GB at Q4	Fits95–135 tok/s	Fits98–138 tok/s	Fits182–258 tok/s
SenseNova-Loop-9B-Preview7.7 GB at Q4	Fits96–136 tok/s	Fits98–139 tok/s	Fits184–260 tok/s
Ornith-1.5-9B7.7 GB at Q4	Fits96–136 tok/s	Fits98–139 tok/s	Fits184–260 tok/s
Qwen3.5-9B7.7 GB at Q4	Fits96–136 tok/s	Fits98–139 tok/s	Fits184–260 tok/s
MiniCPM-SALA7.6 GB at Q4	Fits98–138 tok/s	Fits100–142 tok/s	Fits187–265 tok/s
NorthCUA-Nano7.6 GB at Q4	Fits98–139 tok/s	Fits101–143 tok/s	Fits188–267 tok/s
GLM-4-9B-04147.6 GB at Q4	Fits99–140 tok/s	Fits101–143 tok/s	Fits189–267 tok/s
GLM-Z1-9B-04147.6 GB at Q4	Fits99–140 tok/s	Fits101–143 tok/s	Fits189–267 tok/s
codegeex4-all-9b7.6 GB at Q4	Fits99–140 tok/s	Fits101–143 tok/s	Fits189–267 tok/s
glm-4-9b-chat-hf7.6 GB at Q4	Fits99–140 tok/s	Fits101–143 tok/s	Fits189–267 tok/s
moondream3-preview7.5 GB at Q4	Fits100–142 tok/s	Fits103–145 tok/s	Fits191–271 tok/s
gemma-2-9b-it7.4 GB at Q4	Fits100–142 tok/s	Fits103–146 tok/s	Fits192–272 tok/s
gemma-2-9b7.4 GB at Q4	Fits100–142 tok/s	Fits103–146 tok/s	Fits192–272 tok/s
Gemma-SEA-LION-v3-9B-IT7.4 GB at Q4	Fits100–142 tok/s	Fits103–146 tok/s	Fits192–272 tok/s
EuroLLM-9B-Instruct-25127.4 GB at Q4	Fits101–143 tok/s	Fits104–147 tok/s	Fits194–275 tok/s
NVIDIA-Nemotron-Nano-9B-v27.2 GB at Q4	Fits104–148 tok/s	Fits107–151 tok/s	Fits200–283 tok/s
ZAYA1-base7.2 GB at Q4	Fits105–148 tok/s	Fits107–152 tok/s	Fits201–284 tok/s
Yi-Coder-9B-Chat7.2 GB at Q4	Fits105–149 tok/s	Fits108–152 tok/s	Fits201–285 tok/s
Yi-Coder-9B7.2 GB at Q4	Fits105–149 tok/s	Fits108–152 tok/s	Fits201–285 tok/s
internlm3-8b-instruct7.1 GB at Q4	Fits105–149 tok/s	Fits108–153 tok/s	Fits201–285 tok/s
granite-4.1-8b7.1 GB at Q4	Fits105–149 tok/s	Fits108–153 tok/s	Fits202–286 tok/s
granite-4.2-8b7.1 GB at Q4	Fits105–149 tok/s	Fits108–153 tok/s	Fits202–286 tok/s
granite-switch-4.1-8b7.1 GB at Q4	Fits106–150 tok/s	Fits108–153 tok/s	Fits202–286 tok/s
Qwen3-VL-8B-Instruct7.1 GB at Q4	Fits106–150 tok/s	Fits108–154 tok/s	Fits202–287 tok/s
Qwen3-VL-8B-Thinking7.1 GB at Q4	Fits106–150 tok/s	Fits108–154 tok/s	Fits202–287 tok/s
UI-Venus-1.5-8B7.1 GB at Q4	Fits106–150 tok/s	Fits108–154 tok/s	Fits202–287 tok/s
Llama-3.1-Nemotron-Nano-VL-8B-V17.1 GB at Q4	Fits106–150 tok/s	Fits109–154 tok/s	Fits203–288 tok/s
MiniCPM-V-4_57.1 GB at Q4	Fits107–151 tok/s	Fits109–155 tok/s	Fits204–289 tok/s
aya-vision-8b7.0 GB at Q4	Fits107–152 tok/s	Fits110–156 tok/s	Fits205–291 tok/s
Intern-S1-mini7.0 GB at Q4	Fits108–154 tok/s	Fits111–158 tok/s	Fits208–294 tok/s
Nemotron-Labs-Diffusion-8B-Base6.9 GB at Q4	Fits109–155 tok/s	Fits112–159 tok/s	Fits209–296 tok/s
LFM2.5-8B-A1B6.9 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2512 tok/s
Mistral-NeMo-Minitron-8B-Base6.9 GB at Q4	Fits110–156 tok/s	Fits113–160 tok/s	Fits211–299 tok/s
LFM2-8B-A1B6.8 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2513 tok/s
rnj-1-instruct6.8 GB at Q4	Fits111–158 tok/s	Fits114–162 tok/s	Fits213–302 tok/s
MiMo-VL-7B-RL-25086.8 GB at Q4	Fits112–158 tok/s	Fits114–162 tok/s	Fits214–302 tok/s
MiMo-VL-7B-RL6.8 GB at Q4	Fits112–158 tok/s	Fits114–162 tok/s	Fits214–302 tok/s
MiMo-Embodied-7B6.8 GB at Q4	Fits112–158 tok/s	Fits114–162 tok/s	Fits214–302 tok/s
MiMo-VL-7B-SFT6.8 GB at Q4	Fits112–158 tok/s	Fits114–162 tok/s	Fits214–302 tok/s
Qwen2.5-VL-7B-Instruct6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
UI-TARS-1.5-7B6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
Fara-7B6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
UI-Venus-Ground-7B6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
Qwen2-VL-7B-Instruct6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
UI-TARS-7B-DPO6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
UI-TARS-7B-SFT6.8 GB at Q4	Fits112–158 tok/s	Fits115–162 tok/s	Fits214–303 tok/s
Seed-Coder-8B-Instruct6.8 GB at Q4	Fits112–159 tok/s	Fits115–163 tok/s	Fits215–305 tok/s
Seed-Coder-8B-Reasoning6.8 GB at Q4	Fits112–159 tok/s	Fits115–163 tok/s	Fits215–305 tok/s
Stable-DiffCoder-8B-Instruct6.8 GB at Q4	Fits112–159 tok/s	Fits115–163 tok/s	Fits215–305 tok/s
Qwen3-8B6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
DeepSeek-R1-0528-Qwen3-8B6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
Qwen3-8B-Base6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
WeDLM-8B-Instruct6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
PaCoRe-8B6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
MiniCPM4-8B6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
MiniCPM4.1-8B6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
AgentCPM-Report6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
MiniCPM4-MCP6.7 GB at Q4	Fits113–160 tok/s	Fits116–164 tok/s	Fits217–307 tok/s
granite-3.3-8b-instruct6.7 GB at Q4	Fits113–161 tok/s	Fits116–165 tok/s	Fits217–307 tok/s
granite-3.1-8b-instruct6.7 GB at Q4	Fits113–161 tok/s	Fits116–165 tok/s	Fits217–307 tok/s
granite-3.2-8b-instruct6.7 GB at Q4	Fits113–161 tok/s	Fits116–165 tok/s	Fits217–307 tok/s
granite-3.0-8b-instruct6.7 GB at Q4	Fits113–161 tok/s	Fits116–165 tok/s	Fits217–307 tok/s
Apertus-8B-Instruct-25096.6 GB at Q4	Fits115–163 tok/s	Fits118–167 tok/s	Fits220–312 tok/s
Apertus-8B-25096.6 GB at Q4	Fits115–163 tok/s	Fits118–167 tok/s	Fits220–312 tok/s
Llama-3.1-Tulu-3-8B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Llama-3.1-Tulu-3-8B-DPO6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Llama-3.1-Tulu-3.1-8B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
kanana-1.5-8b-instruct-25056.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Llama-3.1-8B-Instruct6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Llama-3.1-8B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
DeepSeek-R1-Distill-Llama-8B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Meta-Llama-3.1-8B-Instruct6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
L3-8B-Stheno-v3.26.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Llama-3.1-Nemotron-Nano-8B-v16.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Meta-Llama-3.1-8B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
LongWriter-llama3.1-8b6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Hy-MT2-7B6.6 GB at Q4	Fits115–163 tok/s	Fits118–168 tok/s	Fits221–313 tok/s
Nemotron 3.5 Bolt E4B6.6 GB at Q4	Fits116–164 tok/s	Fits119–168 tok/s	Fits222–314 tok/s
Gemma 4 E4B6.6 GB at Q4	Fits116–164 tok/s	Fits119–168 tok/s	Fits222–314 tok/s
Ling-3.0-tiny6.5 GB at Q4 · MoE	Fitsup to 1009 tok/s	Fitsup to 1035 tok/s	Fitsup to 1933 tok/s
MiMo-7B-RL6.5 GB at Q4	Fits118–168 tok/s	Fits121–172 tok/s	Fits226–321 tok/s
MiMo-7B-Base6.5 GB at Q4	Fits118–168 tok/s	Fits121–172 tok/s	Fits226–321 tok/s
MiMo-7B-RL-05306.5 GB at Q4	Fits118–168 tok/s	Fits121–172 tok/s	Fits226–321 tok/s
MiMo-7B-SFT6.5 GB at Q4	Fits118–168 tok/s	Fits121–172 tok/s	Fits226–321 tok/s
EXAONE-3.5-7.8B-Instruct6.5 GB at Q4	Fits118–168 tok/s	Fits122–172 tok/s	Fits227–321 tok/s
EXAONE-Deep-7.8B6.5 GB at Q4	Fits118–168 tok/s	Fits122–172 tok/s	Fits227–321 tok/s
salamandra-7b-instruct6.4 GB at Q4	Fits119–169 tok/s	Fits122–173 tok/s	Fits228–323 tok/s
salamandra-7b6.4 GB at Q4	Fits119–169 tok/s	Fits122–173 tok/s	Fits228–323 tok/s
salamandra-7b-instruct-26066.4 GB at Q4	Fits119–169 tok/s	Fits122–173 tok/s	Fits228–323 tok/s
FastVLM-7B6.4 GB at Q4	Fits119–169 tok/s	Fits122–173 tok/s	Fits228–324 tok/s
Molmo2-O-7B6.4 GB at Q4	Fits119–169 tok/s	Fits122–173 tok/s	Fits229–324 tok/s
internlm2_5-7b-chat6.4 GB at Q4	Fits120–170 tok/s	Fits123–174 tok/s	Fits229–325 tok/s
internlm2_5-7b-chat-1m6.4 GB at Q4	Fits120–170 tok/s	Fits123–174 tok/s	Fits229–325 tok/s
Phi-mini-MoE-instruct6.3 GB at Q4 · MoE	Fitsup to 547 tok/s	Fitsup to 561 tok/s	Fitsup to 1047 tok/s
Qwen2.5-Coder-7B-Instruct6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
DeepSeek-R1-Distill-Qwen-7B6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-7B-Instruct6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-7B6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2-7B-Instruct6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-Coder-7B6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2-7B6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-7B-Instruct-1M6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-Math-7B6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Qwen2.5-Math-7B-Instruct6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Marco-o16.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
SeaLLMs-v3-7B-Chat6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Arcee-Spark6.3 GB at Q4	Fits122–172 tok/s	Fits125–177 tok/s	Fits233–330 tok/s
Falcon-H1-7B-Instruct6.3 GB at Q4	Fits122–173 tok/s	Fits125–177 tok/s	Fits234–331 tok/s
Falcon-H1R-7B6.3 GB at Q4	Fits122–173 tok/s	Fits125–177 tok/s	Fits234–331 tok/s
starvector-8b-im2svg6.2 GB at Q4	Fits123–175 tok/s	Fits127–179 tok/s	Fits236–335 tok/s
Falcon3-7B-Instruct6.2 GB at Q4	Fits124–176 tok/s	Fits127–181 tok/s	Fits238–337 tok/s
Olmo-Hybrid-7B6.2 GB at Q4	Fits125–177 tok/s	Fits128–181 tok/s	Fits239–338 tok/s
LLaDA-MoE-7B-A1B-Base6.1 GB at Q4 · MoE	Fitsup to 1313 tok/s	Fitsup to 1346 tok/s	Fitsup to 2513 tok/s
Zamba2-7B-Instruct6.1 GB at Q4	Fits126–178 tok/s	Fits129–183 tok/s	Fits241–342 tok/s
OLMo-2-1124-7B-Instruct6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Think6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Instruct6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-1025-7B6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Instruct-SFT6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Think-SFT6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Instruct-DPO6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-RL-Zero-Math6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3.1-7B-RL-Zero-Math6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Olmo-3-7B-Think-DPO6.1 GB at Q4	Fits127–180 tok/s	Fits130–184 tok/s	Fits243–344 tok/s
Mistral-7B-Instruct-v0.36.1 GB at Q4	Fits128–181 tok/s	Fits131–186 tok/s	Fits245–347 tok/s
granite-4.0-h-tiny5.8 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2513 tok/s
OLMoE-1B-7B-0924-Instruct5.8 GB at Q4 · MoE	Fitsup to 1009 tok/s	Fitsup to 1035 tok/s	Fitsup to 1933 tok/s
OLMoE-1B-7B-09245.8 GB at Q4 · MoE	Fitsup to 1009 tok/s	Fitsup to 1035 tok/s	Fitsup to 1933 tok/s
OLMoE-1B-7B-0125-Instruct5.8 GB at Q4 · MoE	Fitsup to 1009 tok/s	Fitsup to 1035 tok/s	Fitsup to 1933 tok/s
OLMoE-1B-7B-01255.8 GB at Q4 · MoE	Fitsup to 1009 tok/s	Fitsup to 1035 tok/s	Fitsup to 1933 tok/s
OLMo-7B-0724-hf5.8 GB at Q4	Fits134–191 tok/s	Fits138–195 tok/s	Fits257–365 tok/s
granite-4.0-tiny-preview5.7 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2512 tok/s
YuE-s1-7B-anneal-en-cot5.3 GB at Q4	Fits149–211 tok/s	Fits153–216 tok/s	Fits285–404 tok/s
Trinity-Nano-Preview5.3 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2512 tok/s
Trinity-Nano-Base5.3 GB at Q4 · MoE	Fitsup to 1312 tok/s	Fitsup to 1346 tok/s	Fitsup to 2512 tok/s
gemma-3n-E2B-it4.8 GB at Q4	Fits170–241 tok/s	Fits175–247 tok/s	Fits326–462 tok/s
Youtu-VL-4B-Instruct4.7 GB at Q4	Fits173–246 tok/s	Fits178–252 tok/s	Fits332–470 tok/s
Gemma 4 E2B4.6 GB at Q4	Fits181–256 tok/s	Fits185–263 tok/s	Fits346–490 tok/s
Molmo2-ER4.4 GB at Q4	Fits191–271 tok/s	Fits196–277 tok/s	Fits366–518 tok/s
Apriel-5B-Instruct4.4 GB at Q4	Fits192–272 tok/s	Fits197–279 tok/s	Fits367–520 tok/s
WeVisOne-VL-4B4.4 GB at Q4	Fits192–272 tok/s	Fits197–279 tok/s	Fits367–520 tok/s
Mage-VL4.3 GB at Q4	Fits195–277 tok/s	Fits200–284 tok/s	Fits374–530 tok/s
Mage-VL-Cowork4.3 GB at Q4	Fits195–277 tok/s	Fits200–284 tok/s	Fits374–530 tok/s
Qwen3.5-4B4.2 GB at Q4	Fits199–282 tok/s	Fits204–289 tok/s	Fits381–539 tok/s
S5-nano4.2 GB at Q4	Fits199–282 tok/s	Fits204–289 tok/s	Fits381–539 tok/s
AFM-4.5B4.2 GB at Q4	Fits201–284 tok/s	Fits206–291 tok/s	Fits384–544 tok/s
Llama-3.1-Minitron-4B-Width-Base4.1 GB at Q4	Fits205–291 tok/s	Fits211–298 tok/s	Fits393–557 tok/s
Qwen3-VL-4B-Instruct4.1 GB at Q4	Fits209–296 tok/s	Fits214–303 tok/s	Fits400–566 tok/s
CapRL-Qwen3VL-4B4.1 GB at Q4	Fits209–296 tok/s	Fits214–303 tok/s	Fits400–566 tok/s
glm-edge-4b-chat4.0 GB at Q4	Fits214–303 tok/s	Fits220–311 tok/s	Fits410–581 tok/s
gemma-3-4b-it4.0 GB at Q4	Fits215–305 tok/s	Fits221–313 tok/s	Fits412–584 tok/s
medgemma-4b-it4.0 GB at Q4	Fits215–305 tok/s	Fits221–313 tok/s	Fits412–584 tok/s
medgemma-1.5-4b-it4.0 GB at Q4	Fits215–305 tok/s	Fits221–313 tok/s	Fits412–584 tok/s
gemma-3-4b-pt4.0 GB at Q4	Fits215–305 tok/s	Fits221–313 tok/s	Fits412–584 tok/s
Nanbeige4.2-3B3.9 GB at Q4	Fits222–315 tok/s	Fits228–323 tok/s	Fits425–603 tok/s
Phi-3.5-vision-instruct3.9 GB at Q4	Fits223–316 tok/s	Fits229–325 tok/s	Fits428–606 tok/s
Spark-X2.6-4B3.9 GB at Q4	Fits225–319 tok/s	Fits231–327 tok/s	Fits431–611 tok/s
Spark-X2.5-4B3.9 GB at Q4	Fits225–319 tok/s	Fits231–327 tok/s	Fits431–611 tok/s
POINTS-Reader3.9 GB at Q4	Fits225–319 tok/s	Fits231–328 tok/s	Fits432–611 tok/s
MiniCPM-V-43.8 GB at Q4	Fits228–323 tok/s	Fits234–332 tok/s	Fits437–619 tok/s
Qwen3-4B3.8 GB at Q4	Fits230–326 tok/s	Fits236–335 tok/s	Fits441–625 tok/s
Qwen3-4B-Instruct-25073.8 GB at Q4	Fits230–326 tok/s	Fits236–335 tok/s	Fits441–625 tok/s
Qwen3-4B-Thinking-25073.8 GB at Q4	Fits230–326 tok/s	Fits236–335 tok/s	Fits441–625 tok/s
Mellum-4b-base3.8 GB at Q4	Fits230–327 tok/s	Fits236–335 tok/s	Fits441–625 tok/s
granite-4.0-3b-vision3.8 GB at Q4	Fits232–328 tok/s	Fits238–337 tok/s	Fits444–629 tok/s
granite-vision-4.1-4b3.8 GB at Q4	Fits232–328 tok/s	Fits238–337 tok/s	Fits444–629 tok/s
Nemotron Labs Dory 1.0 4B3.8 GB at Q4	Fits233–330 tok/s	Fits239–339 tok/s	Fits446–632 tok/s
NVIDIA-Nemotron-3-Nano-4B-BF163.8 GB at Q4	Fits233–330 tok/s	Fits239–339 tok/s	Fits446–632 tok/s
Nanbeige4.1-3B3.7 GB at Q4	Fits235–334 tok/s	Fits242–342 tok/s	Fits451–639 tok/s
Nanbeige4-3B-Thinking-25113.7 GB at Q4	Fits235–334 tok/s	Fits242–342 tok/s	Fits451–639 tok/s
Phi-4-mini-flash-reasoning3.7 GB at Q4	Fits240–341 tok/s	Fits247–349 tok/s	Fits460–652 tok/s
Phi-4-mini-instruct3.7 GB at Q4	Fits241–342 tok/s	Fits248–351 tok/s	Fits462–655 tok/s
Phi-4-mini-reasoning3.7 GB at Q4	Fits241–342 tok/s	Fits248–351 tok/s	Fits462–655 tok/s
Nemotron-Labs-Diffusion-3B3.7 GB at Q4	Fits242–342 tok/s	Fits248–351 tok/s	Fits463–656 tok/s
Phi-3.5-mini-instruct3.7 GB at Q4	Fits242–343 tok/s	Fits249–352 tok/s	Fits464–658 tok/s
MediPhi-Instruct3.7 GB at Q4	Fits242–343 tok/s	Fits249–352 tok/s	Fits464–658 tok/s
MediPhi3.7 GB at Q4	Fits242–343 tok/s	Fits249–352 tok/s	Fits464–658 tok/s
MediPhi-Clinical3.7 GB at Q4	Fits242–343 tok/s	Fits249–352 tok/s	Fits464–658 tok/s
MediPhi-PubMed3.7 GB at Q4	Fits242–343 tok/s	Fits249–352 tok/s	Fits464–658 tok/s
HY-Embodied-0.53.6 GB at Q4	Fits245–347 tok/s	Fits251–356 tok/s	Fits469–664 tok/s
Phi-tiny-MoE-instruct3.6 GB at Q4 · MoE	Fitsup to 1193 tok/s	Fitsup to 1224 tok/s	Fitsup to 2284 tok/s
CapRL-3B3.6 GB at Q4	Fits247–350 tok/s	Fits253–358 tok/s	Fits472–669 tok/s
HyperCLOVAX-SEED-Vision-Instruct-3B3.6 GB at Q4	Fits249–353 tok/s	Fits255–362 tok/s	Fits477–675 tok/s
kanana-1.5-v-3b-instruct3.6 GB at Q4	Fits253–358 tok/s	Fits259–367 tok/s	Fits484–685 tok/s
granite-4.2-3b3.6 GB at Q4	Fits253–359 tok/s	Fits260–368 tok/s	Fits485–687 tok/s
kanana-2-3b-instruct3.4 GB at Q4	Fits264–374 tok/s	Fits271–384 tok/s	Fits505–716 tok/s
granite-4.1-3b3.4 GB at Q4	Fits272–386 tok/s	Fits279–396 tok/s	Fits521–738 tok/s
granite-4.0-micro3.4 GB at Q4	Fits272–386 tok/s	Fits279–396 tok/s	Fits521–738 tok/s
granite-3.1-3b-a800m-instruct3.3 GB at Q4 · MoE	Fitsup to 1640 tok/s	Fitsup to 1683 tok/s	Fitsup to 3141 tok/s
Falcon3-3B-Instruct3.3 GB at Q4	Fits287–407 tok/s	Fits294–417 tok/s	Fits549–778 tok/s
Llama-3.2-3B-Instruct3.2 GB at Q4	Fits288–408 tok/s	Fits296–419 tok/s	Fits552–782 tok/s
Llama-3.2-3B3.2 GB at Q4	Fits288–408 tok/s	Fits296–419 tok/s	Fits552–782 tok/s
Hermes-3-Llama-3.2-3B3.2 GB at Q4	Fits288–408 tok/s	Fits296–419 tok/s	Fits552–782 tok/s
DeepHermes-3-Llama-3-3B-Preview3.2 GB at Q4	Fits288–408 tok/s	Fits296–419 tok/s	Fits552–782 tok/s
AI21-Jamba-Reasoning-3B3.2 GB at Q4	Fits290–410 tok/s	Fits297–421 tok/s	Fits555–786 tok/s
granite-4.0-h-micro3.2 GB at Q4	Fits290–411 tok/s	Fits298–422 tok/s	Fits556–787 tok/s
Falcon-H1-3B-Instruct3.2 GB at Q4	Fits294–417 tok/s	Fits302–427 tok/s	Fits563–798 tok/s
d1-3B3.2 GB at Q4	Fits297–420 tok/s	Fits304–431 tok/s	Fits568–804 tok/s
LFM2.5-VL-3B3.2 GB at Q4	Fits297–420 tok/s	Fits304–431 tok/s	Fits568–804 tok/s
Qwen2.5-3B-Instruct3.2 GB at Q4	Fits300–425 tok/s	Fits308–436 tok/s	Fits575–814 tok/s
Qwen2.5-Coder-3B-Instruct3.2 GB at Q4	Fits300–425 tok/s	Fits308–436 tok/s	Fits575–814 tok/s
Qwen2.5-3B3.2 GB at Q4	Fits300–425 tok/s	Fits308–436 tok/s	Fits575–814 tok/s
Qwen2.5-Coder-3B3.2 GB at Q4	Fits300–425 tok/s	Fits308–436 tok/s	Fits575–814 tok/s
SmolLM3-3B3.1 GB at Q4	Fits301–427 tok/s	Fits309–438 tok/s	Fits577–817 tok/s
AI21-Jamba2-3B3.1 GB at Q4	Fits306–433 tok/s	Fits314–444 tok/s	Fits585–829 tok/s
granite-swash-3b-a600m3.1 GB at Q4 · MoE	Fitsup to 2194 tok/s	Fitsup to 2251 tok/s	Fitsup to 4201 tok/s
granite-vision-3.2-2b3.1 GB at Q4	Fits311–441 tok/s	Fits319–452 tok/s	Fits596–844 tok/s
granite-vision-3.1-2b-preview3.1 GB at Q4	Fits311–441 tok/s	Fits319–452 tok/s	Fits596–844 tok/s
granite-vision-3.3-2b-chart2csv-preview3.1 GB at Q4	Fits311–441 tok/s	Fits319–452 tok/s	Fits596–844 tok/s
LFM2.5-2.6B2.9 GB at Q4	Fits343–487 tok/s	Fits352–499 tok/s	Fits658–932 tok/s
Zamba2-2.7B2.9 GB at Q4	Fits348–493 tok/s	Fits357–506 tok/s	Fits666–944 tok/s
Zamba2-2.7B-instruct2.9 GB at Q4	Fits348–493 tok/s	Fits357–506 tok/s	Fits666–944 tok/s
granite-3.0-2b-instruct2.8 GB at Q4	Fits352–498 tok/s	Fits361–511 tok/s	Fits673–954 tok/s
gemma-2-2b-it2.8 GB at Q4	Fits354–502 tok/s	Fits363–515 tok/s	Fits678–961 tok/s
gemma-2-2b2.8 GB at Q4	Fits354–502 tok/s	Fits363–515 tok/s	Fits678–961 tok/s
EuroMoE-2.6B-A0.6B-Instruct-Preview2.8 GB at Q4 · MoE	Fitsup to 2187 tok/s	Fitsup to 2243 tok/s	Fitsup to 4187 tok/s
Motif-2.6B2.8 GB at Q4	Fits357–505 tok/s	Fits366–518 tok/s	Fits683–967 tok/s
LFM2-2.6B2.8 GB at Q4	Fits361–511 tok/s	Fits370–524 tok/s	Fits690–978 tok/s
LFM2-2.6B-Exp2.8 GB at Q4	Fits361–511 tok/s	Fits370–524 tok/s	Fits690–978 tok/s
LFM2-2.6B-Transcript2.8 GB at Q4	Fits361–511 tok/s	Fits370–524 tok/s	Fits690–978 tok/s
granite-3.3-2b-instruct2.8 GB at Q4	Fits366–518 tok/s	Fits375–531 tok/s	Fits700–992 tok/s
granite-3.1-2b-instruct2.8 GB at Q4	Fits366–518 tok/s	Fits375–531 tok/s	Fits700–992 tok/s
granite-3.2-2b-instruct2.8 GB at Q4	Fits366–518 tok/s	Fits375–531 tok/s	Fits700–992 tok/s
MiniCPM5-2B2.8 GB at Q4	Fits368–521 tok/s	Fits378–535 tok/s	Fits705–998 tok/s
MiniCPM5-2B-SFT2.8 GB at Q4	Fits368–521 tok/s	Fits378–535 tok/s	Fits705–998 tok/s
MiniCPM5-2B-Midtrain2.8 GB at Q4	Fits368–521 tok/s	Fits378–535 tok/s	Fits705–998 tok/s
sarvam-1-v0.52.7 GB at Q4	Fits369–523 tok/s	Fits379–536 tok/s	Fits707–1001 tok/s
North-Micro-Vision-Instruct2.7 GB at Q4	Fits373–528 tok/s	Fits382–542 tok/s	Fits714–1011 tok/s
UI-TARS-2B-SFT2.7 GB at Q4	Fits379–537 tok/s	Fits389–551 tok/s	Fits726–1029 tok/s
WeVisOne-VL-2B2.7 GB at Q4	Fits379–537 tok/s	Fits389–551 tok/s	Fits726–1029 tok/s
UI-Venus-1.5-2B2.7 GB at Q4	Fits380–538 tok/s	Fits390–552 tok/s	Fits727–1030 tok/s
bitnet-b1.58-2B-4T-bf162.7 GB at Q4	Fits384–544 tok/s	Fits394–558 tok/s	Fits735–1041 tok/s
EXAONE-3.5-2.4B-Instruct2.7 GB at Q4	Fits385–546 tok/s	Fits395–560 tok/s	Fits737–1045 tok/s
EXAONE-Deep-2.4B2.7 GB at Q4	Fits385–546 tok/s	Fits395–560 tok/s	Fits737–1045 tok/s
kanana-1.5-2.1b-instruct-25052.6 GB at Q4	Fits400–566 tok/s	Fits410–581 tok/s	Fits765–1084 tok/s
Qwen3.5-2B2.6 GB at Q4	Fits407–577 tok/s	Fits418–592 tok/s	Fits780–1105 tok/s
salamandra-2b2.6 GB at Q4	Fits411–582 tok/s	Fits422–597 tok/s	Fits787–1115 tok/s
salamandra-2b-instruct2.6 GB at Q4	Fits411–582 tok/s	Fits422–597 tok/s	Fits787–1115 tok/s
SmolVLM2-2.2B-Instruct2.6 GB at Q4	Fits412–584 tok/s	Fits423–599 tok/s	Fits789–1118 tok/s
SmolVLM-Base2.6 GB at Q4	Fits412–584 tok/s	Fits423–599 tok/s	Fits790–1119 tok/s
Qwen2-VL-2B-Instruct2.5 GB at Q4	Fits419–594 tok/s	Fits430–609 tok/s	Fits803–1137 tok/s
LFM3-Omni-2B2.5 GB at Q4	Fits427–606 tok/s	Fits438–621 tok/s	Fits818–1159 tok/s
granite-swash-2b2.5 GB at Q4	Fits432–612 tok/s	Fits443–628 tok/s	Fits827–1172 tok/s
Qwen3-VL-2B-Instruct2.5 GB at Q4	Fits435–617 tok/s	Fits447–633 tok/s	Fits834–1181 tok/s
kanana-nano-2.1b-instruct2.5 GB at Q4	Fits444–629 tok/s	Fits455–645 tok/s	Fits850–1204 tok/s
Hy-MT2-1.8B2.4 GB at Q4	Fits454–644 tok/s	Fits466–660 tok/s	Fits870–1233 tok/s
Qwen3-1.7B2.4 GB at Q4	Fits456–646 tok/s	Fits468–662 tok/s	Fits873–1237 tok/s
Youtu-LLM-2B2.4 GB at Q4	Fits472–669 tok/s	Fits484–686 tok/s	Fits904–1281 tok/s
LFM3-VL-2B2.3 GB at Q4	Fits483–684 tok/s	Fits495–702 tok/s	Fits925–1310 tok/s
d1-2B2.3 GB at Q4	Fits483–684 tok/s	Fits495–702 tok/s	Fits925–1310 tok/s
FastVLM-1.5B2.3 GB at Q4	Fits485–687 tok/s	Fits498–705 tok/s	Fits929–1316 tok/s
internlm2_5-1_8b-chat2.3 GB at Q4	Fits490–695 tok/s	Fits503–712 tok/s	Fits939–1330 tok/s
DeepSeek-R1-Distill-Qwen-1.5B2.2 GB at Q4	Fits521–738 tok/s	Fits535–757 tok/s	Fits998–1414 tok/s
LFM3-1.7B2.2 GB at Q4	Fits536–759 tok/s	Fits550–779 tok/s	Fits1026–1453 tok/s
Qwen3-1.7B-Base2.2 GB at Q4	Fits538–763 tok/s	Fits552–782 tok/s	Fits1031–1460 tok/s
SmolLM2-1.7B2.2 GB at Q4	Fits541–767 tok/s	Fits555–786 tok/s	Fits1036–1468 tok/s
SmolLM-1.7B2.2 GB at Q4	Fits541–767 tok/s	Fits555–786 tok/s	Fits1036–1468 tok/s
Spark-X2.6-1.7B2.2 GB at Q4	Fits542–768 tok/s	Fits556–788 tok/s	Fits1039–1471 tok/s
Falcon3-1B-Instruct2.2 GB at Q4	Fits555–786 tok/s	Fits569–806 tok/s	Fits1062–1505 tok/s
EuroLLM-1.7B-Instruct2.2 GB at Q4	Fits559–792 tok/s	Fits573–812 tok/s	Fits1070–1516 tok/s
granite-4.0-1b2.1 GB at Q4	Fits568–804 tok/s	Fits582–825 tok/s	Fits1087–1540 tok/s
LFM2.5-VL-1.6B2.1 GB at Q4	Fits580–822 tok/s	Fits595–843 tok/s	Fits1111–1574 tok/s
glm-edge-1.5b-chat2.1 GB at Q4	Fits581–824 tok/s	Fits596–845 tok/s	Fits1113–1577 tok/s
LFM2-VL-1.6B2.1 GB at Q4	Fits585–828 tok/s	Fits600–849 tok/s	Fits1119–1585 tok/s
Falcon-H1-1.5B-Deep-Instruct2.1 GB at Q4	Fits596–844 tok/s	Fits611–866 tok/s	Fits1141–1616 tok/s
Falcon-H1-1.5B-Instruct2.1 GB at Q4	Fits596–844 tok/s	Fits611–866 tok/s	Fits1141–1616 tok/s
MiniCPM-V-4.7-1B2.1 GB at Q4	Fits596–844 tok/s	Fits611–866 tok/s	Fits1141–1616 tok/s
Qwen2.5-1.5B-Instruct2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2.5-1.5B2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2.5-Coder-1.5B-Instruct2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2.5-Coder-1.5B2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2.5-Math-1.5B2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2-1.5B-Instruct2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Qwen2.5-Math-1.5B-Instruct2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
ReaderLM-v22.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
reader-lm-1.5b2.1 GB at Q4	Fits600–850 tok/s	Fits615–872 tok/s	Fits1149–1628 tok/s
Hymba-1.5B-Instruct2.1 GB at Q4	Fits608–862 tok/s	Fits624–884 tok/s	Fits1165–1650 tok/s
OLMo-2-0425-1B2.0 GB at Q4	Fits624–884 tok/s	Fits640–906 tok/s	Fits1194–1692 tok/s
OLMo-2-0425-1B-Instruct2.0 GB at Q4	Fits624–884 tok/s	Fits640–906 tok/s	Fits1194–1692 tok/s
Yi-Coder-1.5B-Chat2.0 GB at Q4	Fits627–889 tok/s	Fits643–912 tok/s	Fits1201–1702 tok/s
Bolmo-1B2.0 GB at Q4	Fits631–893 tok/s	Fits647–916 tok/s	Fits1207–1710 tok/s
granite-4.0-h-1b2.0 GB at Q4	Fits634–898 tok/s	Fits650–921 tok/s	Fits1213–1719 tok/s
granite-3.1-1b-a400m-instruct1.9 GB at Q4 · MoE	Fitsup to 3281 tok/s	Fitsup to 3365 tok/s	Fitsup to 6281 tok/s
granite-3.0-1b-a400m-instruct1.9 GB at Q4 · MoE	Fitsup to 3281 tok/s	Fitsup to 3365 tok/s	Fitsup to 6281 tok/s
MiniCPM-V-4.61.9 GB at Q4	Fits712–1009 tok/s	Fits731–1035 tok/s	Fits1364–1932 tok/s
MiniCPM-V-4.6-Thinking1.9 GB at Q4	Fits712–1009 tok/s	Fits731–1035 tok/s	Fits1364–1932 tok/s
kanana-2-1.3b-instruct1.9 GB at Q4	Fits717–1016 tok/s	Fits736–1042 tok/s	Fits1373–1945 tok/s
OLMo-1B-0724-hf1.9 GB at Q4	Fits724–1025 tok/s	Fits742–1052 tok/s	Fits1386–1963 tok/s
EXAONE-4.0-1.2B1.9 GB at Q4	Fits724–1026 tok/s	Fits743–1052 tok/s	Fits1386–1964 tok/s
Llama-3.2-1B-Instruct1.9 GB at Q4	Fits750–1062 tok/s	Fits769–1089 tok/s	Fits1435–2033 tok/s
Llama-3.2-1B1.9 GB at Q4	Fits750–1062 tok/s	Fits769–1089 tok/s	Fits1435–2033 tok/s
Llama-3.2-1B1.9 GB at Q4	Fits750–1062 tok/s	Fits769–1089 tok/s	Fits1435–2033 tok/s
Zamba2-1.2B-instruct1.8 GB at Q4	Fits762–1080 tok/s	Fits782–1108 tok/s	Fits1460–2068 tok/s
HRM-Text-1B1.8 GB at Q4	Fits783–1110 tok/s	Fits803–1138 tok/s	Fits1499–2124 tok/s
AMD-OLMo-1B1.8 GB at Q4	Fits787–1115 tok/s	Fits807–1144 tok/s	Fits1507–2135 tok/s
LFM2.5-1.2B-Instruct1.8 GB at Q4	Fits792–1121 tok/s	Fits812–1150 tok/s	Fits1515–2147 tok/s
LFM2-1.2B1.8 GB at Q4	Fits792–1121 tok/s	Fits812–1150 tok/s	Fits1515–2147 tok/s
LFM2.5-1.2B-Thinking1.8 GB at Q4	Fits792–1121 tok/s	Fits812–1150 tok/s	Fits1515–2147 tok/s
LFM2.5-1.2B-JP-2026061.8 GB at Q4	Fits792–1121 tok/s	Fits812–1150 tok/s	Fits1515–2147 tok/s
MiniCPM5-1B1.8 GB at Q4	Fits857–1214 tok/s	Fits879–1246 tok/s	Fits1641–2325 tok/s
MiniCPM5-1B-SFT1.8 GB at Q4	Fits857–1214 tok/s	Fits879–1246 tok/s	Fits1641–2325 tok/s
gemma-3-1b-it1.7 GB at Q4	Fits926–1312 tok/s	Fits950–1346 tok/s	Fits1774–2513 tok/s
gemma-3-1b-pt1.7 GB at Q4	Fits926–1312 tok/s	Fits950–1346 tok/s	Fits1774–2513 tok/s
Qwen3.5-0.8B1.6 GB at Q4	Fits1061–1502 tok/s	Fits1088–1541 tok/s	Fits2031–2877 tok/s
Falcon-E-3B-Instruct1.6 GB at Q4	Fits1072–1519 tok/s	Fits1099–1558 tok/s	Fits2052–2907 tok/s
bitnet-b1.58-2B-4T1.6 GB at Q4	Fits1090–1544 tok/s	Fits1118–1584 tok/s	Fits2087–2957 tok/s
FastVLM-0.5B1.5 GB at Q4	Fits1221–1729 tok/s	Fits1252–1774 tok/s	Fits2337–3311 tok/s
Qwen3-0.6B1.5 GB at Q4	Fits1232–1746 tok/s	Fits1264–1791 tok/s	Fits2360–3343 tok/s
LFM2-700M1.5 GB at Q4	Fits1248–1767 tok/s	Fits1280–1813 tok/s	Fits2389–3384 tok/s
Falcon-H1-Tiny-R-0.6B1.4 GB at Q4	Fits1489–2109 tok/s	Fits1527–2163 tok/s	Fits2850–4037 tok/s
Qwen3-0.6B-Base1.4 GB at Q4	Fits1554–2202 tok/s	Fits1594–2258 tok/s	Fits2975–4215 tok/s
d1-omni-600M1.4 GB at Q4	Fits1578–2235 tok/s	Fits1618–2292 tok/s	Fits3021–4279 tok/s
HyperCLOVAX-SEED-Text-Instruct-0.5B1.4 GB at Q4	Fits1636–2317 tok/s	Fits1678–2377 tok/s	Fits3132–4437 tok/s
Falcon-H1-0.5B-Instruct1.4 GB at Q4	Fits1777–2517 tok/s	Fits1822–2581 tok/s	Fits3401–4819 tok/s
Qwen2.5-0.5B-Instruct1.3 GB at Q4	Fits1875–2656 tok/s	Fits1923–2724 tok/s	Fits3590–5086 tok/s
Qwen2.5-0.5B1.3 GB at Q4	Fits1875–2656 tok/s	Fits1923–2724 tok/s	Fits3590–5086 tok/s
Qwen2-0.5B-Instruct1.3 GB at Q4	Fits1875–2656 tok/s	Fits1923–2724 tok/s	Fits3590–5086 tok/s
Qwen2.5-Coder-0.5B-Instruct1.3 GB at Q4	Fits1875–2656 tok/s	Fits1923–2724 tok/s	Fits3590–5086 tok/s
LFM2-VL-450M1.3 GB at Q4	Fits2055–2911 tok/s	Fits2107–2986 tok/s	Fits3934–5573 tok/s
LFM2.5-VL-450M1.3 GB at Q4	Fits2064–2925 tok/s	Fits2117–3000 tok/s	Fits3952–5599 tok/s
Apertus-v1.1-0.5B1.3 GB at Q4	Fits2113–2994 tok/s	Fits2167–3071 tok/s	Fits4046–5732 tok/s
MiniCPM4-0.5B1.3 GB at Q4	Fits2135–3025 tok/s	Fits2190–3102 tok/s	Fits4088–5791 tok/s
circuit-sparsity1.3 GB at Q4	Fits2210–3131 tok/s	Fits2267–3211 tok/s	Fits4231–5995 tok/s
SmolLM2-360M1.3 GB at Q4	Fits2560–3627 tok/s	Fits2626–3720 tok/s	Fits4902–6944 tok/s
SmolLM-360M1.3 GB at Q4	Fits2560–3627 tok/s	Fits2626–3720 tok/s	Fits4902–6944 tok/s
ERNIE-4.5-0.3B-PT1.3 GB at Q4	Fits2568–3638 tok/s	Fits2634–3731 tok/s	Fits4916–6965 tok/s
LFM2.5-350M1.2 GB at Q4	Fits2613–3702 tok/s	Fits2680–3797 tok/s	Fits5003–7088 tok/s
LFM2-350M1.2 GB at Q4	Fits2613–3702 tok/s	Fits2680–3797 tok/s	Fits5003–7088 tok/s
granite-4.0-350m1.2 GB at Q4	Fits2629–3724 tok/s	Fits2696–3820 tok/s	Fits5033–7130 tok/s
granite-4.0-h-350m1.2 GB at Q4	Fits2722–3856 tok/s	Fits2792–3955 tok/s	Fits5211–7382 tok/s

Change a part? Open any build in the planner, tweak it, use "Compare this build", and it lands here as Build A. All build recipes.