What is a mixture-of-experts (MoE) model?

A MoE model splits most of its layers into many small "experts" and a router picks a few of them for each token. So it has a large total parameter count, but only a fraction (the active parameters) does work on each token. Qwen3-30B-A3B, for example, has 30.5B parameters in total but uses about 3.0B per token.

Does a MoE model need less VRAM?

No. All the experts must be in memory, because any of them can be picked for the next token. Memory follows total parameters, just like a dense model of the same size. Qwen3-30B-A3B needs ~22.3 GB at Q4, close to Qwen3-32B's ~23.8 GB.

Why are MoE models so much faster locally?

Generating a token is limited by how many bytes are read from memory, and a MoE model reads only its active experts. With 10% of weights active, decode speed can be several times a dense model of the same size. We show MoE speed as an upper bound ("up to") because routing, attention and shared layers add cost we haven't calibrated.

Is MoE a good fit for Macs and unified-memory PCs?

Yes. Macs, AMD Ryzen AI Max machines and DGX Spark have lots of memory but less bandwidth than a high-end GPU, so dense 70B models run slowly on them. A big MoE model fits in that memory and still runs at a usable speed. On the Framework Desktop (Ryzen AI Max+ 395) 128GB, our estimate for gpt-oss-120b is up to 66 tok/s, against 3–5 tok/s for Llama-3.3-70B-Instruct.

Is a MoE model as smart as a dense model of the same total size?

Usually it lands between a dense model of its active size and one of its total size, and it varies by model. Check benchmarks for your task; this page covers memory and speed only.

Can I offload MoE experts to system RAM?

Yes. llama.cpp can keep the experts in system RAM and the rest on the GPU (for example with --n-cpu-moe or tensor overrides), which lets a big MoE model run on a smaller GPU at a lower speed. We don't model offloading; our fit figures assume the whole model is in GPU or unified memory.

MoE vs dense models for local LLMs

Mixture-of-experts models like Qwen3-30B-A3B, gpt-oss and Gemma 4 26B A4B need memory for every parameter but only read a few billion per token. That makes them the best speed per GB of memory you can run locally. This page lists the 40 MoE models we track, their total and active size, the cheapest hardware for each, and side-by-side speed against similar dense models.

Memory: total params all experts must fit
Speed: active params bytes read per token
MoE models tracked: 40

Short answer: size your memory for the total parameters and expect speed close to a dense model of the active size. On the GeForce RTX 5090, Qwen3-30B-A3B has a speed ceiling of up to 837 tok/s against 54–77 for the dense Qwen3-32B, with about the same memory. Real MoE speeds land well below that ceiling, since routing and attention add work, but the gap stays large.

Every MoE model we track

Active parameters come from the model name (like 30B-A3B) or the maker's model card. Memory is Q4_K_M with ~4k context (our formula). Speed is an upper bound on decode at Q4 on the cheapest current device that fits (how we estimate).

MoE models: total and active parameters, Q4 memory, and the cheapest GPU and Mac or mini PC that run each
Model	Total	Active	Active share	Q4 memory	Cheapest GPU	Cheapest Mac / mini PC
Mellum2.1-12B-A2.5B-ThinkingJetBrains	12.1B	2.5B	21%	9.5 GB	Intel Arc B580$249 · up to 256 tok/s	Mac mini M6 16GB$899 · up to 81 tok/s
gpt-oss-20bopenai	20.9B	3.6B	17%	15.6 GB	Radeon RX 7900 XT$899 · up to 312 tok/s	Mac mini M6 24GB$1,299 · up to 62 tok/s
LFM2.5-24B-A2BLiquidAI	24.0B	2.0B	8%	17.7 GB	Radeon RX 7900 XT$899 · up to 561 tok/s	Mac mini M6 32GB$1,799 · up to 112 tok/s
Gemma 4 26B A4Bgoogle	25.8B	3.8B	15%	19.0 GB	Intel Arc Pro B70 32GB$949 · up to 224 tok/s	Mac mini M6 32GB$1,799 · up to 59 tok/s
diffusiongemma-26B-A4B-itgoogle	25.8B	4.0B	15%	19.0 GB	Intel Arc Pro B70 32GB$949 · up to 213 tok/s	Mac mini M6 32GB$1,799 · up to 56 tok/s
Raon-1 mini previewKRAFTON	28.8B	4.3B	15%	21.1 GB	Intel Arc Pro B70 32GB$949 · up to 198 tok/s	Mac mini M6 32GB$1,799 · up to 52 tok/s
Hy-Memory-30B-A3Btencent	30.1B	3.0B	10%	22.0 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Qwen3-30B-A3BQwen	30.5B	3.0B	10%	22.3 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Qwen3-Coder-30B-A3B-InstructQwen	30.5B	3.0B	10%	22.3 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Qwen3-30B-A3B-Instruct-2507Qwen	30.5B	3.0B	10%	22.3 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
GLM-4.7-Flashzai-org	31.2B	3.0B	10%	22.8 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16nvidia	31.6B	3.0B	10%	23.0 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16nvidia	31.6B	3.0B	10%	23.0 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
MemT-1.0-NanoMemTensor	35.1B	3.0B	9%	25.5 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Ornith-1.5-35B-A3Bornith-ai	36.0B	3.0B	8%	26.1 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Qwen3.6-35B-A3BQwen	36.0B	3.0B	8%	26.1 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Qwen3.5-35B-A3BQwen	36.0B	3.0B	8%	26.1 GB	Intel Arc Pro B70 32GB$949 · up to 284 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 135 tok/s
Recursive FlintRecursiveSI	39.8B	2.5B	6%	28.7 GB	Intel Arc Pro B70 32GB$949 · up to 341 tok/s	Mac mini M5 Pro 64GB$3,199 · up to 162 tok/s
gpt-oss-120bopenai	116.8B	5.1B	4%	82.5 GB	NVIDIA RTX Pro 6000 Blackwell 96GB$8,565 · up to 493 tok/s	Framework Desktop (Ryzen AI Max+ 395) 128GB$3,449 · up to 66 tok/s
Ling-3.0-flash-SanteinclusionAI	127.5B	5.1B	4%	89.9 GB	none we track	Mac Studio M5 Ultra 256GB$9,499 · up to 310 tok/s
MiniMax-M2.5MiniMaxAI	228.7B	10.0B	4%	160.4 GB	none we track	Mac Studio M5 Ultra 256GB$9,499 · up to 158 tok/s
MemT-1.0-FlashMemTensor	287.9B	17.0B	6%	201.8 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 93 tok/s
DeepSeek-V4-Flashdeepseek-ai	290.9B	13.0B	4%	203.8 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 122 tok/s
Hy3tencent	298.8B	21.0B	7%	209.3 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 75 tok/s
DeepSeek-V4-Flash-0731deepseek-ai	304.2B	13.0B	4%	213.1 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 122 tok/s
GLM-5.3-Flashzai-org	321.3B	18.0B	6%	225.0 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 88 tok/s
Qwen3.5-397B-A17BQwen	403.4B	17.0B	4%	282.2 GB	none we track	Mac Studio M5 Ultra 512GB$22,299 · up to 93 tok/s
Beam-1.0-501B-A23Breflection	501.3B	23.0B	5%	350.5 GB	none we track	none we track
DeepSeek-R1deepseek-ai	684.5B	37.0B	5%	478.2 GB	none we track	none we track
DeepSeek-V3-0324deepseek-ai	684.5B	37.0B	5%	478.2 GB	none we track	none we track
DeepSeek-V3.2deepseek-ai	685.4B	37.0B	5%	478.8 GB	none we track	none we track
GLM-5.2zai-org	753.3B	40.0B	5%	526.2 GB	none we track	none we track
GLM-5zai-org	753.9B	40.0B	5%	526.6 GB	none we track	none we track
GLM-5.1zai-org	753.9B	40.0B	5%	526.6 GB	none we track	none we track
Muse-Spark-Open-816B-A42Bmeta-models	822.0B	42.0B	5%	574.1 GB	none we track	none we track
Inklingthinkingmachines	952.4B	41.0B	4%	665.0 GB	none we track	none we track
Kimi-K2-Instructmoonshotai	1026.4B	32.0B	3%	716.6 GB	none we track	none we track
Kimi-K2.5moonshotai	1026.9B	32.0B	3%	716.9 GB	none we track	none we track
DeepSeek-V4-Prodeepseek-ai	1598.8B	49.0B	3%	1115.7 GB	none we track	none we track
Kimi-K3moonshotai	2779.9B	104.0B	4%	1939.1 GB	none we track	none we track

Showing the 40 most popular of 226 MoE models. See all 226 in the model directory →

Models whose makers don't state active parameters (for example GLM-5.3 and Sage-A1.5) are left out here; on their own pages we estimate speed from total parameters, which understates it.

MoE vs dense, head to head

Pairs of similar memory size on 7 current devices. MoE figures are upper bounds; dense figures are a range.

Qwen3-30B-A3B vs Qwen3-32B

Qwen3-30B-A3B: 30.5B total, 3.0B active, ~22.3 GB at Q4. Qwen3-32B: 32.8B dense, ~23.8 GB at Q4. Est. decode tok/s at Q4.

Estimated tokens per second, Qwen3-30B-A3B vs Qwen3-32B
Device	Qwen3-30B-A3B (MoE)	Qwen3-32B (dense)
GeForce RTX 4060 Ti 16GB16 GB usable · 288 GB/s · $499	doesn’t fit	doesn’t fit
Radeon RX 7900 XTX24 GB usable · 960 GB/s · $999	doesn’t fit	doesn’t fit
GeForce RTX 509032 GB usable · 1792 GB/s · $1,999	up to 837	54–77
Mac mini M6 32GB24 GB usable · 170 GB/s · $1,799	doesn’t fit	doesn’t fit
Mac mini M5 Pro 64GB48 GB usable · 307 GB/s · $3,199	up to 135	7–12
Framework Desktop (Ryzen AI Max+ 395) 128GB96 GB usable · 256 GB/s · $3,449	up to 113	6–10
MacBook Pro M5 Max 128GB96 GB usable · 614 GB/s · $6,699	up to 270	14–25

Gemma 4 26B A4B vs Gemma 4 31B

Gemma 4 26B A4B: 25.8B total, 3.8B active, ~19.0 GB at Q4. Gemma 4 31B: 31.3B dense, ~22.8 GB at Q4. Est. decode tok/s at Q4.

Estimated tokens per second, Gemma 4 26B A4B vs Gemma 4 31B
Device	Gemma 4 26B A4B (MoE)	Gemma 4 31B (dense)
GeForce RTX 4060 Ti 16GB16 GB usable · 288 GB/s · $499	doesn’t fit	doesn’t fit
Radeon RX 7900 XTX24 GB usable · 960 GB/s · $999	up to 354	doesn’t fit
GeForce RTX 509032 GB usable · 1792 GB/s · $1,999	up to 661	57–80
Mac mini M6 32GB24 GB usable · 170 GB/s · $1,799	up to 59	doesn’t fit
Mac mini M5 Pro 64GB48 GB usable · 307 GB/s · $3,199	up to 107	7–13
Framework Desktop (Ryzen AI Max+ 395) 128GB96 GB usable · 256 GB/s · $3,449	up to 89	6–11
MacBook Pro M5 Max 128GB96 GB usable · 614 GB/s · $6,699	up to 213	15–26

gpt-oss-120b vs Llama-3.3-70B-Instruct

gpt-oss-120b: 116.8B total, 5.1B active, ~82.5 GB at Q4. Llama-3.3-70B-Instruct: 70.6B dense, ~50.2 GB at Q4. Est. decode tok/s at Q4.

Estimated tokens per second, gpt-oss-120b vs Llama-3.3-70B-Instruct
Device	gpt-oss-120b (MoE)	Llama-3.3-70B-Instruct (dense)
GeForce RTX 4060 Ti 16GB16 GB usable · 288 GB/s · $499	doesn’t fit	doesn’t fit
Radeon RX 7900 XTX24 GB usable · 960 GB/s · $999	doesn’t fit	doesn’t fit
GeForce RTX 509032 GB usable · 1792 GB/s · $1,999	doesn’t fit	doesn’t fit
Mac mini M6 32GB24 GB usable · 170 GB/s · $1,799	doesn’t fit	doesn’t fit
Mac mini M5 Pro 64GB48 GB usable · 307 GB/s · $3,199	doesn’t fit	doesn’t fit
Framework Desktop (Ryzen AI Max+ 395) 128GB96 GB usable · 256 GB/s · $3,449	up to 66	3–5
MacBook Pro M5 Max 128GB96 GB usable · 614 GB/s · $6,699	up to 159	6–11

gpt-oss-20b vs Qwen3-14B

gpt-oss-20b: 20.9B total, 3.6B active, ~15.6 GB at Q4. Qwen3-14B: 14.8B dense, ~11.3 GB at Q4. Est. decode tok/s at Q4.

Estimated tokens per second, gpt-oss-20b vs Qwen3-14B
Device	gpt-oss-20b (MoE)	Qwen3-14B (dense)
GeForce RTX 4060 Ti 16GB16 GB usable · 288 GB/s · $499	doesn’t fit	19–27
Radeon RX 7900 XTX24 GB usable · 960 GB/s · $999	up to 374	64–91
GeForce RTX 509032 GB usable · 1792 GB/s · $1,999	up to 698	120–170
Mac mini M6 32GB24 GB usable · 170 GB/s · $1,799	up to 62	9–15
Mac mini M5 Pro 64GB48 GB usable · 307 GB/s · $3,199	up to 113	15–27
Framework Desktop (Ryzen AI Max+ 395) 128GB96 GB usable · 256 GB/s · $3,449	up to 94	13–23
MacBook Pro M5 Max 128GB96 GB usable · 614 GB/s · $6,699	up to 225	31–55

Quick answers

What is a mixture-of-experts (MoE) model?: A MoE model splits most of its layers into many small "experts" and a router picks a few of them for each token. So it has a large total parameter count, but only a fraction (the active parameters) does work on each token. Qwen3-30B-A3B, for example, has 30.5B parameters in total but uses about 3.0B per token.
Does a MoE model need less VRAM?: No. All the experts must be in memory, because any of them can be picked for the next token. Memory follows total parameters, just like a dense model of the same size. Qwen3-30B-A3B needs ~22.3 GB at Q4, close to Qwen3-32B's ~23.8 GB.
Why are MoE models so much faster locally?: Generating a token is limited by how many bytes are read from memory, and a MoE model reads only its active experts. With 10% of weights active, decode speed can be several times a dense model of the same size. We show MoE speed as an upper bound ("up to") because routing, attention and shared layers add cost we haven't calibrated.
Is MoE a good fit for Macs and unified-memory PCs?: Yes. Macs, AMD Ryzen AI Max machines and DGX Spark have lots of memory but less bandwidth than a high-end GPU, so dense 70B models run slowly on them. A big MoE model fits in that memory and still runs at a usable speed. On the Framework Desktop (Ryzen AI Max+ 395) 128GB, our estimate for gpt-oss-120b is up to 66 tok/s, against 3–5 tok/s for Llama-3.3-70B-Instruct.
Is a MoE model as smart as a dense model of the same total size?: Usually it lands between a dense model of its active size and one of its total size, and it varies by model. Check benchmarks for your task; this page covers memory and speed only.
Can I offload MoE experts to system RAM?: Yes. llama.cpp can keep the experts in system RAM and the rest on the GPU (for example with --n-cpu-moe or tensor overrides), which lets a big MoE model run on a smaller GPU at a lower speed. We don't model offloading; our fit figures assume the whole model is in GPU or unified memory.

Tokens per second · Q4 vs Q8 vs FP16 · Context length and VRAM · NVIDIA vs AMD vs Mac · VRAM calculator