What Can I Run AI?

Builds

Mac Studio M5 Ultra vs. dual RTX 3090 for local LLMs

One quiet box with 256 GB unified memory against a 48 GB two-GPU tower. The Mac fits far bigger models; the 3090s have more bandwidth per GB.

Total
$9,499
Usable memory
192.0 GB
Llama-3.3-70B-Instruct Q4
Fits
Est. tok/s
13–22
$ per tok/s
$542

Parts

PartItemPriceBuy
GPUMac Studio M5 Ultra 256GB$9,499Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TBAmazon
Total$9,499

We may earn a commission when you buy through these links, at no extra cost to you. Prices are dated snapshots (2026-10-23 for parts; GPUs at list price unless marked used). Used prices come from one dated listing each.

Will Llama-3.3-70B-Instruct run?

ModelQ4_K_MQ8_0FP16Est. tok/s (Q4)
Llama-3.3-70B-Instruct50.2 GB at Q4FitsFitsFits13–22

Tok/s = memory bandwidth ÷ model weight size × a 45–80% efficiency band, calibrated on llama.cpp scoreboards. Mixture-of-experts models show only an upper bound ("up to"). How we estimate speed.

Fit checks

Compared with: Dual used RTX 3090 build (48 GB) for 32B to 70B models

Total
$7,216
Usable memory
48.0 GB
Llama-3.3-70B-Instruct Q4
Won’t fit
Est. tok/s
—
PSU needed
1350 W
PartItemPriceBuy
GPU2× Used RTX 3090 24GB (ASUS TUF, "Like New")$4,080used listingAmazon
CPUAMD Ryzen 9 9950X$539Newegg price (sold by Newegg)Amazon
MotherboardASUS ProArt X870E-CREATOR WIFI$512Newegg price (sold by Newegg)Amazon
RAMCorsair Vengeance 64GB (2x32) DDR5-6000 CL30$1,275Corsair store priceAmazon
Power supplyCorsair HX1500i$320Corsair store priceAmazon
CaseCorsair FRAME 5000D RS$160Corsair store priceAmazon
StorageSamsung 990 2TB NVMe$330Newegg price (sold by Newegg)Amazon
Total$7,216

We may earn a commission when you buy through these links, at no extra cost to you. Prices are dated snapshots (2026-10-23 for parts; GPUs at list price unless marked used). Used prices come from one dated listing each.

ModelQ4_K_MQ8_0FP16Est. tok/s (Q4)
Llama-3.3-70B-Instruct50.2 GB at Q4Won’t fitWon’t fitWon’t fit—

Full Dual used RTX 3090 build (48 GB) for 32B to 70B models page

All models on this build

ModelQ4_K_MQ8_0FP16Est. tok/s (Q4)
Kimi-K3.1-Pro1939.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K31939.1 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Qwen3.8-2.4T-A95B1706.4 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
LongCat-2.51241.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
LongCat-2.01238.9 GB at Q4Won’t fitWon’t fitWon’t fit—
DeepSeek-V4-Pro-08131151.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V4-Pro1115.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2.5716.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2.6716.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2.7-Code716.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2-Thinking716.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2-Instruct-0905716.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Kimi-K2-Instruct716.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2.6-Pro-RL715.1 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2.6-Pro-MOPD715.1 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2.5-Pro714.4 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ring-2.5-1T706.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ling-2.5-1T706.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ling-1T698.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ring-1T698.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Inkling665.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Intern-S1-Pro640.4 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Muse-Spark-Open-816B-A42B574.1 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Hy4-preview544.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-5526.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-5.1526.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-5.3526.2 GB at Q4Won’t fitWon’t fitWon’t fit—
GLM-5.2526.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-5.3-BF16526.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
K-EXAONE-2.0-750B-A37B523.4 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V3.2-Exp478.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V3.2-Speciale478.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V3.2478.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V3-0324478.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-R1-0528478.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V3.1-Base478.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-R1478.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-R1-Zero478.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Step-5-Preview422.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ling-3.1-flash399.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ling-3.1-flash-1019399.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
LongCat-Flash-Chat392.7 GB at Q4Won’t fitWon’t fitWon’t fit—
LongCat-Flash-Thinking392.7 GB at Q4Won’t fitWon’t fitWon’t fit—
LongCat-Flash-Thinking-2601392.7 GB at Q4Won’t fitWon’t fitWon’t fit—
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16391.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Beam-1.0-501B-A23B350.5 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Qwen3-Coder-480B-A35B-Instruct335.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiniMax-M1-40k319.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiniMax-M1-80k319.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiniMax-M3298.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
ERNIE-4.5-VL-424B-A47B-PT296.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Intern-S2-397B282.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Qwen3.5-397B-A17B282.2 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Llama-4-Maverick-17B-128E-Instruct281.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Trinity-Large-Thinking278.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Trinity-Large-Preview278.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Trinity-Large-TrueBase278.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Trinity-Large-Base278.9 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Nex-N2.5-Pro277.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Nex-N2-Pro277.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
NorthCUA-Pro277.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-4.5250.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-4.7250.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-4.6249.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
GLM-5.3-Flash225.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Motif-3220.5 GB at Q4Won’t fitWon’t fitWon’t fit—
MiMo-V2.5217.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2.6-Flash-RL217.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2.6-Flash-MOPD217.7 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MiMo-V2-Flash217.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V4-Flash-Vision-Exp213.4 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V4-Flash-0731213.1 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
ERNIE-4.5-300B-A47B-PT210.5 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Hy3209.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Hy3-preview209.3 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Ling-plus205.0 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
DeepSeek-V4-Flash203.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
MemT-1.0-Flash201.8 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
ERNIE-5.5-Flash-Base200.6 GB at Q4 · MoEWon’t fitWon’t fitWon’t fit—
Inkling-Small186.4 GB at Q4 · MoETightWon’t fitWon’t fitup to 132
Llama-3_1-Nemotron-Ultra-253B-v1177.7 GB at Q4TightWon’t fitWon’t fit3.5–6.2
Solar-Open2-250B175.5 GB at Q4 · MoETightWon’t fitWon’t fitup to 106
Intern-S1168.8 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 72
K-EXAONE-236B-A23B166.3 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 69
DeepSeek-V2.5165.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 75
DeepSeek-Coder-V2-Instruct165.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 75
DeepSeek-V2.5-1210165.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 75
DeepSeek-V2-Chat-0628165.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 75
DeepSeek-Coder-V2-Instruct-0724165.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 75
Qwen3-VL-235B-A22B-Instruct165.3 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 72
Qwen3-235B-A22B164.9 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 72
Qwen3-235B-A22B-Instruct-2507164.9 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 72
Qwen3-235B-A22B-Thinking-2507164.9 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 72
MiniMax-M2160.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 158
MiniMax-M2.5160.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 158
MiniMax-M2.7160.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 158
MiniMax-M2.1160.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 158
command-a-plus-05-2026-bf16153.5 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 63
Step-3.7-Flash141.4 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 144
Step-3.5-Flash140.0 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 144
Step-3.5-Flash-Base-Midtrain138.9 GB at Q4 · MoEFitsWon’t fitWon’t fitup to 144
Qwen3.8-Flash-Next126.5 GB at Q4FitsWon’t fitWon’t fit4.9–8.8
Mixtral-8x22B-Instruct-v0.199.0 GB at Q4 · MoEFitsTightWon’t fitup to 41
Ling-3.0-flash-Sante89.9 GB at Q4 · MoEFitsFitsWon’t fitup to 311
Ling-3.0-flash89.9 GB at Q4 · MoEFitsFitsWon’t fitup to 311
Ling-3.0-flash-Fin89.9 GB at Q4 · MoEFitsFitsWon’t fitup to 311
Ornith-1.5-122B88.2 GB at Q4FitsFitsWon’t fit7.1–13
Qwen3.5-122B-A10B88.2 GB at Q4 · MoEFitsFitsWon’t fitup to 158
Devstral-2-123B-Instruct-251288.2 GB at Q4FitsFitsWon’t fit7.1–13
Ling-3.0-flash-VL88.0 GB at Q4 · MoEFitsFitsWon’t fitup to 288
NVIDIA-Nemotron-3-Super-120B-A12B-BF1687.2 GB at Q4 · MoEFitsFitsWon’t fitup to 132
Mistral-Small-4-119B-260384.2 GB at Q4 · MoEFitsFitsWon’t fitup to 244
Youtu-LLM-A7B-Base83.5 GB at Q4 · MoEFitsFitsWon’t fitup to 226
Laguna-S-2.183.0 GB at Q4 · MoEFitsFitsWon’t fitup to 198
gpt-oss-120b82.5 GB at Q4 · MoEFitsFitsWon’t fitup to 310
c4ai-command-a-03-202578.4 GB at Q4FitsFitsWon’t fit8.0–14
GLM-4.5-Air78.0 GB at Q4 · MoEFitsFitsWon’t fitup to 132
Llama-4-Scout-17B-16E-Instruct76.7 GB at Q4 · MoEFitsFitsWon’t fitup to 93
Llama-4-Scout-17B-16E76.7 GB at Q4 · MoEFitsFitsWon’t fitup to 93
GLM-4.5V76.1 GB at Q4 · MoEFitsFitsWon’t fitup to 132
GLM-4.6V76.1 GB at Q4 · MoEFitsFitsWon’t fitup to 132
Ling-2.6-flash75.9 GB at Q4 · MoEFitsFitsWon’t fitup to 214
sarvam-105b74.9 GB at Q4 · MoEFitsFitsWon’t fitup to 154
Ling-flash-2.072.7 GB at Q4 · MoEFitsFitsWon’t fitup to 260
LLaDA2.1-flash72.7 GB at Q4 · MoEFitsFitsWon’t fitup to 260
Ring-flash-2.072.7 GB at Q4 · MoEFitsFitsWon’t fitup to 260
LLaDA2.2-flash72.7 GB at Q4 · MoEFitsFitsWon’t fitup to 260
Solar-Open-100B72.6 GB at Q4 · MoEFitsFitsWon’t fitup to 132
Qwen3-Next-80B-A3B-Instruct57.7 GB at Q4 · MoEFitsFitsTightup to 528
Qwen3-Next-80B-A3B-Thinking57.7 GB at Q4 · MoEFitsFitsTightup to 528
AliceAI-Foundation-80B-A3B-Base57.7 GB at Q4 · MoEFitsFitsTightup to 528
Hunyuan-A13B-Instruct57.0 GB at Q4 · MoEFitsFitsTightup to 122
Qwen3-Coder-Next56.5 GB at Q4 · MoEFitsFitsTightup to 527
Kolibri-155.5 GB at Q4FitsFitsTight11–20
Qwen2.5-VL-72B-Instruct52.2 GB at Q4FitsFitsFits12–22
UI-TARS-72B-DPO52.2 GB at Q4FitsFitsFits12–22
Qwen2.5-72B-Instruct51.7 GB at Q4FitsFitsFits12–22
Apertus-70B-Instruct-250950.2 GB at Q4FitsFitsFits13–22
Apertus-70B-250950.2 GB at Q4FitsFitsFits13–22
Llama-3.3-70B-Instruct50.2 GB at Q4FitsFitsFits13–22
DeepSeek-R1-Distill-Llama-70B50.2 GB at Q4FitsFitsFits13–22
Llama-3.1-70B50.2 GB at Q4FitsFitsFits13–22
Hermes-2-Theta-Llama-3-70B50.2 GB at Q4FitsFitsFits13–22
Meta-Llama-3.1-70B-Instruct50.2 GB at Q4FitsFitsFits13–22
LongCat-Flash-Lite49.2 GB at Q4FitsFitsFits13–23
AI21-Jamba2-Mini37.0 GB at Q4 · MoEFitsFitsFitsup to 132
Llama-3_1-Nemotron-51B-Instruct36.9 GB at Q4FitsFitsFits17–31
Llama-3_3-Nemotron-Super-49B-v1_535.8 GB at Q4FitsFitsFits18–32
Llama-3_3-Nemotron-Super-49B-v135.8 GB at Q4FitsFitsFits18–32
Phi-3.5-MoE-instruct30.2 GB at Q4 · MoEFitsFitsFitsup to 240
GRIN-MoE30.2 GB at Q4 · MoEFitsFitsFitsup to 240
ALIA-40b29.2 GB at Q4FitsFitsFits22–39
IQuest-Coder-V1-40B-Loop-Instruct28.7 GB at Q4FitsFitsFits22–40
IQuest-Coder-V1-40B-Instruct28.7 GB at Q4FitsFitsFits22–40
Recursive Flint28.7 GB at Q4 · MoEFitsFitsFitsup to 634
K2-Horizon-MoVA-36B-A4B27.1 GB at Q4 · MoEFitsFitsFitsup to 396
Seed-OSS-36B-Instruct26.2 GB at Q4FitsFitsFits25–44
Seed-OSS-36B-Base-woSyn26.2 GB at Q4FitsFitsFits25–44
Intern-S2-Preview26.2 GB at Q4 · MoEFitsFitsFitsup to 528
Ornith-1.5-35B-A3B26.1 GB at Q4 · MoEFitsFitsFitsup to 528
Sage-A1.526.1 GB at Q4FitsFitsFits25–44
Qwen3.6-35B-A3B26.1 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3.5-35B-A3B26.1 GB at Q4 · MoEFitsFitsFitsup to 528
Apex-35B-V1.026.1 GB at Q4 · MoEFitsFitsFitsup to 528
MiniCPM-V-4.7-35B-A3B25.5 GB at Q4 · MoEFitsFitsFitsup to 528
MemT-1.0-Nano25.5 GB at Q4 · MoEFitsFitsFitsup to 528
Agents-A125.5 GB at Q4 · MoEFitsFitsFitsup to 528
Nex-N2.5-mini25.5 GB at Q4 · MoEFitsFitsFitsup to 528
Nex-N2-mini25.5 GB at Q4 · MoEFitsFitsFitsup to 528
Axion-1.0-35B25.5 GB at Q4 · MoEFitsFitsFitsup to 528
NorthCUA-Mini25.5 GB at Q4 · MoEFitsFitsFitsup to 528
Clover-35B-A3B-Preview25.2 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen-AgentWorld-35B-A3B25.2 GB at Q4 · MoEFitsFitsFitsup to 528
KAT-Coder-V2.5-Dev25.2 GB at Q4 · MoEFitsFitsFitsup to 528
EXAONE-4.5-33B24.9 GB at Q4FitsFitsFits26–46
Falcon-H1-34B-Instruct24.5 GB at Q4FitsFitsFits26–47
Qwen2.5-VL-32B-Instruct24.3 GB at Q4FitsFitsFits27–47
Laguna-XS.224.3 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-VL-32B-Instruct24.3 GB at Q4FitsFitsFits27–47
HyperCLOVAX-SEED-Think-32B24.2 GB at Q4FitsFitsFits27–48
FlexOlmo-7x7B-1T24.2 GB at Q4FitsFitsFits27–48
GLM-Z1-Rumination-32B-041424.1 GB at Q4FitsFitsFits27–48
Nanbeige4.3-33B24.1 GB at Q4FitsFitsFits27–48
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1624.0 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen2.5-Coder-32B-Instruct23.8 GB at Q4FitsFitsFits27–48
DeepSeek-R1-Distill-Qwen-32B23.8 GB at Q4FitsFitsFits27–48
Qwen2.5-32B-Instruct23.8 GB at Q4FitsFitsFits27–48
QwQ-32B23.8 GB at Q4FitsFitsFits27–48
QwQ-32B-Preview23.8 GB at Q4FitsFitsFits27–48
Qwen2.5-32B23.8 GB at Q4FitsFitsFits27–48
Qwen2.5-Coder-32B23.8 GB at Q4FitsFitsFits27–48
Qwen3-32B23.8 GB at Q4FitsFitsFits27–48
GLM-Z1-32B-041423.7 GB at Q4FitsFitsFits27–49
GLM-4-32B-041423.7 GB at Q4FitsFitsFits27–49
OLMo-2-0325-32B-Instruct23.5 GB at Q4FitsFitsFits28–49
OLMo-2-0325-32B23.5 GB at Q4FitsFitsFits28–49
Olmo-3-32B-Think23.5 GB at Q4FitsFitsFits28–49
Olmo-3-1125-32B23.5 GB at Q4FitsFitsFits28–49
Olmo-3.1-32B-Instruct23.5 GB at Q4FitsFitsFits28–49
Olmo-3.1-32B-Think23.5 GB at Q4FitsFitsFits28–49
Olmo-3.1-32B-Instruct-DPO23.5 GB at Q4FitsFitsFits28–49
granite-4.0-h-small23.5 GB at Q4 · MoEFitsFitsFitsup to 176
sarvam-30b23.4 GB at Q4 · MoEFitsFitsFitsup to 660
EXAONE-4.0-32B23.3 GB at Q4FitsFitsFits28–49
EXAONE-4.0.1-32B23.3 GB at Q4FitsFitsFits28–49
EXAONE-3.5-32B-Instruct23.3 GB at Q4FitsFitsFits28–49
EXAONE-Deep-32B23.3 GB at Q4FitsFitsFits28–49
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1623.0 GB at Q4 · MoEFitsFitsFitsup to 528
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1623.0 GB at Q4 · MoEFitsFitsFitsup to 528
Nemotron-Cascade-2-30B-A3B23.0 GB at Q4 · MoEFitsFitsFitsup to 528
Gemma 4 31B22.8 GB at Q4FitsFitsFits28–51
gemma-4-31B22.8 GB at Q4FitsFitsFits28–51
GLM-4.7-Flash22.8 GB at Q4 · MoEFitsFitsFitsup to 528
Xing4.0-29B-A4B22.8 GB at Q4 · MoEFitsFitsFitsup to 396
Xing4.0-29B-Intent22.8 GB at Q4 · MoEFitsFitsFitsup to 396
Qwen3-VL-30B-A3B-Instruct22.7 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-VL-30B-A3B-Thinking22.7 GB at Q4 · MoEFitsFitsFitsup to 528
UI-Venus-1.5-30B-A3B22.7 GB at Q4 · MoEFitsFitsFitsup to 528
kanana-2-30b-a3b-instruct-260122.4 GB at Q4 · MoEFitsFitsFitsup to 528
kanana-2-30b-a3b-instruct22.4 GB at Q4 · MoEFitsFitsFitsup to 528
kanana-2-30b-a3b-thinking-260122.4 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-30B-A3B22.3 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-Coder-30B-A3B-Instruct22.3 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-30B-A3B-Instruct-250722.3 GB at Q4 · MoEFitsFitsFitsup to 528
Qwen3-30B-A3B-Thinking-250722.3 GB at Q4 · MoEFitsFitsFitsup to 528
Tongyi-DeepResearch-30B-A3B22.3 GB at Q4 · MoEFitsFitsFitsup to 528
Rubicon-Preview22.3 GB at Q4 · MoEFitsFitsFitsup to 528
North-Mini-Code-1.022.3 GB at Q4 · MoEFitsFitsFitsup to 528
Hy-Memory-30B-A3B22.0 GB at Q4 · MoEFitsFitsFitsup to 528
Hy-MT2-30B-A3B22.0 GB at Q4 · MoEFitsFitsFitsup to 528
granite-switch-4.1-30b21.9 GB at Q4FitsFitsFits30–53
Muse-Glimmer-30B21.8 GB at Q4FitsFitsFits30–53
ERNIE-4.5-VL-28B-A3B-Thinking21.7 GB at Q4 · MoEFitsFitsFitsup to 528
ERNIE-4.5-VL-28B-A3B-PT21.5 GB at Q4 · MoEFitsFitsFitsup to 528
granite-4.2-30b21.4 GB at Q4FitsFitsFits30–54
granite-4.1-30b21.1 GB at Q4FitsFitsFits31–55
Raon-1 mini preview21.1 GB at Q4 · MoEFitsFitsFitsup to 368
Qwen3.8-27B20.4 GB at Q4FitsFitsFits32–57
Tess-4.5-27B20.4 GB at Q4FitsFitsFits32–57
Qwen3.6-27B20.4 GB at Q4FitsFitsFits32–57
Qwen3.5-27B20.4 GB at Q4FitsFitsFits32–57
gemma-3-27b-it20.1 GB at Q4FitsFitsFits32–58
Arete-27B20.1 GB at Q4FitsFitsFits33–58
gemma-2-27b-it20.0 GB at Q4FitsFitsFits33–58
gemma-2-27b20.0 GB at Q4FitsFitsFits33–58
Qwen3.8-27B-Longbook27-v219.9 GB at Q4FitsFitsFits33–58
medgemma-27b-text-it19.8 GB at Q4FitsFitsFits33–59
Hemmingway-1.519.8 GB at Q4FitsFitsFits33–59
Trinity-Mini19.2 GB at Q4 · MoEFitsFitsFitsup to 528
diffusiongemma-26B-A4B-it19.0 GB at Q4 · MoEFitsFitsFitsup to 396
Gemma 4 26B A4B19.0 GB at Q4 · MoEFitsFitsFitsup to 417
gemma-4-26B-A4B19.0 GB at Q4 · MoEFitsFitsFitsup to 396
Voxtral-Small-24B-250717.9 GB at Q4FitsFitsFits37–65
Mistral-Small-3.2-24B-Instruct-250617.7 GB at Q4FitsFitsFits37–66
Mistral-Small-3.1-24B-Instruct-250317.7 GB at Q4FitsFitsFits37–66
LFM2.5-24B-A2B17.7 GB at Q4 · MoEFitsFitsFitsup to 792
LFM2-24B-A2B17.6 GB at Q4 · MoEFitsFitsFitsup to 792
Mistral-Small-24B-Instruct-250117.4 GB at Q4FitsFitsFits38–67
EuroLLM-22B-Instruct-251216.8 GB at Q4FitsFitsFits39–70
EuroLLM-22B-251216.8 GB at Q4FitsFitsFits39–70
solar-pro-preview-instruct16.4 GB at Q4FitsFitsFits40–72
ERNIE-4.5-21B-A3B-PT16.3 GB at Q4 · MoEFitsFitsFitsup to 528
ERNIE-4.5-21B-A3B-Thinking16.2 GB at Q4 · MoEFitsFitsFitsup to 528
gpt-oss-20b15.6 GB at Q4 · MoEFitsFitsFitsup to 440
reka-flash-315.6 GB at Q4FitsFitsFits43–76
maple-preview15.1 GB at Q4 · MoEFitsFitsFitsup to 1583
internlm2_5-20b-chat14.8 GB at Q4FitsFitsFits45–80
Groot-19B-A3B-26102314.5 GB at Q4 · MoEFitsFitsFitsup to 528
Ling-Coder-lite12.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ring-lite-distill-preview12.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ling-lite12.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ling-lite-1.5-250712.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ling-lite-1.512.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ring-lite12.7 GB at Q4 · MoEFitsFitsFitsup to 576
Ring-mini-linear-2.012.5 GB at Q4 · MoEFitsFitsFitsup to 990
Kimi-VL-A3B-Instruct12.4 GB at Q4 · MoEFitsFitsFitsup to 528
Kimi-VL-A3B-Thinking12.4 GB at Q4 · MoEFitsFitsFitsup to 528
Kimi-VL-A3B-Thinking-250612.4 GB at Q4 · MoEFitsFitsFitsup to 528
LLaDA2.1-mini12.3 GB at Q4 · MoEFitsFitsFitsup to 1131
LLaDA2.0-mini12.3 GB at Q4 · MoEFitsFitsFitsup to 1131
Ling-mini-2.012.3 GB at Q4 · MoEFitsFitsFitsup to 1131
Ring-mini-2.012.3 GB at Q4 · MoEFitsFitsFitsup to 1131
LLaDA2.2-mini12.3 GB at Q4 · MoEFitsFitsFitsup to 1131
LLaDA2.0-mini-preview12.3 GB at Q4 · MoEFitsFitsFitsup to 1131
Moonlight-16B-A3B-Instruct12.1 GB at Q4 · MoEFitsFitsFitsup to 528
Moonlight-16B-A3B12.1 GB at Q4 · MoEFitsFitsFitsup to 528
Instella-MoE-16B-A3B-Think12.1 GB at Q4 · MoEFitsFitsFitsup to 528
DeepSeek-Coder-V2-Lite-Instruct12.0 GB at Q4 · MoEFitsFitsFitsup to 660
kanana-1.5-15.7b-a3b-instruct11.9 GB at Q4 · MoEFitsFitsFitsup to 528
Phi-4-reasoning-vision-15B11.5 GB at Q4FitsFitsFits59–105
Apriel-Nemotron-15b-Thinker11.4 GB at Q4FitsFitsFits59–106
Apriel-1.5-15b-Thinker11.4 GB at Q4FitsFitsFits60–107
Apriel-1.6-15b-Thinker11.4 GB at Q4FitsFitsFits60–107
Carbon-X (DNA)11.3 GB at Q4FitsFitsFits60–107
Qwen2.5-Coder-14B-Instruct11.3 GB at Q4FitsFitsFits60–107
DeepSeek-R1-Distill-Qwen-14B11.3 GB at Q4FitsFitsFits60–107
Qwen2.5-14B-Instruct11.3 GB at Q4FitsFitsFits60–107
Qwen2.5-14B11.3 GB at Q4FitsFitsFits60–107
Qwen2.5-14B-Instruct-1M11.3 GB at Q4FitsFitsFits60–107
Qwen2.5-Coder-14B11.3 GB at Q4FitsFitsFits60–107
Qwen3-14B11.3 GB at Q4FitsFitsFits60–107
JanusCoder-14B11.3 GB at Q4FitsFitsFits60–107
HyperCLOVAX-SEED-Think-14B11.3 GB at Q4FitsFitsFits60–107
phi-411.2 GB at Q4FitsFitsFits61–108
Phi-4-reasoning11.2 GB at Q4FitsFitsFits61–108
Phi-4-reasoning-plus11.2 GB at Q4FitsFitsFits61–108
OLMo-2-1124-13B-Instruct10.6 GB at Q4FitsFitsFits65–115
Mistral-Nemo-Instruct-24079.5 GB at Q4FitsFitsFits73–129
gemma-3-12b-it9.5 GB at Q4FitsFitsFits73–130
gemma-3-12b-pt9.5 GB at Q4FitsFitsFits73–130
Mellum2.1-12B-A2.5B-Thinking9.5 GB at Q4 · MoEFitsFitsFitsup to 633
Mellum2-12B-A2.5B-Thinking9.5 GB at Q4 · MoEFitsFitsFitsup to 633
Mellum2-12B-A2.5B-Instruct9.5 GB at Q4 · MoEFitsFitsFitsup to 633
Mellum2-12B-A2.5B-Thinking-SFT9.5 GB at Q4 · MoEFitsFitsFitsup to 633
Mellum2.1-12B-A2.5B-Thinking-MTP9.5 GB at Q4 · MoEFitsFitsFitsup to 633
Gemma 4 12B9.3 GB at Q4FitsFitsFits74–132
HyperCLOVAX-SEED-Omni-8B8.5 GB at Q4FitsFitsFits83–147
Llama-3.2-11B-Vision-Instruct8.4 GB at Q4FitsFitsFits83–148
Llama-3.2-11B-Vision8.4 GB at Q4FitsFitsFits84–149
Falcon3-10B-Instruct8.2 GB at Q4FitsFitsFits86–154
GLM-4.1V-9B-Thinking8.2 GB at Q4FitsFitsFits87–154
GLM-4.6V-Flash8.2 GB at Q4FitsFitsFits87–154
GLM-4.1V-9B-Base8.2 GB at Q4FitsFitsFits87–154
Glyph8.2 GB at Q4FitsFitsFits87–154
INTELLECT-1-Instruct8.1 GB at Q4FitsFitsFits87–155
INTELLECT-18.1 GB at Q4FitsFitsFits87–155
ZAYA1-VL-8B7.8 GB at Q4FitsFitsFits92–163
SenseNova-Loop-9B-Preview7.7 GB at Q4FitsFitsFits92–164
Ornith-1.5-9B7.7 GB at Q4FitsFitsFits92–164
Qwen3.5-9B7.7 GB at Q4FitsFitsFits92–164
MiniCPM-SALA7.6 GB at Q4FitsFitsFits94–167
NorthCUA-Nano7.6 GB at Q4FitsFitsFits95–168
GLM-4-9B-04147.6 GB at Q4FitsFitsFits95–168
GLM-Z1-9B-04147.6 GB at Q4FitsFitsFits95–168
codegeex4-all-9b7.6 GB at Q4FitsFitsFits95–168
glm-4-9b-chat-hf7.6 GB at Q4FitsFitsFits95–168
moondream3-preview7.5 GB at Q4FitsFitsFits96–171
gemma-2-9b-it7.4 GB at Q4FitsFitsFits96–171
gemma-2-9b7.4 GB at Q4FitsFitsFits96–171
Gemma-SEA-LION-v3-9B-IT7.4 GB at Q4FitsFitsFits96–171
EuroLLM-9B-Instruct-25127.4 GB at Q4FitsFitsFits97–173
NVIDIA-Nemotron-Nano-9B-v27.2 GB at Q4FitsFitsFits100–178
ZAYA1-base7.2 GB at Q4FitsFitsFits101–179
Yi-Coder-9B-Chat7.2 GB at Q4FitsFitsFits101–179
Yi-Coder-9B7.2 GB at Q4FitsFitsFits101–179
internlm3-8b-instruct7.1 GB at Q4FitsFitsFits101–180
granite-4.1-8b7.1 GB at Q4FitsFitsFits101–180
granite-4.2-8b7.1 GB at Q4FitsFitsFits101–180
granite-switch-4.1-8b7.1 GB at Q4FitsFitsFits101–180
Qwen3-VL-8B-Instruct7.1 GB at Q4FitsFitsFits102–181
Qwen3-VL-8B-Thinking7.1 GB at Q4FitsFitsFits102–181
UI-Venus-1.5-8B7.1 GB at Q4FitsFitsFits102–181
Llama-3.1-Nemotron-Nano-VL-8B-V17.1 GB at Q4FitsFitsFits102–182
MiniCPM-V-4_57.1 GB at Q4FitsFitsFits102–182
aya-vision-8b7.0 GB at Q4FitsFitsFits103–183
Intern-S1-mini7.0 GB at Q4FitsFitsFits104–185
Nemotron-Labs-Diffusion-8B-Base6.9 GB at Q4FitsFitsFits105–187
LFM2.5-8B-A1B6.9 GB at Q4 · MoEFitsFitsFitsup to 1583
Mistral-NeMo-Minitron-8B-Base6.9 GB at Q4FitsFitsFits106–188
LFM2-8B-A1B6.8 GB at Q4 · MoEFitsFitsFitsup to 1584
rnj-1-instruct6.8 GB at Q4FitsFitsFits107–191
MiMo-VL-7B-RL-25086.8 GB at Q4FitsFitsFits107–191
MiMo-VL-7B-RL6.8 GB at Q4FitsFitsFits107–191
MiMo-Embodied-7B6.8 GB at Q4FitsFitsFits107–191
MiMo-VL-7B-SFT6.8 GB at Q4FitsFitsFits107–191
Qwen2.5-VL-7B-Instruct6.8 GB at Q4FitsFitsFits107–191
UI-TARS-1.5-7B6.8 GB at Q4FitsFitsFits107–191
Fara-7B6.8 GB at Q4FitsFitsFits107–191
UI-Venus-Ground-7B6.8 GB at Q4FitsFitsFits107–191
Qwen2-VL-7B-Instruct6.8 GB at Q4FitsFitsFits107–191
UI-TARS-7B-DPO6.8 GB at Q4FitsFitsFits107–191
UI-TARS-7B-SFT6.8 GB at Q4FitsFitsFits107–191
Seed-Coder-8B-Instruct6.8 GB at Q4FitsFitsFits108–192
Seed-Coder-8B-Reasoning6.8 GB at Q4FitsFitsFits108–192
Stable-DiffCoder-8B-Instruct6.8 GB at Q4FitsFitsFits108–192
Qwen3-8B6.7 GB at Q4FitsFitsFits109–193
DeepSeek-R1-0528-Qwen3-8B6.7 GB at Q4FitsFitsFits109–193
Qwen3-8B-Base6.7 GB at Q4FitsFitsFits109–193
WeDLM-8B-Instruct6.7 GB at Q4FitsFitsFits109–193
PaCoRe-8B6.7 GB at Q4FitsFitsFits109–193
MiniCPM4-8B6.7 GB at Q4FitsFitsFits109–193
MiniCPM4.1-8B6.7 GB at Q4FitsFitsFits109–193
AgentCPM-Report6.7 GB at Q4FitsFitsFits109–193
MiniCPM4-MCP6.7 GB at Q4FitsFitsFits109–193
granite-3.3-8b-instruct6.7 GB at Q4FitsFitsFits109–194
granite-3.1-8b-instruct6.7 GB at Q4FitsFitsFits109–194
granite-3.2-8b-instruct6.7 GB at Q4FitsFitsFits109–194
granite-3.0-8b-instruct6.7 GB at Q4FitsFitsFits109–194
Apertus-8B-Instruct-25096.6 GB at Q4FitsFitsFits111–197
Apertus-8B-25096.6 GB at Q4FitsFitsFits111–197
Llama-3.1-Tulu-3-8B6.6 GB at Q4FitsFitsFits111–197
Llama-3.1-Tulu-3-8B-DPO6.6 GB at Q4FitsFitsFits111–197
Llama-3.1-Tulu-3.1-8B6.6 GB at Q4FitsFitsFits111–197
kanana-1.5-8b-instruct-25056.6 GB at Q4FitsFitsFits111–197
Llama-3.1-8B-Instruct6.6 GB at Q4FitsFitsFits111–197
Llama-3.1-8B6.6 GB at Q4FitsFitsFits111–197
DeepSeek-R1-Distill-Llama-8B6.6 GB at Q4FitsFitsFits111–197
Meta-Llama-3.1-8B-Instruct6.6 GB at Q4FitsFitsFits111–197
L3-8B-Stheno-v3.26.6 GB at Q4FitsFitsFits111–197
Llama-3.1-Nemotron-Nano-8B-v16.6 GB at Q4FitsFitsFits111–197
Meta-Llama-3.1-8B6.6 GB at Q4FitsFitsFits111–197
LongWriter-llama3.1-8b6.6 GB at Q4FitsFitsFits111–197
Hy-MT2-7B6.6 GB at Q4FitsFitsFits111–197
Nemotron 3.5 Bolt E4B6.6 GB at Q4FitsFitsFits111–198
Gemma 4 E4B6.6 GB at Q4FitsFitsFits111–198
Ling-3.0-tiny6.5 GB at Q4 · MoEFitsFitsFitsup to 1218
MiMo-7B-RL6.5 GB at Q4FitsFitsFits114–202
MiMo-7B-Base6.5 GB at Q4FitsFitsFits114–202
MiMo-7B-RL-05306.5 GB at Q4FitsFitsFits114–202
MiMo-7B-SFT6.5 GB at Q4FitsFitsFits114–202
EXAONE-3.5-7.8B-Instruct6.5 GB at Q4FitsFitsFits114–203
EXAONE-Deep-7.8B6.5 GB at Q4FitsFitsFits114–203
salamandra-7b-instruct6.4 GB at Q4FitsFitsFits115–204
salamandra-7b6.4 GB at Q4FitsFitsFits115–204
salamandra-7b-instruct-26066.4 GB at Q4FitsFitsFits115–204
FastVLM-7B6.4 GB at Q4FitsFitsFits115–204
Molmo2-O-7B6.4 GB at Q4FitsFitsFits115–204
internlm2_5-7b-chat6.4 GB at Q4FitsFitsFits115–205
internlm2_5-7b-chat-1m6.4 GB at Q4FitsFitsFits115–205
Phi-mini-MoE-instruct6.3 GB at Q4 · MoEFitsFitsFitsup to 660
Qwen2.5-Coder-7B-Instruct6.3 GB at Q4FitsFitsFits117–208
DeepSeek-R1-Distill-Qwen-7B6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-7B-Instruct6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-7B6.3 GB at Q4FitsFitsFits117–208
Qwen2-7B-Instruct6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-Coder-7B6.3 GB at Q4FitsFitsFits117–208
Qwen2-7B6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-7B-Instruct-1M6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-Math-7B6.3 GB at Q4FitsFitsFits117–208
Qwen2.5-Math-7B-Instruct6.3 GB at Q4FitsFitsFits117–208
Marco-o16.3 GB at Q4FitsFitsFits117–208
SeaLLMs-v3-7B-Chat6.3 GB at Q4FitsFitsFits117–208
Arcee-Spark6.3 GB at Q4FitsFitsFits117–208
Falcon-H1-7B-Instruct6.3 GB at Q4FitsFitsFits117–209
Falcon-H1R-7B6.3 GB at Q4FitsFitsFits117–209
starvector-8b-im2svg6.2 GB at Q4FitsFitsFits119–211
Falcon3-7B-Instruct6.2 GB at Q4FitsFitsFits119–212
Olmo-Hybrid-7B6.2 GB at Q4FitsFitsFits120–213
LLaDA-MoE-7B-A1B-Base6.1 GB at Q4 · MoEFitsFitsFitsup to 1584
Zamba2-7B-Instruct6.1 GB at Q4FitsFitsFits121–215
OLMo-2-1124-7B-Instruct6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Think6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Instruct6.1 GB at Q4FitsFitsFits122–217
Olmo-3-1025-7B6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Instruct-SFT6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Think-SFT6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Instruct-DPO6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-RL-Zero-Math6.1 GB at Q4FitsFitsFits122–217
Olmo-3.1-7B-RL-Zero-Math6.1 GB at Q4FitsFitsFits122–217
Olmo-3-7B-Think-DPO6.1 GB at Q4FitsFitsFits122–217
Mistral-7B-Instruct-v0.36.1 GB at Q4FitsFitsFits123–218
granite-4.0-h-tiny5.8 GB at Q4 · MoEFitsFitsFitsup to 1584
OLMoE-1B-7B-0924-Instruct5.8 GB at Q4 · MoEFitsFitsFitsup to 1218
OLMoE-1B-7B-09245.8 GB at Q4 · MoEFitsFitsFitsup to 1218
OLMoE-1B-7B-0125-Instruct5.8 GB at Q4 · MoEFitsFitsFitsup to 1218
OLMoE-1B-7B-01255.8 GB at Q4 · MoEFitsFitsFitsup to 1218
OLMo-7B-0724-hf5.8 GB at Q4FitsFitsFits129–230
granite-4.0-tiny-preview5.7 GB at Q4 · MoEFitsFitsFitsup to 1583
YuE-s1-7B-anneal-en-cot5.3 GB at Q4FitsFitsFits143–254
Trinity-Nano-Preview5.3 GB at Q4 · MoEFitsFitsFitsup to 1583
Trinity-Nano-Base5.3 GB at Q4 · MoEFitsFitsFitsup to 1583
gemma-3n-E2B-it4.8 GB at Q4FitsFitsFits164–291
Youtu-VL-4B-Instruct4.7 GB at Q4FitsFitsFits167–297
Gemma 4 E2B4.6 GB at Q4FitsFitsFits174–309
Molmo2-ER4.4 GB at Q4FitsFitsFits184–326
Apriel-5B-Instruct4.4 GB at Q4FitsFitsFits184–328
WeVisOne-VL-4B4.4 GB at Q4FitsFitsFits184–328
Mage-VL4.3 GB at Q4FitsFitsFits188–334
Mage-VL-Cowork4.3 GB at Q4FitsFitsFits188–334
Qwen3.5-4B4.2 GB at Q4FitsFitsFits191–340
S5-nano4.2 GB at Q4FitsFitsFits191–340
AFM-4.5B4.2 GB at Q4FitsFitsFits193–343
Llama-3.1-Minitron-4B-Width-Base4.1 GB at Q4FitsFitsFits197–351
Qwen3-VL-4B-Instruct4.1 GB at Q4FitsFitsFits201–357
CapRL-Qwen3VL-4B4.1 GB at Q4FitsFitsFits201–357
glm-edge-4b-chat4.0 GB at Q4FitsFitsFits206–366
gemma-3-4b-it4.0 GB at Q4FitsFitsFits207–368
medgemma-4b-it4.0 GB at Q4FitsFitsFits207–368
medgemma-1.5-4b-it4.0 GB at Q4FitsFitsFits207–368
gemma-3-4b-pt4.0 GB at Q4FitsFitsFits207–368
Nanbeige4.2-3B3.9 GB at Q4FitsFitsFits214–380
Phi-3.5-vision-instruct3.9 GB at Q4FitsFitsFits215–382
Spark-X2.6-4B3.9 GB at Q4FitsFitsFits217–385
Spark-X2.5-4B3.9 GB at Q4FitsFitsFits217–385
POINTS-Reader3.9 GB at Q4FitsFitsFits217–385
MiniCPM-V-43.8 GB at Q4FitsFitsFits219–390
Qwen3-4B3.8 GB at Q4FitsFitsFits221–394
Qwen3-4B-Instruct-25073.8 GB at Q4FitsFitsFits221–394
Qwen3-4B-Thinking-25073.8 GB at Q4FitsFitsFits221–394
Mellum-4b-base3.8 GB at Q4FitsFitsFits222–394
granite-4.0-3b-vision3.8 GB at Q4FitsFitsFits223–396
granite-vision-4.1-4b3.8 GB at Q4FitsFitsFits223–396
Nemotron Labs Dory 1.0 4B3.8 GB at Q4FitsFitsFits224–398
NVIDIA-Nemotron-3-Nano-4B-BF163.8 GB at Q4FitsFitsFits224–399
Nanbeige4.1-3B3.7 GB at Q4FitsFitsFits226–403
Nanbeige4-3B-Thinking-25113.7 GB at Q4FitsFitsFits226–403
Phi-4-mini-flash-reasoning3.7 GB at Q4FitsFitsFits231–411
Phi-4-mini-instruct3.7 GB at Q4FitsFitsFits232–413
Phi-4-mini-reasoning3.7 GB at Q4FitsFitsFits232–413
Nemotron-Labs-Diffusion-3B3.7 GB at Q4FitsFitsFits232–413
Phi-3.5-mini-instruct3.7 GB at Q4FitsFitsFits233–414
MediPhi-Instruct3.7 GB at Q4FitsFitsFits233–414
MediPhi3.7 GB at Q4FitsFitsFits233–414
MediPhi-Clinical3.7 GB at Q4FitsFitsFits233–414
MediPhi-PubMed3.7 GB at Q4FitsFitsFits233–414
HY-Embodied-0.53.6 GB at Q4FitsFitsFits235–418
Phi-tiny-MoE-instruct3.6 GB at Q4 · MoEFitsFitsFitsup to 1440
CapRL-3B3.6 GB at Q4FitsFitsFits237–422
HyperCLOVAX-SEED-Vision-Instruct-3B3.6 GB at Q4FitsFitsFits239–426
kanana-1.5-v-3b-instruct3.6 GB at Q4FitsFitsFits243–432
granite-4.2-3b3.6 GB at Q4FitsFitsFits243–433
kanana-2-3b-instruct3.4 GB at Q4FitsFitsFits254–451
granite-4.1-3b3.4 GB at Q4FitsFitsFits262–465
granite-4.0-micro3.4 GB at Q4FitsFitsFits262–465
granite-3.1-3b-a800m-instruct3.3 GB at Q4 · MoEFitsFitsFitsup to 1979
Falcon3-3B-Instruct3.3 GB at Q4FitsFitsFits276–491
Llama-3.2-3B-Instruct3.2 GB at Q4FitsFitsFits277–493
Llama-3.2-3B3.2 GB at Q4FitsFitsFits277–493
Hermes-3-Llama-3.2-3B3.2 GB at Q4FitsFitsFits277–493
DeepHermes-3-Llama-3-3B-Preview3.2 GB at Q4FitsFitsFits277–493
AI21-Jamba-Reasoning-3B3.2 GB at Q4FitsFitsFits279–495
granite-4.0-h-micro3.2 GB at Q4FitsFitsFits279–496
Falcon-H1-3B-Instruct3.2 GB at Q4FitsFitsFits283–503
d1-3B3.2 GB at Q4FitsFitsFits285–507
LFM2.5-VL-3B3.2 GB at Q4FitsFitsFits285–507
Qwen2.5-3B-Instruct3.2 GB at Q4FitsFitsFits289–513
Qwen2.5-Coder-3B-Instruct3.2 GB at Q4FitsFitsFits289–513
Qwen2.5-3B3.2 GB at Q4FitsFitsFits289–513
Qwen2.5-Coder-3B3.2 GB at Q4FitsFitsFits289–513
SmolLM3-3B3.1 GB at Q4FitsFitsFits290–515
AI21-Jamba2-3B3.1 GB at Q4FitsFitsFits294–523
granite-swash-3b-a600m3.1 GB at Q4 · MoEFitsFitsFitsup to 2648
granite-vision-3.2-2b3.1 GB at Q4FitsFitsFits299–532
granite-vision-3.1-2b-preview3.1 GB at Q4FitsFitsFits299–532
granite-vision-3.3-2b-chart2csv-preview3.1 GB at Q4FitsFitsFits299–532
LFM2.5-2.6B2.9 GB at Q4FitsFitsFits330–587
Zamba2-2.7B2.9 GB at Q4FitsFitsFits335–595
Zamba2-2.7B-instruct2.9 GB at Q4FitsFitsFits335–595
granite-3.0-2b-instruct2.8 GB at Q4FitsFitsFits338–601
gemma-2-2b-it2.8 GB at Q4FitsFitsFits341–606
gemma-2-2b2.8 GB at Q4FitsFitsFits341–606
EuroMoE-2.6B-A0.6B-Instruct-Preview2.8 GB at Q4 · MoEFitsFitsFitsup to 2639
Motif-2.6B2.8 GB at Q4FitsFitsFits343–610
LFM2-2.6B2.8 GB at Q4FitsFitsFits347–616
LFM2-2.6B-Exp2.8 GB at Q4FitsFitsFits347–616
LFM2-2.6B-Transcript2.8 GB at Q4FitsFitsFits347–616
granite-3.3-2b-instruct2.8 GB at Q4FitsFitsFits352–625
granite-3.1-2b-instruct2.8 GB at Q4FitsFitsFits352–625
granite-3.2-2b-instruct2.8 GB at Q4FitsFitsFits352–625
MiniCPM5-2B2.8 GB at Q4FitsFitsFits354–629
MiniCPM5-2B-SFT2.8 GB at Q4FitsFitsFits354–629
MiniCPM5-2B-Midtrain2.8 GB at Q4FitsFitsFits354–629
sarvam-1-v0.52.7 GB at Q4FitsFitsFits355–631
North-Micro-Vision-Instruct2.7 GB at Q4FitsFitsFits358–637
UI-TARS-2B-SFT2.7 GB at Q4FitsFitsFits365–648
WeVisOne-VL-2B2.7 GB at Q4FitsFitsFits365–649
UI-Venus-1.5-2B2.7 GB at Q4FitsFitsFits365–649
bitnet-b1.58-2B-4T-bf162.7 GB at Q4FitsFitsFits369–656
EXAONE-3.5-2.4B-Instruct2.7 GB at Q4FitsFitsFits370–658
EXAONE-Deep-2.4B2.7 GB at Q4FitsFitsFits370–658
kanana-1.5-2.1b-instruct-25052.6 GB at Q4FitsFitsFits384–683
Qwen3.5-2B2.6 GB at Q4FitsFitsFits392–696
salamandra-2b2.6 GB at Q4FitsFitsFits395–703
salamandra-2b-instruct2.6 GB at Q4FitsFitsFits395–703
SmolVLM2-2.2B-Instruct2.6 GB at Q4FitsFitsFits396–705
SmolVLM-Base2.6 GB at Q4FitsFitsFits397–705
Qwen2-VL-2B-Instruct2.5 GB at Q4FitsFitsFits403–717
LFM3-Omni-2B2.5 GB at Q4FitsFitsFits411–731
granite-swash-2b2.5 GB at Q4FitsFitsFits415–738
Qwen3-VL-2B-Instruct2.5 GB at Q4FitsFitsFits419–744
kanana-nano-2.1b-instruct2.5 GB at Q4FitsFitsFits427–759
Hy-MT2-1.8B2.4 GB at Q4FitsFitsFits437–777
Qwen3-1.7B2.4 GB at Q4FitsFitsFits438–779
Youtu-LLM-2B2.4 GB at Q4FitsFitsFits454–807
LFM3-VL-2B2.3 GB at Q4FitsFitsFits464–826
d1-2B2.3 GB at Q4FitsFitsFits464–826
FastVLM-1.5B2.3 GB at Q4FitsFitsFits467–829
internlm2_5-1_8b-chat2.3 GB at Q4FitsFitsFits472–838
DeepSeek-R1-Distill-Qwen-1.5B2.2 GB at Q4FitsFitsFits501–891
LFM3-1.7B2.2 GB at Q4FitsFitsFits515–916
Qwen3-1.7B-Base2.2 GB at Q4FitsFitsFits518–920
SmolLM2-1.7B2.2 GB at Q4FitsFitsFits520–925
SmolLM-1.7B2.2 GB at Q4FitsFitsFits520–925
Spark-X2.6-1.7B2.2 GB at Q4FitsFitsFits522–927
Falcon3-1B-Instruct2.2 GB at Q4FitsFitsFits534–949
EuroLLM-1.7B-Instruct2.2 GB at Q4FitsFitsFits538–956
granite-4.0-1b2.1 GB at Q4FitsFitsFits546–970
LFM2.5-VL-1.6B2.1 GB at Q4FitsFitsFits558–992
glm-edge-1.5b-chat2.1 GB at Q4FitsFitsFits559–994
LFM2-VL-1.6B2.1 GB at Q4FitsFitsFits562–999
Falcon-H1-1.5B-Deep-Instruct2.1 GB at Q4FitsFitsFits573–1018
Falcon-H1-1.5B-Instruct2.1 GB at Q4FitsFitsFits573–1018
MiniCPM-V-4.7-1B2.1 GB at Q4FitsFitsFits573–1019
Qwen2.5-1.5B-Instruct2.1 GB at Q4FitsFitsFits577–1026
Qwen2.5-1.5B2.1 GB at Q4FitsFitsFits577–1026
Qwen2.5-Coder-1.5B-Instruct2.1 GB at Q4FitsFitsFits577–1026
Qwen2.5-Coder-1.5B2.1 GB at Q4FitsFitsFits577–1026
Qwen2.5-Math-1.5B2.1 GB at Q4FitsFitsFits577–1026
Qwen2-1.5B-Instruct2.1 GB at Q4FitsFitsFits577–1026
Qwen2.5-Math-1.5B-Instruct2.1 GB at Q4FitsFitsFits577–1026
ReaderLM-v22.1 GB at Q4FitsFitsFits577–1026
reader-lm-1.5b2.1 GB at Q4FitsFitsFits577–1026
Hymba-1.5B-Instruct2.1 GB at Q4FitsFitsFits585–1040
OLMo-2-0425-1B2.0 GB at Q4FitsFitsFits600–1066
OLMo-2-0425-1B-Instruct2.0 GB at Q4FitsFitsFits600–1066
Yi-Coder-1.5B-Chat2.0 GB at Q4FitsFitsFits603–1072
Bolmo-1B2.0 GB at Q4FitsFitsFits606–1078
granite-4.0-h-1b2.0 GB at Q4FitsFitsFits609–1083
granite-3.1-1b-a400m-instruct1.9 GB at Q4 · MoEFitsFitsFitsup to 3959
granite-3.0-1b-a400m-instruct1.9 GB at Q4 · MoEFitsFitsFitsup to 3959
MiniCPM-V-4.61.9 GB at Q4FitsFitsFits685–1218
MiniCPM-V-4.6-Thinking1.9 GB at Q4FitsFitsFits685–1218
kanana-2-1.3b-instruct1.9 GB at Q4FitsFitsFits690–1226
OLMo-1B-0724-hf1.9 GB at Q4FitsFitsFits696–1237
EXAONE-4.0-1.2B1.9 GB at Q4FitsFitsFits696–1238
Llama-3.2-1B-Instruct1.9 GB at Q4FitsFitsFits721–1281
Llama-3.2-1B1.9 GB at Q4FitsFitsFits721–1281
Llama-3.2-1B1.9 GB at Q4FitsFitsFits721–1281
Zamba2-1.2B-instruct1.8 GB at Q4FitsFitsFits733–1303
HRM-Text-1B1.8 GB at Q4FitsFitsFits753–1339
AMD-OLMo-1B1.8 GB at Q4FitsFitsFits757–1346
LFM2.5-1.2B-Instruct1.8 GB at Q4FitsFitsFits761–1353
LFM2-1.2B1.8 GB at Q4FitsFitsFits761–1353
LFM2.5-1.2B-Thinking1.8 GB at Q4FitsFitsFits761–1353
LFM2.5-1.2B-JP-2026061.8 GB at Q4FitsFitsFits761–1353
MiniCPM5-1B1.8 GB at Q4FitsFitsFits824–1465
MiniCPM5-1B-SFT1.8 GB at Q4FitsFitsFits824–1465
gemma-3-1b-it1.7 GB at Q4FitsFitsFits891–1584
gemma-3-1b-pt1.7 GB at Q4FitsFitsFits891–1584
Qwen3.5-0.8B1.6 GB at Q4FitsFitsFits1020–1813
Falcon-E-3B-Instruct1.6 GB at Q4FitsFitsFits1031–1832
bitnet-b1.58-2B-4T1.6 GB at Q4FitsFitsFits1048–1863
FastVLM-0.5B1.5 GB at Q4FitsFitsFits1174–2087
Qwen3-0.6B1.5 GB at Q4FitsFitsFits1185–2107
LFM2-700M1.5 GB at Q4FitsFitsFits1200–2133
Falcon-H1-Tiny-R-0.6B1.4 GB at Q4FitsFitsFits1431–2544
Qwen3-0.6B-Base1.4 GB at Q4FitsFitsFits1494–2657
d1-omni-600M1.4 GB at Q4FitsFitsFits1517–2697
HyperCLOVAX-SEED-Text-Instruct-0.5B1.4 GB at Q4FitsFitsFits1573–2796
Falcon-H1-0.5B-Instruct1.4 GB at Q4FitsFitsFits1708–3037
Qwen2.5-0.5B-Instruct1.3 GB at Q4FitsFitsFits1803–3205
Qwen2.5-0.5B1.3 GB at Q4FitsFitsFits1803–3205
Qwen2-0.5B-Instruct1.3 GB at Q4FitsFitsFits1803–3205
Qwen2.5-Coder-0.5B-Instruct1.3 GB at Q4FitsFitsFits1803–3205
LFM2-VL-450M1.3 GB at Q4FitsFitsFits1976–3512
LFM2.5-VL-450M1.3 GB at Q4FitsFitsFits1985–3529
Apertus-v1.1-0.5B1.3 GB at Q4FitsFitsFits2032–3612
MiniCPM4-0.5B1.3 GB at Q4FitsFitsFits2053–3650
circuit-sparsity1.3 GB at Q4FitsFitsFits2125–3778
SmolLM2-360M1.3 GB at Q4FitsFitsFits2462–4376
SmolLM-360M1.3 GB at Q4FitsFitsFits2462–4376
ERNIE-4.5-0.3B-PT1.3 GB at Q4FitsFitsFits2469–4390
LFM2.5-350M1.2 GB at Q4FitsFitsFits2513–4467
LFM2-350M1.2 GB at Q4FitsFitsFits2513–4467
granite-4.0-350m1.2 GB at Q4FitsFitsFits2528–4494
granite-4.0-h-350m1.2 GB at Q4FitsFitsFits2617–4653

Copy the parts list

Open in planner and tweak Compare with other builds

New models, and what they need to run

One short email a week: fresh open models with their VRAM numbers, plus hardware worth watching.