What Can I Run AI?

What LLMs can I run on NVIDIA RTX Pro 6000 Blackwell 96GB?

You can run 545 of the 656 open models we track fully on the NVIDIA RTX Pro 6000 Blackwell 96GB at Q4_K_M, including Mistral-Small-4-119B-2603 (119.4B parameters, about 84.2 GB). It has 96 GB of usable VRAM. These are estimates, not measured benchmarks.

Specs and list price: NVIDIA RTX Pro 6000 Blackwell 96GB hardware page.

Memory
96 GB VRAM
Memory bandwidth
1792 GB/s
Usable for models
96 GB
Runs fully at Q4
545 / 656

Spec source · Best GPU for local LLMs · Multi-GPU · NVIDIA vs AMD vs Mac · Tokens per second · Best LLMs for 96 GB

$8565 list price (MSRP), source, 2026-10-28
Street prices may be higher than list.

Check price on AmazonCheck price on eBay

We may earn a commission when you buy through these links, at no extra cost to you.

Compare: NVIDIA RTX 6000 Ada 48GB vs NVIDIA RTX Pro 6000 Blackwell 96GB · GeForce RTX 5090 vs NVIDIA RTX Pro 6000 Blackwell 96GB

Every tracked model on this machine

Sorted largest first. The tick marks 90% of usable memory.FitsTightWon’t fit
ModelQ4 need vs. 96 GBVerdict
Kimi-K3.1-Pro2780.1B params1939.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K32779.9B params1939.1 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-2.4T-A95B2446.2B params1706.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
LongCat-2.51779.8B params1241.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
LongCat-2.01775.6B params1238.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro-08131650.5B params1151.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro1598.8B params1115.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.51026.9B params716.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.61026.9B params716.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.7-Code1026.9B params716.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Thinking1026.4B params716.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Instruct-09051026.4B params716.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Instruct1026.4B params716.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-RL1024.2B params715.1 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-MOPD1024.2B params715.1 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.5-Pro1023.2B params714.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ring-2.5-1T1012.2B params706.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-2.5-1T1012.2B params706.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-1T999.7B params698.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ring-1T999.7B params698.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Inkling952.4B params665.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Intern-S1-Pro917.0B params640.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
Muse-Spark-Open-816B-A42B822.0B params574.1 / 96.0 GBQ4Won’t fitQ8Won’t fit
Hy4-preview780.0B params544.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5753.9B params526.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.1753.9B params526.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3753.3B params526.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.2753.3B params526.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3-BF16753.3B params526.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
K-EXAONE-2.0-750B-A37B749.4B params523.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Exp685.4B params478.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Speciale685.4B params478.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2685.4B params478.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3-0324684.5B params478.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-0528684.5B params478.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.1-Base684.5B params478.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1684.5B params478.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-Zero684.5B params478.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
Step-5-Preview604.3B params422.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.1-flash572.1B params399.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.1-flash-1019572.1B params399.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Chat561.9B params392.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking561.9B params392.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking-2601561.9B params392.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16560.5B params391.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
Beam-1.0-501B-A23B501.3B params350.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Coder-480B-A35B-Instruct480.2B params335.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M1-40k456.1B params319.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M1-80k456.1B params319.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M3427.0B params298.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-424B-A47B-PT423.5B params296.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Intern-S2-397B403.4B params282.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.5-397B-A17B403.4B params282.2 / 96.0 GBQ4Won’t fitQ8Won’t fit
Llama-4-Maverick-17B-128E-Instruct401.6B params281.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Thinking398.6B params278.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Preview398.6B params278.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-TrueBase398.6B params278.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Base398.6B params278.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2.5-Pro396.8B params277.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2-Pro396.8B params277.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
NorthCUA-Pro396.8B params277.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.5358.3B params250.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.7358.3B params250.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.6356.8B params249.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3-Flash321.3B params225.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Motif-3314.8B params220.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.5310.8B params217.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-RL310.8B params217.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-MOPD310.8B params217.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2-Flash309.8B params217.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-Vision-Exp304.6B params213.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-0731304.2B params213.1 / 96.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-300B-A47B-PT300.5B params210.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
Hy3298.8B params209.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Hy3-preview298.8B params209.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-plus292.5B params205.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash290.9B params203.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
MemT-1.0-Flash287.9B params201.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-5.5-Flash-Base286.2B params200.6 / 96.0 GBQ4Won’t fitQ8Won’t fit
Inkling-Small266.0B params186.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-Ultra-253B-v1253.4B params177.7 / 96.0 GBQ4Won’t fitQ8Won’t fit
Solar-Open2-250B250.3B params175.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
Intern-S1240.7B params168.8 / 96.0 GBQ4Won’t fitQ8Won’t fit
K-EXAONE-236B-A23B237.1B params166.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2.5235.7B params165.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct235.7B params165.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2.5-1210235.7B params165.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2-Chat-0628235.7B params165.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct-0724235.7B params165.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-VL-235B-A22B-Instruct235.7B params165.3 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B235.1B params164.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Instruct-2507235.1B params164.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Thinking-2507235.1B params164.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2228.7B params160.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.5228.7B params160.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.7228.7B params160.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.1228.7B params160.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
command-a-plus-05-2026-bf16218.8B params153.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
Step-3.7-Flash201.4B params141.4 / 96.0 GBQ4Won’t fitQ8Won’t fit
Step-3.5-Flash199.4B params140.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Step-3.5-Flash-Base-Midtrain197.8B params138.9 / 96.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-Flash-Next180.0B params126.5 / 96.0 GBQ4Won’t fitQ8Won’t fit
Mixtral-8x22B-Instruct-v0.1140.6B params99.0 / 96.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.0-flash-Sante127.5B params89.9 / 96.0 GBQ4TightQ8Won’t fit
Ling-3.0-flash127.5B params89.9 / 96.0 GBQ4TightQ8Won’t fit
Ling-3.0-flash-Fin127.5B params89.9 / 96.0 GBQ4TightQ8Won’t fit
Ornith-1.5-122B125.1B params88.2 / 96.0 GBQ4TightQ8Won’t fit
Qwen3.5-122B-A10B125.1B params88.2 / 96.0 GBQ4TightQ8Won’t fit
Devstral-2-123B-Instruct-2512125.0B params88.2 / 96.0 GBQ4TightQ8Won’t fit
Ling-3.0-flash-VL124.8B params88.0 / 96.0 GBQ4TightQ8Won’t fit
NVIDIA-Nemotron-3-Super-120B-A12B-BF16123.6B params87.2 / 96.0 GBQ4TightQ8Won’t fit
Mistral-Small-4-119B-2603119.4B params84.2 / 96.0 GBQ4FitsQ8Won’t fit
Youtu-LLM-A7B-Base118.4B params83.5 / 96.0 GBQ4FitsQ8Won’t fit
Laguna-S-2.1117.6B params83.0 / 96.0 GBQ4FitsQ8Won’t fit
gpt-oss-120b116.8B params82.5 / 96.0 GBQ4FitsQ8Won’t fit
c4ai-command-a-03-2025111.1B params78.4 / 96.0 GBQ4FitsQ8Won’t fit
GLM-4.5-Air110.5B params78.0 / 96.0 GBQ4FitsQ8Won’t fit
Llama-4-Scout-17B-16E-Instruct108.6B params76.7 / 96.0 GBQ4FitsQ8Won’t fit
Llama-4-Scout-17B-16E108.6B params76.7 / 96.0 GBQ4FitsQ8Won’t fit
GLM-4.5V107.7B params76.1 / 96.0 GBQ4FitsQ8Won’t fit
GLM-4.6V107.7B params76.1 / 96.0 GBQ4FitsQ8Won’t fit
Ling-2.6-flash107.5B params75.9 / 96.0 GBQ4FitsQ8Won’t fit
sarvam-105b106.0B params74.9 / 96.0 GBQ4FitsQ8Won’t fit
Ling-flash-2.0102.9B params72.7 / 96.0 GBQ4FitsQ8Won’t fit
LLaDA2.1-flash102.9B params72.7 / 96.0 GBQ4FitsQ8Won’t fit
Ring-flash-2.0102.9B params72.7 / 96.0 GBQ4FitsQ8Won’t fit
LLaDA2.2-flash102.9B params72.7 / 96.0 GBQ4FitsQ8Won’t fit
Solar-Open-100B102.7B params72.6 / 96.0 GBQ4FitsQ8Won’t fit
Qwen3-Next-80B-A3B-Instruct81.3B params57.7 / 96.0 GBQ4FitsQ8Won’t fit
Qwen3-Next-80B-A3B-Thinking81.3B params57.7 / 96.0 GBQ4FitsQ8Won’t fit
AliceAI-Foundation-80B-A3B-Base81.3B params57.7 / 96.0 GBQ4FitsQ8Won’t fit
Hunyuan-A13B-Instruct80.4B params57.0 / 96.0 GBQ4FitsQ8Won’t fit
Qwen3-Coder-Next79.7B params56.5 / 96.0 GBQ4FitsQ8Won’t fit
Kolibri-178.1B params55.5 / 96.0 GBQ4FitsQ8Won’t fit
Qwen2.5-VL-72B-Instruct73.4B params52.2 / 96.0 GBQ4FitsQ8Tight
UI-TARS-72B-DPO73.4B params52.2 / 96.0 GBQ4FitsQ8Tight
Qwen2.5-72B-Instruct72.7B params51.7 / 96.0 GBQ4FitsQ8Tight
Apertus-70B-Instruct-250970.6B params50.2 / 96.0 GBQ4FitsQ8Tight
Apertus-70B-250970.6B params50.2 / 96.0 GBQ4FitsQ8Tight
Llama-3.3-70B-Instruct70.6B params50.2 / 96.0 GBQ4FitsQ8Tight
DeepSeek-R1-Distill-Llama-70B70.6B params50.2 / 96.0 GBQ4FitsQ8Tight
Llama-3.1-70B70.6B params50.2 / 96.0 GBQ4FitsQ8Tight
Hermes-2-Theta-Llama-3-70B70.6B params50.2 / 96.0 GBQ4FitsQ8Tight
Meta-Llama-3.1-70B-Instruct70.6B params50.2 / 96.0 GBQ4FitsQ8Tight
LongCat-Flash-Lite69.1B params49.2 / 96.0 GBQ4FitsQ8Fits
AI21-Jamba2-Mini51.6B params37.0 / 96.0 GBQ4FitsQ8Fits
Llama-3_1-Nemotron-51B-Instruct51.5B params36.9 / 96.0 GBQ4FitsQ8Fits
Llama-3_3-Nemotron-Super-49B-v1_549.9B params35.8 / 96.0 GBQ4FitsQ8Fits
Llama-3_3-Nemotron-Super-49B-v149.9B params35.8 / 96.0 GBQ4FitsQ8Fits
Phi-3.5-MoE-instruct41.9B params30.2 / 96.0 GBQ4FitsQ8Fits
GRIN-MoE41.9B params30.2 / 96.0 GBQ4FitsQ8Fits
ALIA-40b40.4B params29.2 / 96.0 GBQ4FitsQ8Fits
IQuest-Coder-V1-40B-Loop-Instruct39.8B params28.7 / 96.0 GBQ4FitsQ8Fits
IQuest-Coder-V1-40B-Instruct39.8B params28.7 / 96.0 GBQ4FitsQ8Fits
Recursive Flint39.8B params28.7 / 96.0 GBQ4FitsQ8Fits
K2-Horizon-MoVA-36B-A4B37.4B params27.1 / 96.0 GBQ4FitsQ8Fits
Seed-OSS-36B-Instruct36.2B params26.2 / 96.0 GBQ4FitsQ8Fits
Seed-OSS-36B-Base-woSyn36.2B params26.2 / 96.0 GBQ4FitsQ8Fits
Intern-S2-Preview36.1B params26.2 / 96.0 GBQ4FitsQ8Fits
Ornith-1.5-35B-A3B36.0B params26.1 / 96.0 GBQ4FitsQ8Fits
Sage-A1.536.0B params26.1 / 96.0 GBQ4FitsQ8Fits
Qwen3.6-35B-A3B36.0B params26.1 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-35B-A3B36.0B params26.1 / 96.0 GBQ4FitsQ8Fits
Apex-35B-V1.036.0B params26.1 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-4.7-35B-A3B35.2B params25.5 / 96.0 GBQ4FitsQ8Fits
MemT-1.0-Nano35.1B params25.5 / 96.0 GBQ4FitsQ8Fits
Agents-A135.1B params25.5 / 96.0 GBQ4FitsQ8Fits
Nex-N2.5-mini35.1B params25.5 / 96.0 GBQ4FitsQ8Fits
Nex-N2-mini35.1B params25.5 / 96.0 GBQ4FitsQ8Fits
Axion-1.0-35B35.1B params25.5 / 96.0 GBQ4FitsQ8Fits
NorthCUA-Mini35.1B params25.5 / 96.0 GBQ4FitsQ8Fits
Clover-35B-A3B-Preview34.7B params25.2 / 96.0 GBQ4FitsQ8Fits
Qwen-AgentWorld-35B-A3B34.7B params25.2 / 96.0 GBQ4FitsQ8Fits
KAT-Coder-V2.5-Dev34.7B params25.2 / 96.0 GBQ4FitsQ8Fits
EXAONE-4.5-33B34.4B params24.9 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-34B-Instruct33.6B params24.5 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-VL-32B-Instruct33.5B params24.3 / 96.0 GBQ4FitsQ8Fits
Laguna-XS.233.4B params24.3 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-32B-Instruct33.4B params24.3 / 96.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Think-32B33.3B params24.2 / 96.0 GBQ4FitsQ8Fits
FlexOlmo-7x7B-1T33.3B params24.2 / 96.0 GBQ4FitsQ8Fits
GLM-Z1-Rumination-32B-041433.1B params24.1 / 96.0 GBQ4FitsQ8Fits
Nanbeige4.3-33B33.1B params24.1 / 96.0 GBQ4FitsQ8Fits
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1633.0B params24.0 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-32B-Instruct32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-32B32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-32B-Instruct32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
QwQ-32B32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
QwQ-32B-Preview32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-32B32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-32B32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-32B32.8B params23.8 / 96.0 GBQ4FitsQ8Fits
GLM-Z1-32B-041432.6B params23.7 / 96.0 GBQ4FitsQ8Fits
GLM-4-32B-041432.6B params23.7 / 96.0 GBQ4FitsQ8Fits
OLMo-2-0325-32B-Instruct32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
OLMo-2-0325-32B32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
Olmo-3-32B-Think32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
Olmo-3-1125-32B32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
Olmo-3.1-32B-Instruct32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
Olmo-3.1-32B-Think32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
Olmo-3.1-32B-Instruct-DPO32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
granite-4.0-h-small32.2B params23.5 / 96.0 GBQ4FitsQ8Fits
sarvam-30b32.2B params23.4 / 96.0 GBQ4FitsQ8Fits
EXAONE-4.0-32B32.0B params23.3 / 96.0 GBQ4FitsQ8Fits
EXAONE-4.0.1-32B32.0B params23.3 / 96.0 GBQ4FitsQ8Fits
EXAONE-3.5-32B-Instruct32.0B params23.3 / 96.0 GBQ4FitsQ8Fits
EXAONE-Deep-32B32.0B params23.3 / 96.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1631.6B params23.0 / 96.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1631.6B params23.0 / 96.0 GBQ4FitsQ8Fits
Nemotron-Cascade-2-30B-A3B31.6B params23.0 / 96.0 GBQ4FitsQ8Fits
Gemma 4 31B31.3B params22.8 / 96.0 GBQ4FitsQ8Fits
gemma-4-31B31.3B params22.8 / 96.0 GBQ4FitsQ8Fits
GLM-4.7-Flash31.2B params22.8 / 96.0 GBQ4FitsQ8Fits
Xing4.0-29B-A4B31.2B params22.8 / 96.0 GBQ4FitsQ8Fits
Xing4.0-29B-Intent31.2B params22.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-30B-A3B-Instruct31.1B params22.7 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-30B-A3B-Thinking31.1B params22.7 / 96.0 GBQ4FitsQ8Fits
UI-Venus-1.5-30B-A3B31.1B params22.7 / 96.0 GBQ4FitsQ8Fits
kanana-2-30b-a3b-instruct-260130.7B params22.4 / 96.0 GBQ4FitsQ8Fits
kanana-2-30b-a3b-instruct30.7B params22.4 / 96.0 GBQ4FitsQ8Fits
kanana-2-30b-a3b-thinking-260130.7B params22.4 / 96.0 GBQ4FitsQ8Fits
Qwen3-30B-A3B30.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Qwen3-Coder-30B-A3B-Instruct30.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Qwen3-30B-A3B-Instruct-250730.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Qwen3-30B-A3B-Thinking-250730.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Tongyi-DeepResearch-30B-A3B30.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Rubicon-Preview30.5B params22.3 / 96.0 GBQ4FitsQ8Fits
North-Mini-Code-1.030.5B params22.3 / 96.0 GBQ4FitsQ8Fits
Hy-Memory-30B-A3B30.1B params22.0 / 96.0 GBQ4FitsQ8Fits
Hy-MT2-30B-A3B30.1B params22.0 / 96.0 GBQ4FitsQ8Fits
granite-switch-4.1-30b30.0B params21.9 / 96.0 GBQ4FitsQ8Fits
Muse-Glimmer-30B29.8B params21.8 / 96.0 GBQ4FitsQ8Fits
ERNIE-4.5-VL-28B-A3B-Thinking29.7B params21.7 / 96.0 GBQ4FitsQ8Fits
ERNIE-4.5-VL-28B-A3B-PT29.4B params21.5 / 96.0 GBQ4FitsQ8Fits
granite-4.2-30b29.3B params21.4 / 96.0 GBQ4FitsQ8Fits
granite-4.1-30b28.9B params21.1 / 96.0 GBQ4FitsQ8Fits
Raon-1 mini preview28.8B params21.1 / 96.0 GBQ4FitsQ8Fits
Qwen3.8-27B27.8B params20.4 / 96.0 GBQ4FitsQ8Fits
Tess-4.5-27B27.8B params20.4 / 96.0 GBQ4FitsQ8Fits
Qwen3.6-27B27.8B params20.4 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-27B27.8B params20.4 / 96.0 GBQ4FitsQ8Fits
gemma-3-27b-it27.4B params20.1 / 96.0 GBQ4FitsQ8Fits
Arete-27B27.4B params20.1 / 96.0 GBQ4FitsQ8Fits
gemma-2-27b-it27.2B params20.0 / 96.0 GBQ4FitsQ8Fits
gemma-2-27b27.2B params20.0 / 96.0 GBQ4FitsQ8Fits
Qwen3.8-27B-Longbook27-v227.1B params19.9 / 96.0 GBQ4FitsQ8Fits
medgemma-27b-text-it27.0B params19.8 / 96.0 GBQ4FitsQ8Fits
Hemmingway-1.526.9B params19.8 / 96.0 GBQ4FitsQ8Fits
Trinity-Mini26.1B params19.2 / 96.0 GBQ4FitsQ8Fits
diffusiongemma-26B-A4B-it25.8B params19.0 / 96.0 GBQ4FitsQ8Fits
Gemma 4 26B A4B25.8B params19.0 / 96.0 GBQ4FitsQ8Fits
gemma-4-26B-A4B25.8B params19.0 / 96.0 GBQ4FitsQ8Fits
Voxtral-Small-24B-250724.3B params17.9 / 96.0 GBQ4FitsQ8Fits
Mistral-Small-3.2-24B-Instruct-250624.0B params17.7 / 96.0 GBQ4FitsQ8Fits
Mistral-Small-3.1-24B-Instruct-250324.0B params17.7 / 96.0 GBQ4FitsQ8Fits
LFM2.5-24B-A2B24.0B params17.7 / 96.0 GBQ4FitsQ8Fits
LFM2-24B-A2B23.8B params17.6 / 96.0 GBQ4FitsQ8Fits
Mistral-Small-24B-Instruct-250123.6B params17.4 / 96.0 GBQ4FitsQ8Fits
EuroLLM-22B-Instruct-251222.6B params16.8 / 96.0 GBQ4FitsQ8Fits
EuroLLM-22B-251222.6B params16.8 / 96.0 GBQ4FitsQ8Fits
solar-pro-preview-instruct22.1B params16.4 / 96.0 GBQ4FitsQ8Fits
ERNIE-4.5-21B-A3B-PT21.9B params16.3 / 96.0 GBQ4FitsQ8Fits
ERNIE-4.5-21B-A3B-Thinking21.8B params16.2 / 96.0 GBQ4FitsQ8Fits
gpt-oss-20b20.9B params15.6 / 96.0 GBQ4FitsQ8Fits
reka-flash-320.9B params15.6 / 96.0 GBQ4FitsQ8Fits
maple-preview20.2B params15.1 / 96.0 GBQ4FitsQ8Fits
internlm2_5-20b-chat19.9B params14.8 / 96.0 GBQ4FitsQ8Fits
Groot-19B-A3B-26102319.4B params14.5 / 96.0 GBQ4FitsQ8Fits
Ling-Coder-lite16.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ring-lite-distill-preview16.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ling-lite16.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ling-lite-1.5-250716.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ling-lite-1.516.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ring-lite16.8B params12.7 / 96.0 GBQ4FitsQ8Fits
Ring-mini-linear-2.016.4B params12.5 / 96.0 GBQ4FitsQ8Fits
Kimi-VL-A3B-Instruct16.4B params12.4 / 96.0 GBQ4FitsQ8Fits
Kimi-VL-A3B-Thinking16.4B params12.4 / 96.0 GBQ4FitsQ8Fits
Kimi-VL-A3B-Thinking-250616.4B params12.4 / 96.0 GBQ4FitsQ8Fits
LLaDA2.1-mini16.3B params12.3 / 96.0 GBQ4FitsQ8Fits
LLaDA2.0-mini16.3B params12.3 / 96.0 GBQ4FitsQ8Fits
Ling-mini-2.016.3B params12.3 / 96.0 GBQ4FitsQ8Fits
Ring-mini-2.016.3B params12.3 / 96.0 GBQ4FitsQ8Fits
LLaDA2.2-mini16.3B params12.3 / 96.0 GBQ4FitsQ8Fits
LLaDA2.0-mini-preview16.3B params12.3 / 96.0 GBQ4FitsQ8Fits
Moonlight-16B-A3B-Instruct16.0B params12.1 / 96.0 GBQ4FitsQ8Fits
Moonlight-16B-A3B16.0B params12.1 / 96.0 GBQ4FitsQ8Fits
Instella-MoE-16B-A3B-Think15.9B params12.1 / 96.0 GBQ4FitsQ8Fits
DeepSeek-Coder-V2-Lite-Instruct15.7B params12.0 / 96.0 GBQ4FitsQ8Fits
kanana-1.5-15.7b-a3b-instruct15.7B params11.9 / 96.0 GBQ4FitsQ8Fits
Phi-4-reasoning-vision-15B15.1B params11.5 / 96.0 GBQ4FitsQ8Fits
Apriel-Nemotron-15b-Thinker15.0B params11.4 / 96.0 GBQ4FitsQ8Fits
Apriel-1.5-15b-Thinker14.9B params11.4 / 96.0 GBQ4FitsQ8Fits
Apriel-1.6-15b-Thinker14.9B params11.4 / 96.0 GBQ4FitsQ8Fits
Carbon-X (DNA)14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-14B-Instruct14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-14B14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-14B-Instruct14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-14B14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-14B-Instruct-1M14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-14B14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
Qwen3-14B14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
JanusCoder-14B14.8B params11.3 / 96.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Think-14B14.7B params11.3 / 96.0 GBQ4FitsQ8Fits
phi-414.7B params11.2 / 96.0 GBQ4FitsQ8Fits
Phi-4-reasoning14.7B params11.2 / 96.0 GBQ4FitsQ8Fits
Phi-4-reasoning-plus14.7B params11.2 / 96.0 GBQ4FitsQ8Fits
OLMo-2-1124-13B-Instruct13.7B params10.6 / 96.0 GBQ4FitsQ8Fits
Mistral-Nemo-Instruct-240712.2B params9.5 / 96.0 GBQ4FitsQ8Fits
gemma-3-12b-it12.2B params9.5 / 96.0 GBQ4FitsQ8Fits
gemma-3-12b-pt12.2B params9.5 / 96.0 GBQ4FitsQ8Fits
Mellum2.1-12B-A2.5B-Thinking12.1B params9.5 / 96.0 GBQ4FitsQ8Fits
Mellum2-12B-A2.5B-Thinking12.1B params9.5 / 96.0 GBQ4FitsQ8Fits
Mellum2-12B-A2.5B-Instruct12.1B params9.5 / 96.0 GBQ4FitsQ8Fits
Mellum2-12B-A2.5B-Thinking-SFT12.1B params9.5 / 96.0 GBQ4FitsQ8Fits
Mellum2.1-12B-A2.5B-Thinking-MTP12.1B params9.5 / 96.0 GBQ4FitsQ8Fits
Gemma 4 12B12.0B params9.3 / 96.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Omni-8B10.7B params8.5 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-11B-Vision-Instruct10.7B params8.4 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-11B-Vision10.6B params8.4 / 96.0 GBQ4FitsQ8Fits
Falcon3-10B-Instruct10.3B params8.2 / 96.0 GBQ4FitsQ8Fits
GLM-4.1V-9B-Thinking10.3B params8.2 / 96.0 GBQ4FitsQ8Fits
GLM-4.6V-Flash10.3B params8.2 / 96.0 GBQ4FitsQ8Fits
GLM-4.1V-9B-Base10.3B params8.2 / 96.0 GBQ4FitsQ8Fits
Glyph10.3B params8.2 / 96.0 GBQ4FitsQ8Fits
INTELLECT-1-Instruct10.2B params8.1 / 96.0 GBQ4FitsQ8Fits
INTELLECT-110.2B params8.1 / 96.0 GBQ4FitsQ8Fits
ZAYA1-VL-8B9.7B params7.8 / 96.0 GBQ4FitsQ8Fits
SenseNova-Loop-9B-Preview9.7B params7.7 / 96.0 GBQ4FitsQ8Fits
Ornith-1.5-9B9.7B params7.7 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-9B9.7B params7.7 / 96.0 GBQ4FitsQ8Fits
MiniCPM-SALA9.5B params7.6 / 96.0 GBQ4FitsQ8Fits
NorthCUA-Nano9.4B params7.6 / 96.0 GBQ4FitsQ8Fits
GLM-4-9B-04149.4B params7.6 / 96.0 GBQ4FitsQ8Fits
GLM-Z1-9B-04149.4B params7.6 / 96.0 GBQ4FitsQ8Fits
codegeex4-all-9b9.4B params7.6 / 96.0 GBQ4FitsQ8Fits
glm-4-9b-chat-hf9.4B params7.6 / 96.0 GBQ4FitsQ8Fits
moondream3-preview9.3B params7.5 / 96.0 GBQ4FitsQ8Fits
gemma-2-9b-it9.2B params7.4 / 96.0 GBQ4FitsQ8Fits
gemma-2-9b9.2B params7.4 / 96.0 GBQ4FitsQ8Fits
Gemma-SEA-LION-v3-9B-IT9.2B params7.4 / 96.0 GBQ4FitsQ8Fits
EuroLLM-9B-Instruct-25129.2B params7.4 / 96.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-Nano-9B-v28.9B params7.2 / 96.0 GBQ4FitsQ8Fits
ZAYA1-base8.8B params7.2 / 96.0 GBQ4FitsQ8Fits
Yi-Coder-9B-Chat8.8B params7.2 / 96.0 GBQ4FitsQ8Fits
Yi-Coder-9B8.8B params7.2 / 96.0 GBQ4FitsQ8Fits
internlm3-8b-instruct8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
granite-4.1-8b8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
granite-4.2-8b8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
granite-switch-4.1-8b8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-8B-Instruct8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-8B-Thinking8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
UI-Venus-1.5-8B8.8B params7.1 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Nemotron-Nano-VL-8B-V18.7B params7.1 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-4_58.7B params7.1 / 96.0 GBQ4FitsQ8Fits
aya-vision-8b8.6B params7.0 / 96.0 GBQ4FitsQ8Fits
Intern-S1-mini8.5B params7.0 / 96.0 GBQ4FitsQ8Fits
Nemotron-Labs-Diffusion-8B-Base8.5B params6.9 / 96.0 GBQ4FitsQ8Fits
LFM2.5-8B-A1B8.5B params6.9 / 96.0 GBQ4FitsQ8Fits
Mistral-NeMo-Minitron-8B-Base8.4B params6.9 / 96.0 GBQ4FitsQ8Fits
LFM2-8B-A1B8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
rnj-1-instruct8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
MiMo-VL-7B-RL-25088.3B params6.8 / 96.0 GBQ4FitsQ8Fits
MiMo-VL-7B-RL8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
MiMo-Embodied-7B8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
MiMo-VL-7B-SFT8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-VL-7B-Instruct8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
UI-TARS-1.5-7B8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Fara-7B8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
UI-Venus-Ground-7B8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Qwen2-VL-7B-Instruct8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
UI-TARS-7B-DPO8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
UI-TARS-7B-SFT8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Seed-Coder-8B-Instruct8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Seed-Coder-8B-Reasoning8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Stable-DiffCoder-8B-Instruct8.3B params6.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-8B8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-0528-Qwen3-8B8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
Qwen3-8B-Base8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
WeDLM-8B-Instruct8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
PaCoRe-8B8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
MiniCPM4-8B8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
MiniCPM4.1-8B8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
AgentCPM-Report8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
MiniCPM4-MCP8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
granite-3.3-8b-instruct8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
granite-3.1-8b-instruct8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
granite-3.2-8b-instruct8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
granite-3.0-8b-instruct8.2B params6.7 / 96.0 GBQ4FitsQ8Fits
Apertus-8B-Instruct-25098.1B params6.6 / 96.0 GBQ4FitsQ8Fits
Apertus-8B-25098.1B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3-8B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3-8B-DPO8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3.1-8B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
kanana-1.5-8b-instruct-25058.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-8B-Instruct8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-8B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Llama-8B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Meta-Llama-3.1-8B-Instruct8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
L3-8B-Stheno-v3.28.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Nemotron-Nano-8B-v18.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Meta-Llama-3.1-8B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
LongWriter-llama3.1-8b8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Hy-MT2-7B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Nemotron 3.5 Bolt E4B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Gemma 4 E4B8.0B params6.6 / 96.0 GBQ4FitsQ8Fits
Ling-3.0-tiny7.9B params6.5 / 96.0 GBQ4FitsQ8Fits
MiMo-7B-RL7.8B params6.5 / 96.0 GBQ4FitsQ8Fits
MiMo-7B-Base7.8B params6.5 / 96.0 GBQ4FitsQ8Fits
MiMo-7B-RL-05307.8B params6.5 / 96.0 GBQ4FitsQ8Fits
MiMo-7B-SFT7.8B params6.5 / 96.0 GBQ4FitsQ8Fits
EXAONE-3.5-7.8B-Instruct7.8B params6.5 / 96.0 GBQ4FitsQ8Fits
EXAONE-Deep-7.8B7.8B params6.5 / 96.0 GBQ4FitsQ8Fits
salamandra-7b-instruct7.8B params6.4 / 96.0 GBQ4FitsQ8Fits
salamandra-7b7.8B params6.4 / 96.0 GBQ4FitsQ8Fits
salamandra-7b-instruct-26067.8B params6.4 / 96.0 GBQ4FitsQ8Fits
FastVLM-7B7.8B params6.4 / 96.0 GBQ4FitsQ8Fits
Molmo2-O-7B7.8B params6.4 / 96.0 GBQ4FitsQ8Fits
internlm2_5-7b-chat7.7B params6.4 / 96.0 GBQ4FitsQ8Fits
internlm2_5-7b-chat-1m7.7B params6.4 / 96.0 GBQ4FitsQ8Fits
Phi-mini-MoE-instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-7B-Instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-7B-Instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2-7B-Instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-7B-Instruct-1M7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Math-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Math-7B-Instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Marco-o17.6B params6.3 / 96.0 GBQ4FitsQ8Fits
SeaLLMs-v3-7B-Chat7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Arcee-Spark7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-7B-Instruct7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
Falcon-H1R-7B7.6B params6.3 / 96.0 GBQ4FitsQ8Fits
starvector-8b-im2svg7.5B params6.2 / 96.0 GBQ4FitsQ8Fits
Falcon3-7B-Instruct7.5B params6.2 / 96.0 GBQ4FitsQ8Fits
Olmo-Hybrid-7B7.4B params6.2 / 96.0 GBQ4FitsQ8Fits
LLaDA-MoE-7B-A1B-Base7.4B params6.1 / 96.0 GBQ4FitsQ8Fits
Zamba2-7B-Instruct7.4B params6.1 / 96.0 GBQ4FitsQ8Fits
OLMo-2-1124-7B-Instruct7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-1025-7B7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct-SFT7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think-SFT7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct-DPO7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-RL-Zero-Math7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3.1-7B-RL-Zero-Math7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think-DPO7.3B params6.1 / 96.0 GBQ4FitsQ8Fits
Mistral-7B-Instruct-v0.37.2B params6.1 / 96.0 GBQ4FitsQ8Fits
granite-4.0-h-tiny6.9B params5.8 / 96.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-0924-Instruct6.9B params5.8 / 96.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-09246.9B params5.8 / 96.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-0125-Instruct6.9B params5.8 / 96.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-01256.9B params5.8 / 96.0 GBQ4FitsQ8Fits
OLMo-7B-0724-hf6.9B params5.8 / 96.0 GBQ4FitsQ8Fits
granite-4.0-tiny-preview6.7B params5.7 / 96.0 GBQ4FitsQ8Fits
YuE-s1-7B-anneal-en-cot6.2B params5.3 / 96.0 GBQ4FitsQ8Fits
Trinity-Nano-Preview6.1B params5.3 / 96.0 GBQ4FitsQ8Fits
Trinity-Nano-Base6.1B params5.3 / 96.0 GBQ4FitsQ8Fits
gemma-3n-E2B-it5.4B params4.8 / 96.0 GBQ4FitsQ8Fits
Youtu-VL-4B-Instruct5.3B params4.7 / 96.0 GBQ4FitsQ8Fits
Gemma 4 E2B5.1B params4.6 / 96.0 GBQ4FitsQ8Fits
Molmo2-ER4.9B params4.4 / 96.0 GBQ4FitsQ8Fits
Apriel-5B-Instruct4.8B params4.4 / 96.0 GBQ4FitsQ8Fits
WeVisOne-VL-4B4.8B params4.4 / 96.0 GBQ4FitsQ8Fits
Mage-VL4.7B params4.3 / 96.0 GBQ4FitsQ8Fits
Mage-VL-Cowork4.7B params4.3 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-4B4.7B params4.2 / 96.0 GBQ4FitsQ8Fits
S5-nano4.7B params4.2 / 96.0 GBQ4FitsQ8Fits
AFM-4.5B4.6B params4.2 / 96.0 GBQ4FitsQ8Fits
Llama-3.1-Minitron-4B-Width-Base4.5B params4.1 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-4B-Instruct4.4B params4.1 / 96.0 GBQ4FitsQ8Fits
CapRL-Qwen3VL-4B4.4B params4.1 / 96.0 GBQ4FitsQ8Fits
glm-edge-4b-chat4.3B params4.0 / 96.0 GBQ4FitsQ8Fits
gemma-3-4b-it4.3B params4.0 / 96.0 GBQ4FitsQ8Fits
medgemma-4b-it4.3B params4.0 / 96.0 GBQ4FitsQ8Fits
medgemma-1.5-4b-it4.3B params4.0 / 96.0 GBQ4FitsQ8Fits
gemma-3-4b-pt4.3B params4.0 / 96.0 GBQ4FitsQ8Fits
Nanbeige4.2-3B4.2B params3.9 / 96.0 GBQ4FitsQ8Fits
Phi-3.5-vision-instruct4.1B params3.9 / 96.0 GBQ4FitsQ8Fits
Spark-X2.6-4B4.1B params3.9 / 96.0 GBQ4FitsQ8Fits
Spark-X2.5-4B4.1B params3.9 / 96.0 GBQ4FitsQ8Fits
POINTS-Reader4.1B params3.9 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-44.1B params3.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-4B4.0B params3.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-4B-Instruct-25074.0B params3.8 / 96.0 GBQ4FitsQ8Fits
Qwen3-4B-Thinking-25074.0B params3.8 / 96.0 GBQ4FitsQ8Fits
Mellum-4b-base4.0B params3.8 / 96.0 GBQ4FitsQ8Fits
granite-4.0-3b-vision4.0B params3.8 / 96.0 GBQ4FitsQ8Fits
granite-vision-4.1-4b4.0B params3.8 / 96.0 GBQ4FitsQ8Fits
Nemotron Labs Dory 1.0 4B4.0B params3.8 / 96.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-3-Nano-4B-BF164.0B params3.8 / 96.0 GBQ4FitsQ8Fits
Nanbeige4.1-3B3.9B params3.7 / 96.0 GBQ4FitsQ8Fits
Nanbeige4-3B-Thinking-25113.9B params3.7 / 96.0 GBQ4FitsQ8Fits
Phi-4-mini-flash-reasoning3.9B params3.7 / 96.0 GBQ4FitsQ8Fits
Phi-4-mini-instruct3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
Phi-4-mini-reasoning3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
Nemotron-Labs-Diffusion-3B3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
Phi-3.5-mini-instruct3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
MediPhi-Instruct3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
MediPhi3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
MediPhi-Clinical3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
MediPhi-PubMed3.8B params3.7 / 96.0 GBQ4FitsQ8Fits
HY-Embodied-0.53.8B params3.6 / 96.0 GBQ4FitsQ8Fits
Phi-tiny-MoE-instruct3.8B params3.6 / 96.0 GBQ4FitsQ8Fits
CapRL-3B3.8B params3.6 / 96.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Vision-Instruct-3B3.7B params3.6 / 96.0 GBQ4FitsQ8Fits
kanana-1.5-v-3b-instruct3.7B params3.6 / 96.0 GBQ4FitsQ8Fits
granite-4.2-3b3.7B params3.6 / 96.0 GBQ4FitsQ8Fits
kanana-2-3b-instruct3.5B params3.4 / 96.0 GBQ4FitsQ8Fits
granite-4.1-3b3.4B params3.4 / 96.0 GBQ4FitsQ8Fits
granite-4.0-micro3.4B params3.4 / 96.0 GBQ4FitsQ8Fits
granite-3.1-3b-a800m-instruct3.3B params3.3 / 96.0 GBQ4FitsQ8Fits
Falcon3-3B-Instruct3.2B params3.3 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-3B-Instruct3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-3B3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
Hermes-3-Llama-3.2-3B3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
DeepHermes-3-Llama-3-3B-Preview3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
AI21-Jamba-Reasoning-3B3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
granite-4.0-h-micro3.2B params3.2 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-3B-Instruct3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
d1-3B3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
LFM2.5-VL-3B3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-3B-Instruct3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-3B-Instruct3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-3B3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-3B3.1B params3.2 / 96.0 GBQ4FitsQ8Fits
SmolLM3-3B3.1B params3.1 / 96.0 GBQ4FitsQ8Fits
AI21-Jamba2-3B3.0B params3.1 / 96.0 GBQ4FitsQ8Fits
granite-swash-3b-a600m3.0B params3.1 / 96.0 GBQ4FitsQ8Fits
granite-vision-3.2-2b3.0B params3.1 / 96.0 GBQ4FitsQ8Fits
granite-vision-3.1-2b-preview3.0B params3.1 / 96.0 GBQ4FitsQ8Fits
granite-vision-3.3-2b-chart2csv-preview3.0B params3.1 / 96.0 GBQ4FitsQ8Fits
LFM2.5-2.6B2.7B params2.9 / 96.0 GBQ4FitsQ8Fits
Zamba2-2.7B2.7B params2.9 / 96.0 GBQ4FitsQ8Fits
Zamba2-2.7B-instruct2.7B params2.9 / 96.0 GBQ4FitsQ8Fits
granite-3.0-2b-instruct2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
gemma-2-2b-it2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
gemma-2-2b2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
EuroMoE-2.6B-A0.6B-Instruct-Preview2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
Motif-2.6B2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
LFM2-2.6B2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
LFM2-2.6B-Exp2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
LFM2-2.6B-Transcript2.6B params2.8 / 96.0 GBQ4FitsQ8Fits
granite-3.3-2b-instruct2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
granite-3.1-2b-instruct2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
granite-3.2-2b-instruct2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
MiniCPM5-2B2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
MiniCPM5-2B-SFT2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
MiniCPM5-2B-Midtrain2.5B params2.8 / 96.0 GBQ4FitsQ8Fits
sarvam-1-v0.52.5B params2.7 / 96.0 GBQ4FitsQ8Fits
North-Micro-Vision-Instruct2.5B params2.7 / 96.0 GBQ4FitsQ8Fits
UI-TARS-2B-SFT2.4B params2.7 / 96.0 GBQ4FitsQ8Fits
WeVisOne-VL-2B2.4B params2.7 / 96.0 GBQ4FitsQ8Fits
UI-Venus-1.5-2B2.4B params2.7 / 96.0 GBQ4FitsQ8Fits
bitnet-b1.58-2B-4T-bf162.4B params2.7 / 96.0 GBQ4FitsQ8Fits
EXAONE-3.5-2.4B-Instruct2.4B params2.7 / 96.0 GBQ4FitsQ8Fits
EXAONE-Deep-2.4B2.4B params2.7 / 96.0 GBQ4FitsQ8Fits
kanana-1.5-2.1b-instruct-25052.3B params2.6 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-2B2.3B params2.6 / 96.0 GBQ4FitsQ8Fits
salamandra-2b2.3B params2.6 / 96.0 GBQ4FitsQ8Fits
salamandra-2b-instruct2.3B params2.6 / 96.0 GBQ4FitsQ8Fits
SmolVLM2-2.2B-Instruct2.2B params2.6 / 96.0 GBQ4FitsQ8Fits
SmolVLM-Base2.2B params2.6 / 96.0 GBQ4FitsQ8Fits
Qwen2-VL-2B-Instruct2.2B params2.5 / 96.0 GBQ4FitsQ8Fits
LFM3-Omni-2B2.2B params2.5 / 96.0 GBQ4FitsQ8Fits
granite-swash-2b2.1B params2.5 / 96.0 GBQ4FitsQ8Fits
Qwen3-VL-2B-Instruct2.1B params2.5 / 96.0 GBQ4FitsQ8Fits
kanana-nano-2.1b-instruct2.1B params2.5 / 96.0 GBQ4FitsQ8Fits
Hy-MT2-1.8B2.0B params2.4 / 96.0 GBQ4FitsQ8Fits
Qwen3-1.7B2.0B params2.4 / 96.0 GBQ4FitsQ8Fits
Youtu-LLM-2B2.0B params2.4 / 96.0 GBQ4FitsQ8Fits
LFM3-VL-2B1.9B params2.3 / 96.0 GBQ4FitsQ8Fits
d1-2B1.9B params2.3 / 96.0 GBQ4FitsQ8Fits
FastVLM-1.5B1.9B params2.3 / 96.0 GBQ4FitsQ8Fits
internlm2_5-1_8b-chat1.9B params2.3 / 96.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-1.5B1.8B params2.2 / 96.0 GBQ4FitsQ8Fits
LFM3-1.7B1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
Qwen3-1.7B-Base1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
SmolLM2-1.7B1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
SmolLM-1.7B1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
Spark-X2.6-1.7B1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
Falcon3-1B-Instruct1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
EuroLLM-1.7B-Instruct1.7B params2.2 / 96.0 GBQ4FitsQ8Fits
granite-4.0-1b1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
LFM2.5-VL-1.6B1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
glm-edge-1.5b-chat1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
LFM2-VL-1.6B1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-1.5B-Deep-Instruct1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-1.5B-Instruct1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-4.7-1B1.6B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-1.5B-Instruct1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-1.5B1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-1.5B-Instruct1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-1.5B1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Math-1.5B1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2-1.5B-Instruct1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Math-1.5B-Instruct1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
ReaderLM-v21.5B params2.1 / 96.0 GBQ4FitsQ8Fits
reader-lm-1.5b1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
Hymba-1.5B-Instruct1.5B params2.1 / 96.0 GBQ4FitsQ8Fits
OLMo-2-0425-1B1.5B params2.0 / 96.0 GBQ4FitsQ8Fits
OLMo-2-0425-1B-Instruct1.5B params2.0 / 96.0 GBQ4FitsQ8Fits
Yi-Coder-1.5B-Chat1.5B params2.0 / 96.0 GBQ4FitsQ8Fits
Bolmo-1B1.5B params2.0 / 96.0 GBQ4FitsQ8Fits
granite-4.0-h-1b1.5B params2.0 / 96.0 GBQ4FitsQ8Fits
granite-3.1-1b-a400m-instruct1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
granite-3.0-1b-a400m-instruct1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-4.61.3B params1.9 / 96.0 GBQ4FitsQ8Fits
MiniCPM-V-4.6-Thinking1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
kanana-2-1.3b-instruct1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
OLMo-1B-0724-hf1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
EXAONE-4.0-1.2B1.3B params1.9 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-1B-Instruct1.2B params1.9 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-1B1.2B params1.9 / 96.0 GBQ4FitsQ8Fits
Llama-3.2-1B1.2B params1.9 / 96.0 GBQ4FitsQ8Fits
Zamba2-1.2B-instruct1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
HRM-Text-1B1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
AMD-OLMo-1B1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-Instruct1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
LFM2-1.2B1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-Thinking1.2B params1.8 / 96.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-JP-2026061.2B params1.8 / 96.0 GBQ4FitsQ8Fits
MiniCPM5-1B1.1B params1.8 / 96.0 GBQ4FitsQ8Fits
MiniCPM5-1B-SFT1.1B params1.8 / 96.0 GBQ4FitsQ8Fits
gemma-3-1b-it1.0B params1.7 / 96.0 GBQ4FitsQ8Fits
gemma-3-1b-pt1.0B params1.7 / 96.0 GBQ4FitsQ8Fits
Qwen3.5-0.8B0.9B params1.6 / 96.0 GBQ4FitsQ8Fits
Falcon-E-3B-Instruct0.9B params1.6 / 96.0 GBQ4FitsQ8Fits
bitnet-b1.58-2B-4T0.8B params1.6 / 96.0 GBQ4FitsQ8Fits
FastVLM-0.5B0.8B params1.5 / 96.0 GBQ4FitsQ8Fits
Qwen3-0.6B0.8B params1.5 / 96.0 GBQ4FitsQ8Fits
LFM2-700M0.7B params1.5 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-Tiny-R-0.6B0.6B params1.4 / 96.0 GBQ4FitsQ8Fits
Qwen3-0.6B-Base0.6B params1.4 / 96.0 GBQ4FitsQ8Fits
d1-omni-600M0.6B params1.4 / 96.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Text-Instruct-0.5B0.6B params1.4 / 96.0 GBQ4FitsQ8Fits
Falcon-H1-0.5B-Instruct0.5B params1.4 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-0.5B-Instruct0.5B params1.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-0.5B0.5B params1.3 / 96.0 GBQ4FitsQ8Fits
Qwen2-0.5B-Instruct0.5B params1.3 / 96.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-0.5B-Instruct0.5B params1.3 / 96.0 GBQ4FitsQ8Fits
LFM2-VL-450M0.5B params1.3 / 96.0 GBQ4FitsQ8Fits
LFM2.5-VL-450M0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
Apertus-v1.1-0.5B0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
MiniCPM4-0.5B0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
circuit-sparsity0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
SmolLM2-360M0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
SmolLM-360M0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
ERNIE-4.5-0.3B-PT0.4B params1.3 / 96.0 GBQ4FitsQ8Fits
LFM2.5-350M0.4B params1.2 / 96.0 GBQ4FitsQ8Fits
LFM2-350M0.4B params1.2 / 96.0 GBQ4FitsQ8Fits
granite-4.0-350m0.4B params1.2 / 96.0 GBQ4FitsQ8Fits
granite-4.0-h-350m0.3B params1.2 / 96.0 GBQ4FitsQ8Fits

Quick answers

What is the biggest LLM that runs on NVIDIA RTX Pro 6000 Blackwell 96GB?
Mistral-Small-4-119B-2603 (119.4B parameters, about 84.2 GB at Q4) is the largest model we track that runs fully in its 96 GB of usable memory.
How many local LLMs can NVIDIA RTX Pro 6000 Blackwell 96GB run?
545 of the 656 open models we track run fully at Q4 on it.
How much memory can NVIDIA RTX Pro 6000 Blackwell 96GB use for AI models?
It has 96 GB of VRAM, and we count 96 GB as usable for model weights and cache.

New models, and what they need to run

One short email a week: fresh open models with their VRAM numbers, plus hardware worth watching.

Or subscribe here