What is the biggest LLM that runs on NVIDIA RTX A5000?

ERNIE-4.5-VL-28B-A3B-PT (29.4B parameters, about 21.5 GB at Q4) is the largest model we track that runs fully in its 24 GB of usable memory.

How many local LLMs can NVIDIA RTX A5000 run?

423 of the 656 open models we track run fully at Q4 on it.

How much memory can NVIDIA RTX A5000 use for AI models?

It has 24 GB of VRAM, and we count 24 GB as usable for model weights and cache.

What LLMs can I run on NVIDIA RTX A5000?

You can run 423 of the 656 open models we track fully on the NVIDIA RTX A5000 at Q4_K_M, including ERNIE-4.5-VL-28B-A3B-PT (29.4B parameters, about 21.5 GB). It has 24 GB of usable VRAM. These are estimates, not measured benchmarks.

Specs and list price: NVIDIA RTX A5000 hardware page.

Memory: 24 GB VRAM
Memory bandwidth: 768 GB/s
Usable for models: 24 GB
Runs fully at Q4: 423 / 656

Spec source · Best GPU under $3,500 · Best GPU for local LLMs · Multi-GPU · Best used GPU for local LLMs · NVIDIA vs AMD vs Mac · Tokens per second · Best LLMs for 24 GB

$2250 list (launch MSRP; discontinued), source, 2026-10-28
Street prices may be higher than list.
Discontinued; buy used. Price shown is the original launch MSRP.

Check price on Amazon Check price on eBay

We may earn a commission when you buy through these links, at no extra cost to you.

Every tracked model on this machine

Sorted largest first. The tick marks 90% of usable memory.FitsTightWon’t fit

Model	Q4 need vs. 24 GB	Verdict
Kimi-K3.1-Pro2780.1B params	1939.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K32779.9B params	1939.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-2.4T-A95B2446.2B params	1706.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-2.51779.8B params	1241.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-2.01775.6B params	1238.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro-08131650.5B params	1151.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro1598.8B params	1115.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.51026.9B params	716.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.61026.9B params	716.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.7-Code1026.9B params	716.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Thinking1026.4B params	716.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Instruct-09051026.4B params	716.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Instruct1026.4B params	716.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-RL1024.2B params	715.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-MOPD1024.2B params	715.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.5-Pro1023.2B params	714.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ring-2.5-1T1012.2B params	706.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-2.5-1T1012.2B params	706.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-1T999.7B params	698.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ring-1T999.7B params	698.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Inkling952.4B params	665.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S1-Pro917.0B params	640.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Muse-Spark-Open-816B-A42B822.0B params	574.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Hy4-preview780.0B params	544.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5753.9B params	526.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.1753.9B params	526.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3753.3B params	526.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.2753.3B params	526.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3-BF16753.3B params	526.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
K-EXAONE-2.0-750B-A37B749.4B params	523.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Exp685.4B params	478.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Speciale685.4B params	478.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2685.4B params	478.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3-0324684.5B params	478.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-0528684.5B params	478.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.1-Base684.5B params	478.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1684.5B params	478.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Zero684.5B params	478.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Step-5-Preview604.3B params	422.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.1-flash572.1B params	399.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.1-flash-1019572.1B params	399.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Chat561.9B params	392.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking561.9B params	392.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking-2601561.9B params	392.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16560.5B params	391.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Beam-1.0-501B-A23B501.3B params	350.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Coder-480B-A35B-Instruct480.2B params	335.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M1-40k456.1B params	319.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M1-80k456.1B params	319.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M3427.0B params	298.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-424B-A47B-PT423.5B params	296.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S2-397B403.4B params	282.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-397B-A17B403.4B params	282.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Maverick-17B-128E-Instruct401.6B params	281.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Thinking398.6B params	278.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Preview398.6B params	278.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-TrueBase398.6B params	278.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Base398.6B params	278.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2.5-Pro396.8B params	277.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2-Pro396.8B params	277.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
NorthCUA-Pro396.8B params	277.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5358.3B params	250.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.7358.3B params	250.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.6356.8B params	249.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3-Flash321.3B params	225.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Motif-3314.8B params	220.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.5310.8B params	217.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-RL310.8B params	217.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-MOPD310.8B params	217.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2-Flash309.8B params	217.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-Vision-Exp304.6B params	213.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-0731304.2B params	213.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-300B-A47B-PT300.5B params	210.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Hy3298.8B params	209.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Hy3-preview298.8B params	209.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-plus292.5B params	205.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash290.9B params	203.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MemT-1.0-Flash287.9B params	201.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-5.5-Flash-Base286.2B params	200.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Inkling-Small266.0B params	186.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-Ultra-253B-v1253.4B params	177.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Solar-Open2-250B250.3B params	175.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S1240.7B params	168.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
K-EXAONE-236B-A23B237.1B params	166.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2.5235.7B params	165.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct235.7B params	165.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2.5-1210235.7B params	165.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2-Chat-0628235.7B params	165.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct-0724235.7B params	165.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-235B-A22B-Instruct235.7B params	165.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B235.1B params	164.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Instruct-2507235.1B params	164.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Thinking-2507235.1B params	164.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2228.7B params	160.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.5228.7B params	160.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.7228.7B params	160.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.1228.7B params	160.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
command-a-plus-05-2026-bf16218.8B params	153.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.7-Flash201.4B params	141.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.5-Flash199.4B params	140.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.5-Flash-Base-Midtrain197.8B params	138.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-Flash-Next180.0B params	126.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Mixtral-8x22B-Instruct-v0.1140.6B params	99.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-Sante127.5B params	89.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash127.5B params	89.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-Fin127.5B params	89.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ornith-1.5-122B125.1B params	88.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-122B-A10B125.1B params	88.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Devstral-2-123B-Instruct-2512125.0B params	88.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-VL124.8B params	88.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Super-120B-A12B-BF16123.6B params	87.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Small-4-119B-2603119.4B params	84.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Youtu-LLM-A7B-Base118.4B params	83.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Laguna-S-2.1117.6B params	83.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
gpt-oss-120b116.8B params	82.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
c4ai-command-a-03-2025111.1B params	78.4 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5-Air110.5B params	78.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E-Instruct108.6B params	76.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E108.6B params	76.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5V107.7B params	76.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.6V107.7B params	76.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-2.6-flash107.5B params	75.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
sarvam-105b106.0B params	74.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ling-flash-2.0102.9B params	72.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.1-flash102.9B params	72.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ring-flash-2.0102.9B params	72.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.2-flash102.9B params	72.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Solar-Open-100B102.7B params	72.6 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Instruct81.3B params	57.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Thinking81.3B params	57.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
AliceAI-Foundation-80B-A3B-Base81.3B params	57.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Hunyuan-A13B-Instruct80.4B params	57.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Coder-Next79.7B params	56.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Kolibri-178.1B params	55.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-VL-72B-Instruct73.4B params	52.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
UI-TARS-72B-DPO73.4B params	52.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-72B-Instruct72.7B params	51.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Apertus-70B-Instruct-250970.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Apertus-70B-250970.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.3-70B-Instruct70.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Llama-70B70.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.1-70B70.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Hermes-2-Theta-Llama-3-70B70.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Meta-Llama-3.1-70B-Instruct70.6B params	50.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Lite69.1B params	49.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
AI21-Jamba2-Mini51.6B params	37.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-51B-Instruct51.5B params	36.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v1_549.9B params	35.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v149.9B params	35.8 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Phi-3.5-MoE-instruct41.9B params	30.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GRIN-MoE41.9B params	30.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
ALIA-40b40.4B params	29.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Loop-Instruct39.8B params	28.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Instruct39.8B params	28.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Recursive Flint39.8B params	28.7 / 24.0 GB	Q4Won’t fitQ8Won’t fit
K2-Horizon-MoVA-36B-A4B37.4B params	27.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Seed-OSS-36B-Instruct36.2B params	26.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Seed-OSS-36B-Base-woSyn36.2B params	26.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S2-Preview36.1B params	26.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Ornith-1.5-35B-A3B36.0B params	26.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Sage-A1.536.0B params	26.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.6-35B-A3B36.0B params	26.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-35B-A3B36.0B params	26.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Apex-35B-V1.036.0B params	26.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MiniCPM-V-4.7-35B-A3B35.2B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
MemT-1.0-Nano35.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Agents-A135.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2.5-mini35.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2-mini35.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Axion-1.0-35B35.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
NorthCUA-Mini35.1B params	25.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Clover-35B-A3B-Preview34.7B params	25.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen-AgentWorld-35B-A3B34.7B params	25.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
KAT-Coder-V2.5-Dev34.7B params	25.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-4.5-33B34.4B params	24.9 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Falcon-H1-34B-Instruct33.6B params	24.5 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-VL-32B-Instruct33.5B params	24.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Laguna-XS.233.4B params	24.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-32B-Instruct33.4B params	24.3 / 24.0 GB	Q4Won’t fitQ8Won’t fit
HyperCLOVAX-SEED-Think-32B33.3B params	24.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
FlexOlmo-7x7B-1T33.3B params	24.2 / 24.0 GB	Q4Won’t fitQ8Won’t fit
GLM-Z1-Rumination-32B-041433.1B params	24.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nanbeige4.3-33B33.1B params	24.1 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1633.0B params	24.0 / 24.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-Coder-32B-Instruct32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
DeepSeek-R1-Distill-Qwen-32B32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
Qwen2.5-32B-Instruct32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
QwQ-32B32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
QwQ-32B-Preview32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
Qwen2.5-32B32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
Qwen2.5-Coder-32B32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-32B32.8B params	23.8 / 24.0 GB	Q4TightQ8Won’t fit
GLM-Z1-32B-041432.6B params	23.7 / 24.0 GB	Q4TightQ8Won’t fit
GLM-4-32B-041432.6B params	23.7 / 24.0 GB	Q4TightQ8Won’t fit
OLMo-2-0325-32B-Instruct32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
OLMo-2-0325-32B32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
Olmo-3-32B-Think32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
Olmo-3-1125-32B32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
Olmo-3.1-32B-Instruct32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
Olmo-3.1-32B-Think32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
Olmo-3.1-32B-Instruct-DPO32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
granite-4.0-h-small32.2B params	23.5 / 24.0 GB	Q4TightQ8Won’t fit
sarvam-30b32.2B params	23.4 / 24.0 GB	Q4TightQ8Won’t fit
EXAONE-4.0-32B32.0B params	23.3 / 24.0 GB	Q4TightQ8Won’t fit
EXAONE-4.0.1-32B32.0B params	23.3 / 24.0 GB	Q4TightQ8Won’t fit
EXAONE-3.5-32B-Instruct32.0B params	23.3 / 24.0 GB	Q4TightQ8Won’t fit
EXAONE-Deep-32B32.0B params	23.3 / 24.0 GB	Q4TightQ8Won’t fit
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1631.6B params	23.0 / 24.0 GB	Q4TightQ8Won’t fit
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1631.6B params	23.0 / 24.0 GB	Q4TightQ8Won’t fit
Nemotron-Cascade-2-30B-A3B31.6B params	23.0 / 24.0 GB	Q4TightQ8Won’t fit
Gemma 4 31B31.3B params	22.8 / 24.0 GB	Q4TightQ8Won’t fit
gemma-4-31B31.3B params	22.8 / 24.0 GB	Q4TightQ8Won’t fit
GLM-4.7-Flash31.2B params	22.8 / 24.0 GB	Q4TightQ8Won’t fit
Xing4.0-29B-A4B31.2B params	22.8 / 24.0 GB	Q4TightQ8Won’t fit
Xing4.0-29B-Intent31.2B params	22.8 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-VL-30B-A3B-Instruct31.1B params	22.7 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-VL-30B-A3B-Thinking31.1B params	22.7 / 24.0 GB	Q4TightQ8Won’t fit
UI-Venus-1.5-30B-A3B31.1B params	22.7 / 24.0 GB	Q4TightQ8Won’t fit
kanana-2-30b-a3b-instruct-260130.7B params	22.4 / 24.0 GB	Q4TightQ8Won’t fit
kanana-2-30b-a3b-instruct30.7B params	22.4 / 24.0 GB	Q4TightQ8Won’t fit
kanana-2-30b-a3b-thinking-260130.7B params	22.4 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-30B-A3B30.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-Coder-30B-A3B-Instruct30.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-30B-A3B-Instruct-250730.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Qwen3-30B-A3B-Thinking-250730.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Tongyi-DeepResearch-30B-A3B30.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Rubicon-Preview30.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
North-Mini-Code-1.030.5B params	22.3 / 24.0 GB	Q4TightQ8Won’t fit
Hy-Memory-30B-A3B30.1B params	22.0 / 24.0 GB	Q4TightQ8Won’t fit
Hy-MT2-30B-A3B30.1B params	22.0 / 24.0 GB	Q4TightQ8Won’t fit
granite-switch-4.1-30b30.0B params	21.9 / 24.0 GB	Q4TightQ8Won’t fit
Muse-Glimmer-30B29.8B params	21.8 / 24.0 GB	Q4TightQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-Thinking29.7B params	21.7 / 24.0 GB	Q4TightQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-PT29.4B params	21.5 / 24.0 GB	Q4FitsQ8Won’t fit
granite-4.2-30b29.3B params	21.4 / 24.0 GB	Q4FitsQ8Won’t fit
granite-4.1-30b28.9B params	21.1 / 24.0 GB	Q4FitsQ8Won’t fit
Raon-1 mini preview28.8B params	21.1 / 24.0 GB	Q4FitsQ8Won’t fit
Qwen3.8-27B27.8B params	20.4 / 24.0 GB	Q4FitsQ8Won’t fit
Tess-4.5-27B27.8B params	20.4 / 24.0 GB	Q4FitsQ8Won’t fit
Qwen3.6-27B27.8B params	20.4 / 24.0 GB	Q4FitsQ8Won’t fit
Qwen3.5-27B27.8B params	20.4 / 24.0 GB	Q4FitsQ8Won’t fit
gemma-3-27b-it27.4B params	20.1 / 24.0 GB	Q4FitsQ8Won’t fit
Arete-27B27.4B params	20.1 / 24.0 GB	Q4FitsQ8Won’t fit
gemma-2-27b-it27.2B params	20.0 / 24.0 GB	Q4FitsQ8Won’t fit
gemma-2-27b27.2B params	20.0 / 24.0 GB	Q4FitsQ8Won’t fit
Qwen3.8-27B-Longbook27-v227.1B params	19.9 / 24.0 GB	Q4FitsQ8Won’t fit
medgemma-27b-text-it27.0B params	19.8 / 24.0 GB	Q4FitsQ8Won’t fit
Hemmingway-1.526.9B params	19.8 / 24.0 GB	Q4FitsQ8Won’t fit
Trinity-Mini26.1B params	19.2 / 24.0 GB	Q4FitsQ8Won’t fit
diffusiongemma-26B-A4B-it25.8B params	19.0 / 24.0 GB	Q4FitsQ8Won’t fit
Gemma 4 26B A4B25.8B params	19.0 / 24.0 GB	Q4FitsQ8Won’t fit
gemma-4-26B-A4B25.8B params	19.0 / 24.0 GB	Q4FitsQ8Won’t fit
Voxtral-Small-24B-250724.3B params	17.9 / 24.0 GB	Q4FitsQ8Won’t fit
Mistral-Small-3.2-24B-Instruct-250624.0B params	17.7 / 24.0 GB	Q4FitsQ8Won’t fit
Mistral-Small-3.1-24B-Instruct-250324.0B params	17.7 / 24.0 GB	Q4FitsQ8Won’t fit
LFM2.5-24B-A2B24.0B params	17.7 / 24.0 GB	Q4FitsQ8Won’t fit
LFM2-24B-A2B23.8B params	17.6 / 24.0 GB	Q4FitsQ8Won’t fit
Mistral-Small-24B-Instruct-250123.6B params	17.4 / 24.0 GB	Q4FitsQ8Won’t fit
EuroLLM-22B-Instruct-251222.6B params	16.8 / 24.0 GB	Q4FitsQ8Won’t fit
EuroLLM-22B-251222.6B params	16.8 / 24.0 GB	Q4FitsQ8Won’t fit
solar-pro-preview-instruct22.1B params	16.4 / 24.0 GB	Q4FitsQ8Won’t fit
ERNIE-4.5-21B-A3B-PT21.9B params	16.3 / 24.0 GB	Q4FitsQ8Won’t fit
ERNIE-4.5-21B-A3B-Thinking21.8B params	16.2 / 24.0 GB	Q4FitsQ8Won’t fit
gpt-oss-20b20.9B params	15.6 / 24.0 GB	Q4FitsQ8Won’t fit
reka-flash-320.9B params	15.6 / 24.0 GB	Q4FitsQ8Won’t fit
maple-preview20.2B params	15.1 / 24.0 GB	Q4FitsQ8Won’t fit
internlm2_5-20b-chat19.9B params	14.8 / 24.0 GB	Q4FitsQ8Won’t fit
Groot-19B-A3B-26102319.4B params	14.5 / 24.0 GB	Q4FitsQ8Won’t fit
Ling-Coder-lite16.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ring-lite-distill-preview16.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ling-lite16.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ling-lite-1.5-250716.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ling-lite-1.516.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ring-lite16.8B params	12.7 / 24.0 GB	Q4FitsQ8Fits
Ring-mini-linear-2.016.4B params	12.5 / 24.0 GB	Q4FitsQ8Fits
Kimi-VL-A3B-Instruct16.4B params	12.4 / 24.0 GB	Q4FitsQ8Fits
Kimi-VL-A3B-Thinking16.4B params	12.4 / 24.0 GB	Q4FitsQ8Fits
Kimi-VL-A3B-Thinking-250616.4B params	12.4 / 24.0 GB	Q4FitsQ8Fits
LLaDA2.1-mini16.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
LLaDA2.0-mini16.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
Ling-mini-2.016.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
Ring-mini-2.016.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
LLaDA2.2-mini16.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
LLaDA2.0-mini-preview16.3B params	12.3 / 24.0 GB	Q4FitsQ8Fits
Moonlight-16B-A3B-Instruct16.0B params	12.1 / 24.0 GB	Q4FitsQ8Fits
Moonlight-16B-A3B16.0B params	12.1 / 24.0 GB	Q4FitsQ8Fits
Instella-MoE-16B-A3B-Think15.9B params	12.1 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-Coder-V2-Lite-Instruct15.7B params	12.0 / 24.0 GB	Q4FitsQ8Fits
kanana-1.5-15.7b-a3b-instruct15.7B params	11.9 / 24.0 GB	Q4FitsQ8Fits
Phi-4-reasoning-vision-15B15.1B params	11.5 / 24.0 GB	Q4FitsQ8Fits
Apriel-Nemotron-15b-Thinker15.0B params	11.4 / 24.0 GB	Q4FitsQ8Fits
Apriel-1.5-15b-Thinker14.9B params	11.4 / 24.0 GB	Q4FitsQ8Fits
Apriel-1.6-15b-Thinker14.9B params	11.4 / 24.0 GB	Q4FitsQ8Fits
Carbon-X (DNA)14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-14B-Instruct14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-14B14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-14B-Instruct14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-14B14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-14B-Instruct-1M14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-14B14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
Qwen3-14B14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
JanusCoder-14B14.8B params	11.3 / 24.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Think-14B14.7B params	11.3 / 24.0 GB	Q4FitsQ8Fits
phi-414.7B params	11.2 / 24.0 GB	Q4FitsQ8Fits
Phi-4-reasoning14.7B params	11.2 / 24.0 GB	Q4FitsQ8Fits
Phi-4-reasoning-plus14.7B params	11.2 / 24.0 GB	Q4FitsQ8Fits
OLMo-2-1124-13B-Instruct13.7B params	10.6 / 24.0 GB	Q4FitsQ8Fits
Mistral-Nemo-Instruct-240712.2B params	9.5 / 24.0 GB	Q4FitsQ8Fits
gemma-3-12b-it12.2B params	9.5 / 24.0 GB	Q4FitsQ8Fits
gemma-3-12b-pt12.2B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Mellum2.1-12B-A2.5B-Thinking12.1B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Mellum2-12B-A2.5B-Thinking12.1B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Mellum2-12B-A2.5B-Instruct12.1B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Mellum2-12B-A2.5B-Thinking-SFT12.1B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Mellum2.1-12B-A2.5B-Thinking-MTP12.1B params	9.5 / 24.0 GB	Q4FitsQ8Fits
Gemma 4 12B12.0B params	9.3 / 24.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Omni-8B10.7B params	8.5 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-11B-Vision-Instruct10.7B params	8.4 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-11B-Vision10.6B params	8.4 / 24.0 GB	Q4FitsQ8Fits
Falcon3-10B-Instruct10.3B params	8.2 / 24.0 GB	Q4FitsQ8Fits
GLM-4.1V-9B-Thinking10.3B params	8.2 / 24.0 GB	Q4FitsQ8Fits
GLM-4.6V-Flash10.3B params	8.2 / 24.0 GB	Q4FitsQ8Fits
GLM-4.1V-9B-Base10.3B params	8.2 / 24.0 GB	Q4FitsQ8Fits
Glyph10.3B params	8.2 / 24.0 GB	Q4FitsQ8Fits
INTELLECT-1-Instruct10.2B params	8.1 / 24.0 GB	Q4FitsQ8Fits
INTELLECT-110.2B params	8.1 / 24.0 GB	Q4FitsQ8Fits
ZAYA1-VL-8B9.7B params	7.8 / 24.0 GB	Q4FitsQ8Fits
SenseNova-Loop-9B-Preview9.7B params	7.7 / 24.0 GB	Q4FitsQ8Fits
Ornith-1.5-9B9.7B params	7.7 / 24.0 GB	Q4FitsQ8Fits
Qwen3.5-9B9.7B params	7.7 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-SALA9.5B params	7.6 / 24.0 GB	Q4FitsQ8Fits
NorthCUA-Nano9.4B params	7.6 / 24.0 GB	Q4FitsQ8Fits
GLM-4-9B-04149.4B params	7.6 / 24.0 GB	Q4FitsQ8Fits
GLM-Z1-9B-04149.4B params	7.6 / 24.0 GB	Q4FitsQ8Fits
codegeex4-all-9b9.4B params	7.6 / 24.0 GB	Q4FitsQ8Fits
glm-4-9b-chat-hf9.4B params	7.6 / 24.0 GB	Q4FitsQ8Fits
moondream3-preview9.3B params	7.5 / 24.0 GB	Q4FitsQ8Fits
gemma-2-9b-it9.2B params	7.4 / 24.0 GB	Q4FitsQ8Fits
gemma-2-9b9.2B params	7.4 / 24.0 GB	Q4FitsQ8Fits
Gemma-SEA-LION-v3-9B-IT9.2B params	7.4 / 24.0 GB	Q4FitsQ8Fits
EuroLLM-9B-Instruct-25129.2B params	7.4 / 24.0 GB	Q4FitsQ8Fits
NVIDIA-Nemotron-Nano-9B-v28.9B params	7.2 / 24.0 GB	Q4FitsQ8Fits
ZAYA1-base8.8B params	7.2 / 24.0 GB	Q4FitsQ8Fits
Yi-Coder-9B-Chat8.8B params	7.2 / 24.0 GB	Q4FitsQ8Fits
Yi-Coder-9B8.8B params	7.2 / 24.0 GB	Q4FitsQ8Fits
internlm3-8b-instruct8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
granite-4.1-8b8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
granite-4.2-8b8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
granite-switch-4.1-8b8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
Qwen3-VL-8B-Instruct8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
Qwen3-VL-8B-Thinking8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
UI-Venus-1.5-8B8.8B params	7.1 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Nemotron-Nano-VL-8B-V18.7B params	7.1 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-V-4_58.7B params	7.1 / 24.0 GB	Q4FitsQ8Fits
aya-vision-8b8.6B params	7.0 / 24.0 GB	Q4FitsQ8Fits
Intern-S1-mini8.5B params	7.0 / 24.0 GB	Q4FitsQ8Fits
Nemotron-Labs-Diffusion-8B-Base8.5B params	6.9 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-8B-A1B8.5B params	6.9 / 24.0 GB	Q4FitsQ8Fits
Mistral-NeMo-Minitron-8B-Base8.4B params	6.9 / 24.0 GB	Q4FitsQ8Fits
LFM2-8B-A1B8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
rnj-1-instruct8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
MiMo-VL-7B-RL-25088.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
MiMo-VL-7B-RL8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
MiMo-Embodied-7B8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
MiMo-VL-7B-SFT8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-VL-7B-Instruct8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
UI-TARS-1.5-7B8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Fara-7B8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
UI-Venus-Ground-7B8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Qwen2-VL-7B-Instruct8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
UI-TARS-7B-DPO8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
UI-TARS-7B-SFT8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Seed-Coder-8B-Instruct8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Seed-Coder-8B-Reasoning8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Stable-DiffCoder-8B-Instruct8.3B params	6.8 / 24.0 GB	Q4FitsQ8Fits
Qwen3-8B8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-R1-0528-Qwen3-8B8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
Qwen3-8B-Base8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
WeDLM-8B-Instruct8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
PaCoRe-8B8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
MiniCPM4-8B8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
MiniCPM4.1-8B8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
AgentCPM-Report8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
MiniCPM4-MCP8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
granite-3.3-8b-instruct8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
granite-3.1-8b-instruct8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
granite-3.2-8b-instruct8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
granite-3.0-8b-instruct8.2B params	6.7 / 24.0 GB	Q4FitsQ8Fits
Apertus-8B-Instruct-25098.1B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Apertus-8B-25098.1B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Tulu-3-8B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Tulu-3-8B-DPO8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Tulu-3.1-8B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
kanana-1.5-8b-instruct-25058.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-8B-Instruct8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-8B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-R1-Distill-Llama-8B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Meta-Llama-3.1-8B-Instruct8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
L3-8B-Stheno-v3.28.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Nemotron-Nano-8B-v18.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Meta-Llama-3.1-8B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
LongWriter-llama3.1-8b8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Hy-MT2-7B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Nemotron 3.5 Bolt E4B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Gemma 4 E4B8.0B params	6.6 / 24.0 GB	Q4FitsQ8Fits
Ling-3.0-tiny7.9B params	6.5 / 24.0 GB	Q4FitsQ8Fits
MiMo-7B-RL7.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
MiMo-7B-Base7.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
MiMo-7B-RL-05307.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
MiMo-7B-SFT7.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
EXAONE-3.5-7.8B-Instruct7.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
EXAONE-Deep-7.8B7.8B params	6.5 / 24.0 GB	Q4FitsQ8Fits
salamandra-7b-instruct7.8B params	6.4 / 24.0 GB	Q4FitsQ8Fits
salamandra-7b7.8B params	6.4 / 24.0 GB	Q4FitsQ8Fits
salamandra-7b-instruct-26067.8B params	6.4 / 24.0 GB	Q4FitsQ8Fits
FastVLM-7B7.8B params	6.4 / 24.0 GB	Q4FitsQ8Fits
Molmo2-O-7B7.8B params	6.4 / 24.0 GB	Q4FitsQ8Fits
internlm2_5-7b-chat7.7B params	6.4 / 24.0 GB	Q4FitsQ8Fits
internlm2_5-7b-chat-1m7.7B params	6.4 / 24.0 GB	Q4FitsQ8Fits
Phi-mini-MoE-instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-7B-Instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-7B-Instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2-7B-Instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-7B-Instruct-1M7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-7B-Instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Marco-o17.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
SeaLLMs-v3-7B-Chat7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Arcee-Spark7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-7B-Instruct7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1R-7B7.6B params	6.3 / 24.0 GB	Q4FitsQ8Fits
starvector-8b-im2svg7.5B params	6.2 / 24.0 GB	Q4FitsQ8Fits
Falcon3-7B-Instruct7.5B params	6.2 / 24.0 GB	Q4FitsQ8Fits
Olmo-Hybrid-7B7.4B params	6.2 / 24.0 GB	Q4FitsQ8Fits
LLaDA-MoE-7B-A1B-Base7.4B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Zamba2-7B-Instruct7.4B params	6.1 / 24.0 GB	Q4FitsQ8Fits
OLMo-2-1124-7B-Instruct7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Think7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Instruct7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-1025-7B7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Instruct-SFT7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Think-SFT7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Instruct-DPO7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-RL-Zero-Math7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3.1-7B-RL-Zero-Math7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Olmo-3-7B-Think-DPO7.3B params	6.1 / 24.0 GB	Q4FitsQ8Fits
Mistral-7B-Instruct-v0.37.2B params	6.1 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-h-tiny6.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
OLMoE-1B-7B-0924-Instruct6.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
OLMoE-1B-7B-09246.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
OLMoE-1B-7B-0125-Instruct6.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
OLMoE-1B-7B-01256.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
OLMo-7B-0724-hf6.9B params	5.8 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-tiny-preview6.7B params	5.7 / 24.0 GB	Q4FitsQ8Fits
YuE-s1-7B-anneal-en-cot6.2B params	5.3 / 24.0 GB	Q4FitsQ8Fits
Trinity-Nano-Preview6.1B params	5.3 / 24.0 GB	Q4FitsQ8Fits
Trinity-Nano-Base6.1B params	5.3 / 24.0 GB	Q4FitsQ8Fits
gemma-3n-E2B-it5.4B params	4.8 / 24.0 GB	Q4FitsQ8Fits
Youtu-VL-4B-Instruct5.3B params	4.7 / 24.0 GB	Q4FitsQ8Fits
Gemma 4 E2B5.1B params	4.6 / 24.0 GB	Q4FitsQ8Fits
Molmo2-ER4.9B params	4.4 / 24.0 GB	Q4FitsQ8Fits
Apriel-5B-Instruct4.8B params	4.4 / 24.0 GB	Q4FitsQ8Fits
WeVisOne-VL-4B4.8B params	4.4 / 24.0 GB	Q4FitsQ8Fits
Mage-VL4.7B params	4.3 / 24.0 GB	Q4FitsQ8Fits
Mage-VL-Cowork4.7B params	4.3 / 24.0 GB	Q4FitsQ8Fits
Qwen3.5-4B4.7B params	4.2 / 24.0 GB	Q4FitsQ8Fits
S5-nano4.7B params	4.2 / 24.0 GB	Q4FitsQ8Fits
AFM-4.5B4.6B params	4.2 / 24.0 GB	Q4FitsQ8Fits
Llama-3.1-Minitron-4B-Width-Base4.5B params	4.1 / 24.0 GB	Q4FitsQ8Fits
Qwen3-VL-4B-Instruct4.4B params	4.1 / 24.0 GB	Q4FitsQ8Fits
CapRL-Qwen3VL-4B4.4B params	4.1 / 24.0 GB	Q4FitsQ8Fits
glm-edge-4b-chat4.3B params	4.0 / 24.0 GB	Q4FitsQ8Fits
gemma-3-4b-it4.3B params	4.0 / 24.0 GB	Q4FitsQ8Fits
medgemma-4b-it4.3B params	4.0 / 24.0 GB	Q4FitsQ8Fits
medgemma-1.5-4b-it4.3B params	4.0 / 24.0 GB	Q4FitsQ8Fits
gemma-3-4b-pt4.3B params	4.0 / 24.0 GB	Q4FitsQ8Fits
Nanbeige4.2-3B4.2B params	3.9 / 24.0 GB	Q4FitsQ8Fits
Phi-3.5-vision-instruct4.1B params	3.9 / 24.0 GB	Q4FitsQ8Fits
Spark-X2.6-4B4.1B params	3.9 / 24.0 GB	Q4FitsQ8Fits
Spark-X2.5-4B4.1B params	3.9 / 24.0 GB	Q4FitsQ8Fits
POINTS-Reader4.1B params	3.9 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-V-44.1B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Qwen3-4B4.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Qwen3-4B-Instruct-25074.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Qwen3-4B-Thinking-25074.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Mellum-4b-base4.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-3b-vision4.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
granite-vision-4.1-4b4.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Nemotron Labs Dory 1.0 4B4.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
NVIDIA-Nemotron-3-Nano-4B-BF164.0B params	3.8 / 24.0 GB	Q4FitsQ8Fits
Nanbeige4.1-3B3.9B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Nanbeige4-3B-Thinking-25113.9B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Phi-4-mini-flash-reasoning3.9B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Phi-4-mini-instruct3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Phi-4-mini-reasoning3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Nemotron-Labs-Diffusion-3B3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
Phi-3.5-mini-instruct3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
MediPhi-Instruct3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
MediPhi3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
MediPhi-Clinical3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
MediPhi-PubMed3.8B params	3.7 / 24.0 GB	Q4FitsQ8Fits
HY-Embodied-0.53.8B params	3.6 / 24.0 GB	Q4FitsQ8Fits
Phi-tiny-MoE-instruct3.8B params	3.6 / 24.0 GB	Q4FitsQ8Fits
CapRL-3B3.8B params	3.6 / 24.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Vision-Instruct-3B3.7B params	3.6 / 24.0 GB	Q4FitsQ8Fits
kanana-1.5-v-3b-instruct3.7B params	3.6 / 24.0 GB	Q4FitsQ8Fits
granite-4.2-3b3.7B params	3.6 / 24.0 GB	Q4FitsQ8Fits
kanana-2-3b-instruct3.5B params	3.4 / 24.0 GB	Q4FitsQ8Fits
granite-4.1-3b3.4B params	3.4 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-micro3.4B params	3.4 / 24.0 GB	Q4FitsQ8Fits
granite-3.1-3b-a800m-instruct3.3B params	3.3 / 24.0 GB	Q4FitsQ8Fits
Falcon3-3B-Instruct3.2B params	3.3 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-3B-Instruct3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-3B3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Hermes-3-Llama-3.2-3B3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
DeepHermes-3-Llama-3-3B-Preview3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
AI21-Jamba-Reasoning-3B3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-h-micro3.2B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-3B-Instruct3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
d1-3B3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-VL-3B3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-3B-Instruct3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-3B-Instruct3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-3B3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-3B3.1B params	3.2 / 24.0 GB	Q4FitsQ8Fits
SmolLM3-3B3.1B params	3.1 / 24.0 GB	Q4FitsQ8Fits
AI21-Jamba2-3B3.0B params	3.1 / 24.0 GB	Q4FitsQ8Fits
granite-swash-3b-a600m3.0B params	3.1 / 24.0 GB	Q4FitsQ8Fits
granite-vision-3.2-2b3.0B params	3.1 / 24.0 GB	Q4FitsQ8Fits
granite-vision-3.1-2b-preview3.0B params	3.1 / 24.0 GB	Q4FitsQ8Fits
granite-vision-3.3-2b-chart2csv-preview3.0B params	3.1 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-2.6B2.7B params	2.9 / 24.0 GB	Q4FitsQ8Fits
Zamba2-2.7B2.7B params	2.9 / 24.0 GB	Q4FitsQ8Fits
Zamba2-2.7B-instruct2.7B params	2.9 / 24.0 GB	Q4FitsQ8Fits
granite-3.0-2b-instruct2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
gemma-2-2b-it2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
gemma-2-2b2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
EuroMoE-2.6B-A0.6B-Instruct-Preview2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
Motif-2.6B2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
LFM2-2.6B2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
LFM2-2.6B-Exp2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
LFM2-2.6B-Transcript2.6B params	2.8 / 24.0 GB	Q4FitsQ8Fits
granite-3.3-2b-instruct2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
granite-3.1-2b-instruct2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
granite-3.2-2b-instruct2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
MiniCPM5-2B2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
MiniCPM5-2B-SFT2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
MiniCPM5-2B-Midtrain2.5B params	2.8 / 24.0 GB	Q4FitsQ8Fits
sarvam-1-v0.52.5B params	2.7 / 24.0 GB	Q4FitsQ8Fits
North-Micro-Vision-Instruct2.5B params	2.7 / 24.0 GB	Q4FitsQ8Fits
UI-TARS-2B-SFT2.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
WeVisOne-VL-2B2.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
UI-Venus-1.5-2B2.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
bitnet-b1.58-2B-4T-bf162.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
EXAONE-3.5-2.4B-Instruct2.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
EXAONE-Deep-2.4B2.4B params	2.7 / 24.0 GB	Q4FitsQ8Fits
kanana-1.5-2.1b-instruct-25052.3B params	2.6 / 24.0 GB	Q4FitsQ8Fits
Qwen3.5-2B2.3B params	2.6 / 24.0 GB	Q4FitsQ8Fits
salamandra-2b2.3B params	2.6 / 24.0 GB	Q4FitsQ8Fits
salamandra-2b-instruct2.3B params	2.6 / 24.0 GB	Q4FitsQ8Fits
SmolVLM2-2.2B-Instruct2.2B params	2.6 / 24.0 GB	Q4FitsQ8Fits
SmolVLM-Base2.2B params	2.6 / 24.0 GB	Q4FitsQ8Fits
Qwen2-VL-2B-Instruct2.2B params	2.5 / 24.0 GB	Q4FitsQ8Fits
LFM3-Omni-2B2.2B params	2.5 / 24.0 GB	Q4FitsQ8Fits
granite-swash-2b2.1B params	2.5 / 24.0 GB	Q4FitsQ8Fits
Qwen3-VL-2B-Instruct2.1B params	2.5 / 24.0 GB	Q4FitsQ8Fits
kanana-nano-2.1b-instruct2.1B params	2.5 / 24.0 GB	Q4FitsQ8Fits
Hy-MT2-1.8B2.0B params	2.4 / 24.0 GB	Q4FitsQ8Fits
Qwen3-1.7B2.0B params	2.4 / 24.0 GB	Q4FitsQ8Fits
Youtu-LLM-2B2.0B params	2.4 / 24.0 GB	Q4FitsQ8Fits
LFM3-VL-2B1.9B params	2.3 / 24.0 GB	Q4FitsQ8Fits
d1-2B1.9B params	2.3 / 24.0 GB	Q4FitsQ8Fits
FastVLM-1.5B1.9B params	2.3 / 24.0 GB	Q4FitsQ8Fits
internlm2_5-1_8b-chat1.9B params	2.3 / 24.0 GB	Q4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-1.5B1.8B params	2.2 / 24.0 GB	Q4FitsQ8Fits
LFM3-1.7B1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
Qwen3-1.7B-Base1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
SmolLM2-1.7B1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
SmolLM-1.7B1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
Spark-X2.6-1.7B1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
Falcon3-1B-Instruct1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
EuroLLM-1.7B-Instruct1.7B params	2.2 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-1b1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-VL-1.6B1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
glm-edge-1.5b-chat1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
LFM2-VL-1.6B1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-1.5B-Deep-Instruct1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-1.5B-Instruct1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.7-1B1.6B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-1.5B-Instruct1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-1.5B1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-1.5B-Instruct1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-1.5B1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-1.5B1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2-1.5B-Instruct1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-1.5B-Instruct1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
ReaderLM-v21.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
reader-lm-1.5b1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
Hymba-1.5B-Instruct1.5B params	2.1 / 24.0 GB	Q4FitsQ8Fits
OLMo-2-0425-1B1.5B params	2.0 / 24.0 GB	Q4FitsQ8Fits
OLMo-2-0425-1B-Instruct1.5B params	2.0 / 24.0 GB	Q4FitsQ8Fits
Yi-Coder-1.5B-Chat1.5B params	2.0 / 24.0 GB	Q4FitsQ8Fits
Bolmo-1B1.5B params	2.0 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-h-1b1.5B params	2.0 / 24.0 GB	Q4FitsQ8Fits
granite-3.1-1b-a400m-instruct1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
granite-3.0-1b-a400m-instruct1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.61.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.6-Thinking1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
kanana-2-1.3b-instruct1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
OLMo-1B-0724-hf1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
EXAONE-4.0-1.2B1.3B params	1.9 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-1B-Instruct1.2B params	1.9 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-1B1.2B params	1.9 / 24.0 GB	Q4FitsQ8Fits
Llama-3.2-1B1.2B params	1.9 / 24.0 GB	Q4FitsQ8Fits
Zamba2-1.2B-instruct1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
HRM-Text-1B1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
AMD-OLMo-1B1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-Instruct1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
LFM2-1.2B1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-Thinking1.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-JP-2026061.2B params	1.8 / 24.0 GB	Q4FitsQ8Fits
MiniCPM5-1B1.1B params	1.8 / 24.0 GB	Q4FitsQ8Fits
MiniCPM5-1B-SFT1.1B params	1.8 / 24.0 GB	Q4FitsQ8Fits
gemma-3-1b-it1.0B params	1.7 / 24.0 GB	Q4FitsQ8Fits
gemma-3-1b-pt1.0B params	1.7 / 24.0 GB	Q4FitsQ8Fits
Qwen3.5-0.8B0.9B params	1.6 / 24.0 GB	Q4FitsQ8Fits
Falcon-E-3B-Instruct0.9B params	1.6 / 24.0 GB	Q4FitsQ8Fits
bitnet-b1.58-2B-4T0.8B params	1.6 / 24.0 GB	Q4FitsQ8Fits
FastVLM-0.5B0.8B params	1.5 / 24.0 GB	Q4FitsQ8Fits
Qwen3-0.6B0.8B params	1.5 / 24.0 GB	Q4FitsQ8Fits
LFM2-700M0.7B params	1.5 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-Tiny-R-0.6B0.6B params	1.4 / 24.0 GB	Q4FitsQ8Fits
Qwen3-0.6B-Base0.6B params	1.4 / 24.0 GB	Q4FitsQ8Fits
d1-omni-600M0.6B params	1.4 / 24.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Text-Instruct-0.5B0.6B params	1.4 / 24.0 GB	Q4FitsQ8Fits
Falcon-H1-0.5B-Instruct0.5B params	1.4 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-0.5B-Instruct0.5B params	1.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-0.5B0.5B params	1.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2-0.5B-Instruct0.5B params	1.3 / 24.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-0.5B-Instruct0.5B params	1.3 / 24.0 GB	Q4FitsQ8Fits
LFM2-VL-450M0.5B params	1.3 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-VL-450M0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
Apertus-v1.1-0.5B0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
MiniCPM4-0.5B0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
circuit-sparsity0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
SmolLM2-360M0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
SmolLM-360M0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
ERNIE-4.5-0.3B-PT0.4B params	1.3 / 24.0 GB	Q4FitsQ8Fits
LFM2.5-350M0.4B params	1.2 / 24.0 GB	Q4FitsQ8Fits
LFM2-350M0.4B params	1.2 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-350m0.4B params	1.2 / 24.0 GB	Q4FitsQ8Fits
granite-4.0-h-350m0.3B params	1.2 / 24.0 GB	Q4FitsQ8Fits

Quick answers

What is the biggest LLM that runs on NVIDIA RTX A5000?: ERNIE-4.5-VL-28B-A3B-PT (29.4B parameters, about 21.5 GB at Q4) is the largest model we track that runs fully in its 24 GB of usable memory.
How many local LLMs can NVIDIA RTX A5000 run?: 423 of the 656 open models we track run fully at Q4 on it.
How much memory can NVIDIA RTX A5000 use for AI models?: It has 24 GB of VRAM, and we count 24 GB as usable for model weights and cache.

What LLMs can I run on NVIDIA RTX A5000?

Every tracked model on this machine

Quick answers

New models, and what they need to run