What is the biggest LLM that runs on GeForce RTX 4060 Ti 8GB?

NVIDIA-Nemotron-Nano-9B-v2 (8.9B parameters, about 7.2 GB at Q4) is the largest model we track that runs fully in its 8 GB of usable memory.

How many local LLMs can GeForce RTX 4060 Ti 8GB run?

315 of the 656 open models we track run fully at Q4 on it.

How much memory can GeForce RTX 4060 Ti 8GB use for AI models?

It has 8 GB of VRAM, and we count 8 GB as usable for model weights and cache.

GeForce RTX 4060 Ti 8GB for local LLMs

The GeForce RTX 4060 Ti 8GB has 8 GB of VRAM (8 GB usable for models) and runs 315 of the 656 open models we track at Q4_K_M, up to NVIDIA-Nemotron-Nano-9B-v2 (8.9B parameters, about 7.2 GB). These are estimates, not measured benchmarks.

Looking for the model list first? See what you can run on the GeForce RTX 4060 Ti 8GB.

Memory: 8 GB VRAM
Memory bandwidth: 288 GB/s
Usable for models: 8 GB
Runs fully at Q4: 315 / 656

Spec source · Best GPU under $500 · Best GPU for local LLMs · Multi-GPU · Best used GPU for local LLMs · NVIDIA vs AMD vs Mac · Tokens per second · Best LLMs for 8 GB

$399 list (launch MSRP; discontinued), source, 2026-10-28
Street prices may be higher than list.
Discontinued; buy used. Price shown is the original launch MSRP.

Check price on Amazon Check price on eBay

We may earn a commission when you buy through these links, at no extra cost to you.

Every tracked model on this machine

Sorted largest first. The tick marks 90% of usable memory.FitsTightWon’t fit

Model	Q4 need vs. 8 GB	Verdict
Kimi-K3.1-Pro2780.1B params	1939.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K32779.9B params	1939.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-2.4T-A95B2446.2B params	1706.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-2.51779.8B params	1241.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-2.01775.6B params	1238.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro-08131650.5B params	1151.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro1598.8B params	1115.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.51026.9B params	716.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.61026.9B params	716.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2.7-Code1026.9B params	716.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Thinking1026.4B params	716.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Instruct-09051026.4B params	716.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-K2-Instruct1026.4B params	716.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-RL1024.2B params	715.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-MOPD1024.2B params	715.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.5-Pro1023.2B params	714.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-2.5-1T1012.2B params	706.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-2.5-1T1012.2B params	706.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-1T999.7B params	698.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-1T999.7B params	698.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Inkling952.4B params	665.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S1-Pro917.0B params	640.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Muse-Spark-Open-816B-A42B822.0B params	574.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hy4-preview780.0B params	544.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5753.9B params	526.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.1753.9B params	526.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3753.3B params	526.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.2753.3B params	526.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3-BF16753.3B params	526.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
K-EXAONE-2.0-750B-A37B749.4B params	523.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Exp685.4B params	478.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Speciale685.4B params	478.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.2685.4B params	478.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3-0324684.5B params	478.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-0528684.5B params	478.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V3.1-Base684.5B params	478.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1684.5B params	478.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Zero684.5B params	478.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Step-5-Preview604.3B params	422.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.1-flash572.1B params	399.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.1-flash-1019572.1B params	399.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Chat561.9B params	392.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking561.9B params	392.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking-2601561.9B params	392.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16560.5B params	391.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Beam-1.0-501B-A23B501.3B params	350.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Coder-480B-A35B-Instruct480.2B params	335.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M1-40k456.1B params	319.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M1-80k456.1B params	319.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M3427.0B params	298.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-424B-A47B-PT423.5B params	296.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S2-397B403.4B params	282.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-397B-A17B403.4B params	282.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Maverick-17B-128E-Instruct401.6B params	281.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Thinking398.6B params	278.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Preview398.6B params	278.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-TrueBase398.6B params	278.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Large-Base398.6B params	278.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2.5-Pro396.8B params	277.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2-Pro396.8B params	277.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NorthCUA-Pro396.8B params	277.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5358.3B params	250.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.7358.3B params	250.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.6356.8B params	249.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-5.3-Flash321.3B params	225.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Motif-3314.8B params	220.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.5310.8B params	217.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-RL310.8B params	217.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-MOPD310.8B params	217.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiMo-V2-Flash309.8B params	217.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-Vision-Exp304.6B params	213.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-0731304.2B params	213.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-300B-A47B-PT300.5B params	210.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hy3298.8B params	209.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hy3-preview298.8B params	209.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-plus292.5B params	205.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash290.9B params	203.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MemT-1.0-Flash287.9B params	201.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-5.5-Flash-Base286.2B params	200.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Inkling-Small266.0B params	186.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-Ultra-253B-v1253.4B params	177.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Solar-Open2-250B250.3B params	175.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S1240.7B params	168.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
K-EXAONE-236B-A23B237.1B params	166.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2.5235.7B params	165.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct235.7B params	165.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2.5-1210235.7B params	165.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-V2-Chat-0628235.7B params	165.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct-0724235.7B params	165.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-235B-A22B-Instruct235.7B params	165.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B235.1B params	164.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Instruct-2507235.1B params	164.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Thinking-2507235.1B params	164.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2228.7B params	160.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.5228.7B params	160.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.7228.7B params	160.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniMax-M2.1228.7B params	160.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
command-a-plus-05-2026-bf16218.8B params	153.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.7-Flash201.4B params	141.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.5-Flash199.4B params	140.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Step-3.5-Flash-Base-Midtrain197.8B params	138.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-Flash-Next180.0B params	126.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mixtral-8x22B-Instruct-v0.1140.6B params	99.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-Sante127.5B params	89.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash127.5B params	89.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-Fin127.5B params	89.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ornith-1.5-122B125.1B params	88.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-122B-A10B125.1B params	88.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Devstral-2-123B-Instruct-2512125.0B params	88.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-3.0-flash-VL124.8B params	88.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Super-120B-A12B-BF16123.6B params	87.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Small-4-119B-2603119.4B params	84.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Youtu-LLM-A7B-Base118.4B params	83.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Laguna-S-2.1117.6B params	83.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gpt-oss-120b116.8B params	82.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
c4ai-command-a-03-2025111.1B params	78.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5-Air110.5B params	78.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E-Instruct108.6B params	76.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E108.6B params	76.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.5V107.7B params	76.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.6V107.7B params	76.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-2.6-flash107.5B params	75.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
sarvam-105b106.0B params	74.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-flash-2.0102.9B params	72.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.1-flash102.9B params	72.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-flash-2.0102.9B params	72.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.2-flash102.9B params	72.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Solar-Open-100B102.7B params	72.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Instruct81.3B params	57.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Thinking81.3B params	57.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
AliceAI-Foundation-80B-A3B-Base81.3B params	57.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hunyuan-A13B-Instruct80.4B params	57.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Coder-Next79.7B params	56.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kolibri-178.1B params	55.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-VL-72B-Instruct73.4B params	52.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
UI-TARS-72B-DPO73.4B params	52.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-72B-Instruct72.7B params	51.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apertus-70B-Instruct-250970.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apertus-70B-250970.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.3-70B-Instruct70.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Llama-70B70.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.1-70B70.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hermes-2-Theta-Llama-3-70B70.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Meta-Llama-3.1-70B-Instruct70.6B params	50.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LongCat-Flash-Lite69.1B params	49.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
AI21-Jamba2-Mini51.6B params	37.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-51B-Instruct51.5B params	36.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v1_549.9B params	35.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v149.9B params	35.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Phi-3.5-MoE-instruct41.9B params	30.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GRIN-MoE41.9B params	30.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ALIA-40b40.4B params	29.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Loop-Instruct39.8B params	28.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Instruct39.8B params	28.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Recursive Flint39.8B params	28.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
K2-Horizon-MoVA-36B-A4B37.4B params	27.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Seed-OSS-36B-Instruct36.2B params	26.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Seed-OSS-36B-Base-woSyn36.2B params	26.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Intern-S2-Preview36.1B params	26.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ornith-1.5-35B-A3B36.0B params	26.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Sage-A1.536.0B params	26.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.6-35B-A3B36.0B params	26.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-35B-A3B36.0B params	26.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apex-35B-V1.036.0B params	26.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MiniCPM-V-4.7-35B-A3B35.2B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
MemT-1.0-Nano35.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Agents-A135.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2.5-mini35.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nex-N2-mini35.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Axion-1.0-35B35.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NorthCUA-Mini35.1B params	25.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Clover-35B-A3B-Preview34.7B params	25.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen-AgentWorld-35B-A3B34.7B params	25.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
KAT-Coder-V2.5-Dev34.7B params	25.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-4.5-33B34.4B params	24.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Falcon-H1-34B-Instruct33.6B params	24.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-VL-32B-Instruct33.5B params	24.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Laguna-XS.233.4B params	24.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-32B-Instruct33.4B params	24.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
HyperCLOVAX-SEED-Think-32B33.3B params	24.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
FlexOlmo-7x7B-1T33.3B params	24.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-Z1-Rumination-32B-041433.1B params	24.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nanbeige4.3-33B33.1B params	24.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1633.0B params	24.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-Coder-32B-Instruct32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Qwen-32B32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-32B-Instruct32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
QwQ-32B32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
QwQ-32B-Preview32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-32B32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-Coder-32B32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-32B32.8B params	23.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-Z1-32B-041432.6B params	23.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4-32B-041432.6B params	23.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
OLMo-2-0325-32B-Instruct32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
OLMo-2-0325-32B32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Olmo-3-32B-Think32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Olmo-3-1125-32B32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Instruct32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Think32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Instruct-DPO32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
granite-4.0-h-small32.2B params	23.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
sarvam-30b32.2B params	23.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-4.0-32B32.0B params	23.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-4.0.1-32B32.0B params	23.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-3.5-32B-Instruct32.0B params	23.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EXAONE-Deep-32B32.0B params	23.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1631.6B params	23.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1631.6B params	23.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Nemotron-Cascade-2-30B-A3B31.6B params	23.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Gemma 4 31B31.3B params	22.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-4-31B31.3B params	22.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.7-Flash31.2B params	22.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Xing4.0-29B-A4B31.2B params	22.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Xing4.0-29B-Intent31.2B params	22.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-30B-A3B-Instruct31.1B params	22.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-VL-30B-A3B-Thinking31.1B params	22.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
UI-Venus-1.5-30B-A3B31.1B params	22.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-instruct-260130.7B params	22.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-instruct30.7B params	22.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-thinking-260130.7B params	22.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-30B-A3B30.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-Coder-30B-A3B-Instruct30.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-30B-A3B-Instruct-250730.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-30B-A3B-Thinking-250730.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Tongyi-DeepResearch-30B-A3B30.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Rubicon-Preview30.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
North-Mini-Code-1.030.5B params	22.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hy-Memory-30B-A3B30.1B params	22.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hy-MT2-30B-A3B30.1B params	22.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
granite-switch-4.1-30b30.0B params	21.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Muse-Glimmer-30B29.8B params	21.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-Thinking29.7B params	21.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-PT29.4B params	21.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
granite-4.2-30b29.3B params	21.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
granite-4.1-30b28.9B params	21.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Raon-1 mini preview28.8B params	21.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-27B27.8B params	20.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Tess-4.5-27B27.8B params	20.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.6-27B27.8B params	20.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.5-27B27.8B params	20.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-3-27b-it27.4B params	20.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Arete-27B27.4B params	20.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-2-27b-it27.2B params	20.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-2-27b27.2B params	20.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3.8-27B-Longbook27-v227.1B params	19.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
medgemma-27b-text-it27.0B params	19.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Hemmingway-1.526.9B params	19.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Trinity-Mini26.1B params	19.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
diffusiongemma-26B-A4B-it25.8B params	19.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Gemma 4 26B A4B25.8B params	19.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-4-26B-A4B25.8B params	19.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Voxtral-Small-24B-250724.3B params	17.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Small-3.2-24B-Instruct-250624.0B params	17.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Small-3.1-24B-Instruct-250324.0B params	17.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LFM2.5-24B-A2B24.0B params	17.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LFM2-24B-A2B23.8B params	17.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Small-24B-Instruct-250123.6B params	17.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EuroLLM-22B-Instruct-251222.6B params	16.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
EuroLLM-22B-251222.6B params	16.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
solar-pro-preview-instruct22.1B params	16.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-21B-A3B-PT21.9B params	16.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ERNIE-4.5-21B-A3B-Thinking21.8B params	16.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gpt-oss-20b20.9B params	15.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
reka-flash-320.9B params	15.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
maple-preview20.2B params	15.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
internlm2_5-20b-chat19.9B params	14.8 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Groot-19B-A3B-26102319.4B params	14.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-Coder-lite16.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-lite-distill-preview16.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-lite16.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-lite-1.5-250716.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-lite-1.516.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-lite16.8B params	12.7 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-mini-linear-2.016.4B params	12.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-VL-A3B-Instruct16.4B params	12.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-VL-A3B-Thinking16.4B params	12.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Kimi-VL-A3B-Thinking-250616.4B params	12.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.1-mini16.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.0-mini16.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ling-mini-2.016.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Ring-mini-2.016.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.2-mini16.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
LLaDA2.0-mini-preview16.3B params	12.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Moonlight-16B-A3B-Instruct16.0B params	12.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Moonlight-16B-A3B16.0B params	12.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Instella-MoE-16B-A3B-Think15.9B params	12.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Lite-Instruct15.7B params	12.0 / 8.0 GB	Q4Won’t fitQ8Won’t fit
kanana-1.5-15.7b-a3b-instruct15.7B params	11.9 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Phi-4-reasoning-vision-15B15.1B params	11.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apriel-Nemotron-15b-Thinker15.0B params	11.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apriel-1.5-15b-Thinker14.9B params	11.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Apriel-1.6-15b-Thinker14.9B params	11.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Carbon-X (DNA)14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-Coder-14B-Instruct14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Qwen-14B14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-14B-Instruct14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-14B14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-14B-Instruct-1M14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen2.5-Coder-14B14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Qwen3-14B14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
JanusCoder-14B14.8B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
HyperCLOVAX-SEED-Think-14B14.7B params	11.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
phi-414.7B params	11.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Phi-4-reasoning14.7B params	11.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Phi-4-reasoning-plus14.7B params	11.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
OLMo-2-1124-13B-Instruct13.7B params	10.6 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mistral-Nemo-Instruct-240712.2B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-3-12b-it12.2B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
gemma-3-12b-pt12.2B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mellum2.1-12B-A2.5B-Thinking12.1B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mellum2-12B-A2.5B-Thinking12.1B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mellum2-12B-A2.5B-Instruct12.1B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mellum2-12B-A2.5B-Thinking-SFT12.1B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Mellum2.1-12B-A2.5B-Thinking-MTP12.1B params	9.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Gemma 4 12B12.0B params	9.3 / 8.0 GB	Q4Won’t fitQ8Won’t fit
HyperCLOVAX-SEED-Omni-8B10.7B params	8.5 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.2-11B-Vision-Instruct10.7B params	8.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Llama-3.2-11B-Vision10.6B params	8.4 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Falcon3-10B-Instruct10.3B params	8.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.1V-9B-Thinking10.3B params	8.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.6V-Flash10.3B params	8.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
GLM-4.1V-9B-Base10.3B params	8.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
Glyph10.3B params	8.2 / 8.0 GB	Q4Won’t fitQ8Won’t fit
INTELLECT-1-Instruct10.2B params	8.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
INTELLECT-110.2B params	8.1 / 8.0 GB	Q4Won’t fitQ8Won’t fit
ZAYA1-VL-8B9.7B params	7.8 / 8.0 GB	Q4TightQ8Won’t fit
SenseNova-Loop-9B-Preview9.7B params	7.7 / 8.0 GB	Q4TightQ8Won’t fit
Ornith-1.5-9B9.7B params	7.7 / 8.0 GB	Q4TightQ8Won’t fit
Qwen3.5-9B9.7B params	7.7 / 8.0 GB	Q4TightQ8Won’t fit
MiniCPM-SALA9.5B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
NorthCUA-Nano9.4B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
GLM-4-9B-04149.4B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
GLM-Z1-9B-04149.4B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
codegeex4-all-9b9.4B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
glm-4-9b-chat-hf9.4B params	7.6 / 8.0 GB	Q4TightQ8Won’t fit
moondream3-preview9.3B params	7.5 / 8.0 GB	Q4TightQ8Won’t fit
gemma-2-9b-it9.2B params	7.4 / 8.0 GB	Q4TightQ8Won’t fit
gemma-2-9b9.2B params	7.4 / 8.0 GB	Q4TightQ8Won’t fit
Gemma-SEA-LION-v3-9B-IT9.2B params	7.4 / 8.0 GB	Q4TightQ8Won’t fit
EuroLLM-9B-Instruct-25129.2B params	7.4 / 8.0 GB	Q4TightQ8Won’t fit
NVIDIA-Nemotron-Nano-9B-v28.9B params	7.2 / 8.0 GB	Q4FitsQ8Won’t fit
ZAYA1-base8.8B params	7.2 / 8.0 GB	Q4FitsQ8Won’t fit
Yi-Coder-9B-Chat8.8B params	7.2 / 8.0 GB	Q4FitsQ8Won’t fit
Yi-Coder-9B8.8B params	7.2 / 8.0 GB	Q4FitsQ8Won’t fit
internlm3-8b-instruct8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
granite-4.1-8b8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
granite-4.2-8b8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
granite-switch-4.1-8b8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen3-VL-8B-Instruct8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen3-VL-8B-Thinking8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
UI-Venus-1.5-8B8.8B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-Nemotron-Nano-VL-8B-V18.7B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
MiniCPM-V-4_58.7B params	7.1 / 8.0 GB	Q4FitsQ8Won’t fit
aya-vision-8b8.6B params	7.0 / 8.0 GB	Q4FitsQ8Won’t fit
Intern-S1-mini8.5B params	7.0 / 8.0 GB	Q4FitsQ8Won’t fit
Nemotron-Labs-Diffusion-8B-Base8.5B params	6.9 / 8.0 GB	Q4FitsQ8Won’t fit
LFM2.5-8B-A1B8.5B params	6.9 / 8.0 GB	Q4FitsQ8Won’t fit
Mistral-NeMo-Minitron-8B-Base8.4B params	6.9 / 8.0 GB	Q4FitsQ8Won’t fit
LFM2-8B-A1B8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
rnj-1-instruct8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-VL-7B-RL-25088.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-VL-7B-RL8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-Embodied-7B8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-VL-7B-SFT8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-VL-7B-Instruct8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
UI-TARS-1.5-7B8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Fara-7B8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
UI-Venus-Ground-7B8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2-VL-7B-Instruct8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
UI-TARS-7B-DPO8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
UI-TARS-7B-SFT8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Seed-Coder-8B-Instruct8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Seed-Coder-8B-Reasoning8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Stable-DiffCoder-8B-Instruct8.3B params	6.8 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen3-8B8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
DeepSeek-R1-0528-Qwen3-8B8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen3-8B-Base8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
WeDLM-8B-Instruct8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
PaCoRe-8B8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
MiniCPM4-8B8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
MiniCPM4.1-8B8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
AgentCPM-Report8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
MiniCPM4-MCP8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
granite-3.3-8b-instruct8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
granite-3.1-8b-instruct8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
granite-3.2-8b-instruct8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
granite-3.0-8b-instruct8.2B params	6.7 / 8.0 GB	Q4FitsQ8Won’t fit
Apertus-8B-Instruct-25098.1B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Apertus-8B-25098.1B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-Tulu-3-8B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-Tulu-3-8B-DPO8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-Tulu-3.1-8B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
kanana-1.5-8b-instruct-25058.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-8B-Instruct8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-8B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
DeepSeek-R1-Distill-Llama-8B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Meta-Llama-3.1-8B-Instruct8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
L3-8B-Stheno-v3.28.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Llama-3.1-Nemotron-Nano-8B-v18.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Meta-Llama-3.1-8B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
LongWriter-llama3.1-8b8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Hy-MT2-7B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Nemotron 3.5 Bolt E4B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Gemma 4 E4B8.0B params	6.6 / 8.0 GB	Q4FitsQ8Won’t fit
Ling-3.0-tiny7.9B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-7B-RL7.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-7B-Base7.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-7B-RL-05307.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
MiMo-7B-SFT7.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
EXAONE-3.5-7.8B-Instruct7.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
EXAONE-Deep-7.8B7.8B params	6.5 / 8.0 GB	Q4FitsQ8Won’t fit
salamandra-7b-instruct7.8B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
salamandra-7b7.8B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
salamandra-7b-instruct-26067.8B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
FastVLM-7B7.8B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
Molmo2-O-7B7.8B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
internlm2_5-7b-chat7.7B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
internlm2_5-7b-chat-1m7.7B params	6.4 / 8.0 GB	Q4FitsQ8Won’t fit
Phi-mini-MoE-instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-Coder-7B-Instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
DeepSeek-R1-Distill-Qwen-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-7B-Instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2-7B-Instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-Coder-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-7B-Instruct-1M7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-Math-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Qwen2.5-Math-7B-Instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Marco-o17.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
SeaLLMs-v3-7B-Chat7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Arcee-Spark7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Falcon-H1-7B-Instruct7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
Falcon-H1R-7B7.6B params	6.3 / 8.0 GB	Q4FitsQ8Won’t fit
starvector-8b-im2svg7.5B params	6.2 / 8.0 GB	Q4FitsQ8Won’t fit
Falcon3-7B-Instruct7.5B params	6.2 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-Hybrid-7B7.4B params	6.2 / 8.0 GB	Q4FitsQ8Won’t fit
LLaDA-MoE-7B-A1B-Base7.4B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Zamba2-7B-Instruct7.4B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
OLMo-2-1124-7B-Instruct7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Think7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Instruct7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-1025-7B7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Instruct-SFT7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Think-SFT7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Instruct-DPO7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-RL-Zero-Math7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3.1-7B-RL-Zero-Math7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Olmo-3-7B-Think-DPO7.3B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
Mistral-7B-Instruct-v0.37.2B params	6.1 / 8.0 GB	Q4FitsQ8Won’t fit
granite-4.0-h-tiny6.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
OLMoE-1B-7B-0924-Instruct6.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
OLMoE-1B-7B-09246.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
OLMoE-1B-7B-0125-Instruct6.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
OLMoE-1B-7B-01256.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
OLMo-7B-0724-hf6.9B params	5.8 / 8.0 GB	Q4FitsQ8Won’t fit
granite-4.0-tiny-preview6.7B params	5.7 / 8.0 GB	Q4FitsQ8Won’t fit
YuE-s1-7B-anneal-en-cot6.2B params	5.3 / 8.0 GB	Q4FitsQ8Won’t fit
Trinity-Nano-Preview6.1B params	5.3 / 8.0 GB	Q4FitsQ8Won’t fit
Trinity-Nano-Base6.1B params	5.3 / 8.0 GB	Q4FitsQ8Won’t fit
gemma-3n-E2B-it5.4B params	4.8 / 8.0 GB	Q4FitsQ8Tight
Youtu-VL-4B-Instruct5.3B params	4.7 / 8.0 GB	Q4FitsQ8Tight
Gemma 4 E2B5.1B params	4.6 / 8.0 GB	Q4FitsQ8Tight
Molmo2-ER4.9B params	4.4 / 8.0 GB	Q4FitsQ8Fits
Apriel-5B-Instruct4.8B params	4.4 / 8.0 GB	Q4FitsQ8Fits
WeVisOne-VL-4B4.8B params	4.4 / 8.0 GB	Q4FitsQ8Fits
Mage-VL4.7B params	4.3 / 8.0 GB	Q4FitsQ8Fits
Mage-VL-Cowork4.7B params	4.3 / 8.0 GB	Q4FitsQ8Fits
Qwen3.5-4B4.7B params	4.2 / 8.0 GB	Q4FitsQ8Fits
S5-nano4.7B params	4.2 / 8.0 GB	Q4FitsQ8Fits
AFM-4.5B4.6B params	4.2 / 8.0 GB	Q4FitsQ8Fits
Llama-3.1-Minitron-4B-Width-Base4.5B params	4.1 / 8.0 GB	Q4FitsQ8Fits
Qwen3-VL-4B-Instruct4.4B params	4.1 / 8.0 GB	Q4FitsQ8Fits
CapRL-Qwen3VL-4B4.4B params	4.1 / 8.0 GB	Q4FitsQ8Fits
glm-edge-4b-chat4.3B params	4.0 / 8.0 GB	Q4FitsQ8Fits
gemma-3-4b-it4.3B params	4.0 / 8.0 GB	Q4FitsQ8Fits
medgemma-4b-it4.3B params	4.0 / 8.0 GB	Q4FitsQ8Fits
medgemma-1.5-4b-it4.3B params	4.0 / 8.0 GB	Q4FitsQ8Fits
gemma-3-4b-pt4.3B params	4.0 / 8.0 GB	Q4FitsQ8Fits
Nanbeige4.2-3B4.2B params	3.9 / 8.0 GB	Q4FitsQ8Fits
Phi-3.5-vision-instruct4.1B params	3.9 / 8.0 GB	Q4FitsQ8Fits
Spark-X2.6-4B4.1B params	3.9 / 8.0 GB	Q4FitsQ8Fits
Spark-X2.5-4B4.1B params	3.9 / 8.0 GB	Q4FitsQ8Fits
POINTS-Reader4.1B params	3.9 / 8.0 GB	Q4FitsQ8Fits
MiniCPM-V-44.1B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Qwen3-4B4.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Qwen3-4B-Instruct-25074.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Qwen3-4B-Thinking-25074.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Mellum-4b-base4.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-3b-vision4.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
granite-vision-4.1-4b4.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Nemotron Labs Dory 1.0 4B4.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
NVIDIA-Nemotron-3-Nano-4B-BF164.0B params	3.8 / 8.0 GB	Q4FitsQ8Fits
Nanbeige4.1-3B3.9B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Nanbeige4-3B-Thinking-25113.9B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Phi-4-mini-flash-reasoning3.9B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Phi-4-mini-instruct3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Phi-4-mini-reasoning3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Nemotron-Labs-Diffusion-3B3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
Phi-3.5-mini-instruct3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
MediPhi-Instruct3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
MediPhi3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
MediPhi-Clinical3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
MediPhi-PubMed3.8B params	3.7 / 8.0 GB	Q4FitsQ8Fits
HY-Embodied-0.53.8B params	3.6 / 8.0 GB	Q4FitsQ8Fits
Phi-tiny-MoE-instruct3.8B params	3.6 / 8.0 GB	Q4FitsQ8Fits
CapRL-3B3.8B params	3.6 / 8.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Vision-Instruct-3B3.7B params	3.6 / 8.0 GB	Q4FitsQ8Fits
kanana-1.5-v-3b-instruct3.7B params	3.6 / 8.0 GB	Q4FitsQ8Fits
granite-4.2-3b3.7B params	3.6 / 8.0 GB	Q4FitsQ8Fits
kanana-2-3b-instruct3.5B params	3.4 / 8.0 GB	Q4FitsQ8Fits
granite-4.1-3b3.4B params	3.4 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-micro3.4B params	3.4 / 8.0 GB	Q4FitsQ8Fits
granite-3.1-3b-a800m-instruct3.3B params	3.3 / 8.0 GB	Q4FitsQ8Fits
Falcon3-3B-Instruct3.2B params	3.3 / 8.0 GB	Q4FitsQ8Fits
Llama-3.2-3B-Instruct3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Llama-3.2-3B3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Hermes-3-Llama-3.2-3B3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
DeepHermes-3-Llama-3-3B-Preview3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
AI21-Jamba-Reasoning-3B3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-h-micro3.2B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Falcon-H1-3B-Instruct3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
d1-3B3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-VL-3B3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-3B-Instruct3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-3B-Instruct3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-3B3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-3B3.1B params	3.2 / 8.0 GB	Q4FitsQ8Fits
SmolLM3-3B3.1B params	3.1 / 8.0 GB	Q4FitsQ8Fits
AI21-Jamba2-3B3.0B params	3.1 / 8.0 GB	Q4FitsQ8Fits
granite-swash-3b-a600m3.0B params	3.1 / 8.0 GB	Q4FitsQ8Fits
granite-vision-3.2-2b3.0B params	3.1 / 8.0 GB	Q4FitsQ8Fits
granite-vision-3.1-2b-preview3.0B params	3.1 / 8.0 GB	Q4FitsQ8Fits
granite-vision-3.3-2b-chart2csv-preview3.0B params	3.1 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-2.6B2.7B params	2.9 / 8.0 GB	Q4FitsQ8Fits
Zamba2-2.7B2.7B params	2.9 / 8.0 GB	Q4FitsQ8Fits
Zamba2-2.7B-instruct2.7B params	2.9 / 8.0 GB	Q4FitsQ8Fits
granite-3.0-2b-instruct2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
gemma-2-2b-it2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
gemma-2-2b2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
EuroMoE-2.6B-A0.6B-Instruct-Preview2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
Motif-2.6B2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
LFM2-2.6B2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
LFM2-2.6B-Exp2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
LFM2-2.6B-Transcript2.6B params	2.8 / 8.0 GB	Q4FitsQ8Fits
granite-3.3-2b-instruct2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
granite-3.1-2b-instruct2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
granite-3.2-2b-instruct2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
MiniCPM5-2B2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
MiniCPM5-2B-SFT2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
MiniCPM5-2B-Midtrain2.5B params	2.8 / 8.0 GB	Q4FitsQ8Fits
sarvam-1-v0.52.5B params	2.7 / 8.0 GB	Q4FitsQ8Fits
North-Micro-Vision-Instruct2.5B params	2.7 / 8.0 GB	Q4FitsQ8Fits
UI-TARS-2B-SFT2.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
WeVisOne-VL-2B2.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
UI-Venus-1.5-2B2.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
bitnet-b1.58-2B-4T-bf162.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
EXAONE-3.5-2.4B-Instruct2.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
EXAONE-Deep-2.4B2.4B params	2.7 / 8.0 GB	Q4FitsQ8Fits
kanana-1.5-2.1b-instruct-25052.3B params	2.6 / 8.0 GB	Q4FitsQ8Fits
Qwen3.5-2B2.3B params	2.6 / 8.0 GB	Q4FitsQ8Fits
salamandra-2b2.3B params	2.6 / 8.0 GB	Q4FitsQ8Fits
salamandra-2b-instruct2.3B params	2.6 / 8.0 GB	Q4FitsQ8Fits
SmolVLM2-2.2B-Instruct2.2B params	2.6 / 8.0 GB	Q4FitsQ8Fits
SmolVLM-Base2.2B params	2.6 / 8.0 GB	Q4FitsQ8Fits
Qwen2-VL-2B-Instruct2.2B params	2.5 / 8.0 GB	Q4FitsQ8Fits
LFM3-Omni-2B2.2B params	2.5 / 8.0 GB	Q4FitsQ8Fits
granite-swash-2b2.1B params	2.5 / 8.0 GB	Q4FitsQ8Fits
Qwen3-VL-2B-Instruct2.1B params	2.5 / 8.0 GB	Q4FitsQ8Fits
kanana-nano-2.1b-instruct2.1B params	2.5 / 8.0 GB	Q4FitsQ8Fits
Hy-MT2-1.8B2.0B params	2.4 / 8.0 GB	Q4FitsQ8Fits
Qwen3-1.7B2.0B params	2.4 / 8.0 GB	Q4FitsQ8Fits
Youtu-LLM-2B2.0B params	2.4 / 8.0 GB	Q4FitsQ8Fits
LFM3-VL-2B1.9B params	2.3 / 8.0 GB	Q4FitsQ8Fits
d1-2B1.9B params	2.3 / 8.0 GB	Q4FitsQ8Fits
FastVLM-1.5B1.9B params	2.3 / 8.0 GB	Q4FitsQ8Fits
internlm2_5-1_8b-chat1.9B params	2.3 / 8.0 GB	Q4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-1.5B1.8B params	2.2 / 8.0 GB	Q4FitsQ8Fits
LFM3-1.7B1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
Qwen3-1.7B-Base1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
SmolLM2-1.7B1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
SmolLM-1.7B1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
Spark-X2.6-1.7B1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
Falcon3-1B-Instruct1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
EuroLLM-1.7B-Instruct1.7B params	2.2 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-1b1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-VL-1.6B1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
glm-edge-1.5b-chat1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
LFM2-VL-1.6B1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Falcon-H1-1.5B-Deep-Instruct1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Falcon-H1-1.5B-Instruct1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.7-1B1.6B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-1.5B-Instruct1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-1.5B1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-1.5B-Instruct1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-1.5B1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-1.5B1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2-1.5B-Instruct1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Math-1.5B-Instruct1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
ReaderLM-v21.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
reader-lm-1.5b1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
Hymba-1.5B-Instruct1.5B params	2.1 / 8.0 GB	Q4FitsQ8Fits
OLMo-2-0425-1B1.5B params	2.0 / 8.0 GB	Q4FitsQ8Fits
OLMo-2-0425-1B-Instruct1.5B params	2.0 / 8.0 GB	Q4FitsQ8Fits
Yi-Coder-1.5B-Chat1.5B params	2.0 / 8.0 GB	Q4FitsQ8Fits
Bolmo-1B1.5B params	2.0 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-h-1b1.5B params	2.0 / 8.0 GB	Q4FitsQ8Fits
granite-3.1-1b-a400m-instruct1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
granite-3.0-1b-a400m-instruct1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.61.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
MiniCPM-V-4.6-Thinking1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
kanana-2-1.3b-instruct1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
OLMo-1B-0724-hf1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
EXAONE-4.0-1.2B1.3B params	1.9 / 8.0 GB	Q4FitsQ8Fits
Llama-3.2-1B-Instruct1.2B params	1.9 / 8.0 GB	Q4FitsQ8Fits
Llama-3.2-1B1.2B params	1.9 / 8.0 GB	Q4FitsQ8Fits
Llama-3.2-1B1.2B params	1.9 / 8.0 GB	Q4FitsQ8Fits
Zamba2-1.2B-instruct1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
HRM-Text-1B1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
AMD-OLMo-1B1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-Instruct1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
LFM2-1.2B1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-Thinking1.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-1.2B-JP-2026061.2B params	1.8 / 8.0 GB	Q4FitsQ8Fits
MiniCPM5-1B1.1B params	1.8 / 8.0 GB	Q4FitsQ8Fits
MiniCPM5-1B-SFT1.1B params	1.8 / 8.0 GB	Q4FitsQ8Fits
gemma-3-1b-it1.0B params	1.7 / 8.0 GB	Q4FitsQ8Fits
gemma-3-1b-pt1.0B params	1.7 / 8.0 GB	Q4FitsQ8Fits
Qwen3.5-0.8B0.9B params	1.6 / 8.0 GB	Q4FitsQ8Fits
Falcon-E-3B-Instruct0.9B params	1.6 / 8.0 GB	Q4FitsQ8Fits
bitnet-b1.58-2B-4T0.8B params	1.6 / 8.0 GB	Q4FitsQ8Fits
FastVLM-0.5B0.8B params	1.5 / 8.0 GB	Q4FitsQ8Fits
Qwen3-0.6B0.8B params	1.5 / 8.0 GB	Q4FitsQ8Fits
LFM2-700M0.7B params	1.5 / 8.0 GB	Q4FitsQ8Fits
Falcon-H1-Tiny-R-0.6B0.6B params	1.4 / 8.0 GB	Q4FitsQ8Fits
Qwen3-0.6B-Base0.6B params	1.4 / 8.0 GB	Q4FitsQ8Fits
d1-omni-600M0.6B params	1.4 / 8.0 GB	Q4FitsQ8Fits
HyperCLOVAX-SEED-Text-Instruct-0.5B0.6B params	1.4 / 8.0 GB	Q4FitsQ8Fits
Falcon-H1-0.5B-Instruct0.5B params	1.4 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-0.5B-Instruct0.5B params	1.3 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-0.5B0.5B params	1.3 / 8.0 GB	Q4FitsQ8Fits
Qwen2-0.5B-Instruct0.5B params	1.3 / 8.0 GB	Q4FitsQ8Fits
Qwen2.5-Coder-0.5B-Instruct0.5B params	1.3 / 8.0 GB	Q4FitsQ8Fits
LFM2-VL-450M0.5B params	1.3 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-VL-450M0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
Apertus-v1.1-0.5B0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
MiniCPM4-0.5B0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
circuit-sparsity0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
SmolLM2-360M0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
SmolLM-360M0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
ERNIE-4.5-0.3B-PT0.4B params	1.3 / 8.0 GB	Q4FitsQ8Fits
LFM2.5-350M0.4B params	1.2 / 8.0 GB	Q4FitsQ8Fits
LFM2-350M0.4B params	1.2 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-350m0.4B params	1.2 / 8.0 GB	Q4FitsQ8Fits
granite-4.0-h-350m0.3B params	1.2 / 8.0 GB	Q4FitsQ8Fits

Quick answers

What is the biggest LLM that runs on GeForce RTX 4060 Ti 8GB?: NVIDIA-Nemotron-Nano-9B-v2 (8.9B parameters, about 7.2 GB at Q4) is the largest model we track that runs fully in its 8 GB of usable memory.
How many local LLMs can GeForce RTX 4060 Ti 8GB run?: 315 of the 656 open models we track run fully at Q4 on it.
How much memory can GeForce RTX 4060 Ti 8GB use for AI models?: It has 8 GB of VRAM, and we count 8 GB as usable for model weights and cache.

GeForce RTX 4060 Ti 8GB for local LLMs

Every tracked model on this machine

Quick answers

New models, and what they need to run