What Can I Run AI?

GeForce RTX 5070 Ti for local LLMs

The GeForce RTX 5070 Ti has 16 GB of VRAM (16 GB usable for models) and runs 388 of the 656 open models we track at Q4_K_M, up to Ling-Coder-lite (16.8B parameters, about 12.7 GB). These are estimates, not measured benchmarks.

Looking for the model list first? See what you can run on the GeForce RTX 5070 Ti.

Memory
16 GB VRAM
Memory bandwidth
896 GB/s
Usable for models
16 GB
Runs fully at Q4
388 / 656

Spec source · Best GPU under $800 · Best GPU for local LLMs · Multi-GPU · NVIDIA vs AMD vs Mac · Tokens per second · Best LLMs for 16 GB

$749 list price (MSRP), source, 2026-10-28
Street prices may be higher than list.

Check price on AmazonCheck price on eBay

We may earn a commission when you buy through these links, at no extra cost to you.

Compare: GeForce RTX 3090 vs GeForce RTX 5070 Ti · GeForce RTX 5070 Ti vs Radeon RX 9070 XT

Can I run this model on GeForce RTX 5070 Ti?

Answer-first pages for popular models on this machine. Each one is a short verdict, not the full model table.

Every tracked model on this machine

Sorted largest first. The tick marks 90% of usable memory.FitsTightWon’t fit
ModelQ4 need vs. 16 GBVerdict
Kimi-K3.1-Pro2780.1B params1939.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K32779.9B params1939.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-2.4T-A95B2446.2B params1706.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-2.51779.8B params1241.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-2.01775.6B params1238.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro-08131650.5B params1151.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Pro1598.8B params1115.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.51026.9B params716.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.61026.9B params716.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2.7-Code1026.9B params716.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Thinking1026.4B params716.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Instruct-09051026.4B params716.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kimi-K2-Instruct1026.4B params716.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-RL1024.2B params715.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Pro-MOPD1024.2B params715.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.5-Pro1023.2B params714.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ring-2.5-1T1012.2B params706.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-2.5-1T1012.2B params706.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-1T999.7B params698.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ring-1T999.7B params698.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Inkling952.4B params665.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Intern-S1-Pro917.0B params640.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Muse-Spark-Open-816B-A42B822.0B params574.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hy4-preview780.0B params544.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5753.9B params526.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.1753.9B params526.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3753.3B params526.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.2753.3B params526.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3-BF16753.3B params526.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
K-EXAONE-2.0-750B-A37B749.4B params523.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Exp685.4B params478.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2-Speciale685.4B params478.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.2685.4B params478.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3-0324684.5B params478.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-0528684.5B params478.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V3.1-Base684.5B params478.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1684.5B params478.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-Zero684.5B params478.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Step-5-Preview604.3B params422.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.1-flash572.1B params399.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.1-flash-1019572.1B params399.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Chat561.9B params392.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking561.9B params392.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Thinking-2601561.9B params392.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16560.5B params391.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Beam-1.0-501B-A23B501.3B params350.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Coder-480B-A35B-Instruct480.2B params335.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M1-40k456.1B params319.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M1-80k456.1B params319.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M3427.0B params298.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-424B-A47B-PT423.5B params296.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Intern-S2-397B403.4B params282.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.5-397B-A17B403.4B params282.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-4-Maverick-17B-128E-Instruct401.6B params281.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Thinking398.6B params278.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Preview398.6B params278.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-TrueBase398.6B params278.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Large-Base398.6B params278.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2.5-Pro396.8B params277.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2-Pro396.8B params277.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
NorthCUA-Pro396.8B params277.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.5358.3B params250.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.7358.3B params250.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.6356.8B params249.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-5.3-Flash321.3B params225.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Motif-3314.8B params220.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.5310.8B params217.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-RL310.8B params217.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2.6-Flash-MOPD310.8B params217.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiMo-V2-Flash309.8B params217.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-Vision-Exp304.6B params213.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash-0731304.2B params213.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-300B-A47B-PT300.5B params210.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hy3298.8B params209.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hy3-preview298.8B params209.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-plus292.5B params205.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V4-Flash290.9B params203.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
MemT-1.0-Flash287.9B params201.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-5.5-Flash-Base286.2B params200.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Inkling-Small266.0B params186.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-Ultra-253B-v1253.4B params177.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Solar-Open2-250B250.3B params175.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Intern-S1240.7B params168.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
K-EXAONE-236B-A23B237.1B params166.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2.5235.7B params165.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct235.7B params165.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2.5-1210235.7B params165.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-V2-Chat-0628235.7B params165.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-Coder-V2-Instruct-0724235.7B params165.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-VL-235B-A22B-Instruct235.7B params165.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B235.1B params164.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Instruct-2507235.1B params164.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-235B-A22B-Thinking-2507235.1B params164.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2228.7B params160.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.5228.7B params160.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.7228.7B params160.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniMax-M2.1228.7B params160.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
command-a-plus-05-2026-bf16218.8B params153.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Step-3.7-Flash201.4B params141.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Step-3.5-Flash199.4B params140.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Step-3.5-Flash-Base-Midtrain197.8B params138.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-Flash-Next180.0B params126.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Mixtral-8x22B-Instruct-v0.1140.6B params99.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.0-flash-Sante127.5B params89.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.0-flash127.5B params89.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.0-flash-Fin127.5B params89.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ornith-1.5-122B125.1B params88.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.5-122B-A10B125.1B params88.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Devstral-2-123B-Instruct-2512125.0B params88.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-3.0-flash-VL124.8B params88.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Super-120B-A12B-BF16123.6B params87.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Mistral-Small-4-119B-2603119.4B params84.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Youtu-LLM-A7B-Base118.4B params83.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Laguna-S-2.1117.6B params83.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
gpt-oss-120b116.8B params82.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
c4ai-command-a-03-2025111.1B params78.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.5-Air110.5B params78.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E-Instruct108.6B params76.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-4-Scout-17B-16E108.6B params76.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.5V107.7B params76.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.6V107.7B params76.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-2.6-flash107.5B params75.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
sarvam-105b106.0B params74.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ling-flash-2.0102.9B params72.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LLaDA2.1-flash102.9B params72.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ring-flash-2.0102.9B params72.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LLaDA2.2-flash102.9B params72.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Solar-Open-100B102.7B params72.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Instruct81.3B params57.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Next-80B-A3B-Thinking81.3B params57.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
AliceAI-Foundation-80B-A3B-Base81.3B params57.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hunyuan-A13B-Instruct80.4B params57.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Coder-Next79.7B params56.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Kolibri-178.1B params55.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-VL-72B-Instruct73.4B params52.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
UI-TARS-72B-DPO73.4B params52.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-72B-Instruct72.7B params51.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Apertus-70B-Instruct-250970.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Apertus-70B-250970.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3.3-70B-Instruct70.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Llama-70B70.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3.1-70B70.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hermes-2-Theta-Llama-3-70B70.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Meta-Llama-3.1-70B-Instruct70.6B params50.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
LongCat-Flash-Lite69.1B params49.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
AI21-Jamba2-Mini51.6B params37.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3_1-Nemotron-51B-Instruct51.5B params36.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v1_549.9B params35.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Llama-3_3-Nemotron-Super-49B-v149.9B params35.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Phi-3.5-MoE-instruct41.9B params30.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
GRIN-MoE41.9B params30.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
ALIA-40b40.4B params29.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Loop-Instruct39.8B params28.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
IQuest-Coder-V1-40B-Instruct39.8B params28.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Recursive Flint39.8B params28.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
K2-Horizon-MoVA-36B-A4B37.4B params27.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Seed-OSS-36B-Instruct36.2B params26.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Seed-OSS-36B-Base-woSyn36.2B params26.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Intern-S2-Preview36.1B params26.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Ornith-1.5-35B-A3B36.0B params26.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Sage-A1.536.0B params26.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.6-35B-A3B36.0B params26.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.5-35B-A3B36.0B params26.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Apex-35B-V1.036.0B params26.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
MiniCPM-V-4.7-35B-A3B35.2B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
MemT-1.0-Nano35.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Agents-A135.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2.5-mini35.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nex-N2-mini35.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Axion-1.0-35B35.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
NorthCUA-Mini35.1B params25.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Clover-35B-A3B-Preview34.7B params25.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen-AgentWorld-35B-A3B34.7B params25.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
KAT-Coder-V2.5-Dev34.7B params25.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
EXAONE-4.5-33B34.4B params24.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Falcon-H1-34B-Instruct33.6B params24.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-VL-32B-Instruct33.5B params24.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Laguna-XS.233.4B params24.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-VL-32B-Instruct33.4B params24.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
HyperCLOVAX-SEED-Think-32B33.3B params24.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
FlexOlmo-7x7B-1T33.3B params24.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-Z1-Rumination-32B-041433.1B params24.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nanbeige4.3-33B33.1B params24.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF1633.0B params24.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-Coder-32B-Instruct32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
DeepSeek-R1-Distill-Qwen-32B32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-32B-Instruct32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
QwQ-32B32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
QwQ-32B-Preview32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-32B32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen2.5-Coder-32B32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-32B32.8B params23.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-Z1-32B-041432.6B params23.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4-32B-041432.6B params23.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
OLMo-2-0325-32B-Instruct32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
OLMo-2-0325-32B32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Olmo-3-32B-Think32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Olmo-3-1125-32B32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Instruct32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Think32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
Olmo-3.1-32B-Instruct-DPO32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
granite-4.0-h-small32.2B params23.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
sarvam-30b32.2B params23.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
EXAONE-4.0-32B32.0B params23.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
EXAONE-4.0.1-32B32.0B params23.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
EXAONE-3.5-32B-Instruct32.0B params23.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
EXAONE-Deep-32B32.0B params23.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF1631.6B params23.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
NVIDIA-Nemotron-3-Nano-30B-A3B-BF1631.6B params23.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Nemotron-Cascade-2-30B-A3B31.6B params23.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Gemma 4 31B31.3B params22.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
gemma-4-31B31.3B params22.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
GLM-4.7-Flash31.2B params22.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Xing4.0-29B-A4B31.2B params22.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Xing4.0-29B-Intent31.2B params22.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-VL-30B-A3B-Instruct31.1B params22.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-VL-30B-A3B-Thinking31.1B params22.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
UI-Venus-1.5-30B-A3B31.1B params22.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-instruct-260130.7B params22.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-instruct30.7B params22.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
kanana-2-30b-a3b-thinking-260130.7B params22.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-30B-A3B30.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-Coder-30B-A3B-Instruct30.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-30B-A3B-Instruct-250730.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3-30B-A3B-Thinking-250730.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Tongyi-DeepResearch-30B-A3B30.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Rubicon-Preview30.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
North-Mini-Code-1.030.5B params22.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hy-Memory-30B-A3B30.1B params22.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hy-MT2-30B-A3B30.1B params22.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
granite-switch-4.1-30b30.0B params21.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Muse-Glimmer-30B29.8B params21.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-Thinking29.7B params21.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-VL-28B-A3B-PT29.4B params21.5 / 16.0 GBQ4Won’t fitQ8Won’t fit
granite-4.2-30b29.3B params21.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
granite-4.1-30b28.9B params21.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Raon-1 mini preview28.8B params21.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-27B27.8B params20.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Tess-4.5-27B27.8B params20.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.6-27B27.8B params20.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.5-27B27.8B params20.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
gemma-3-27b-it27.4B params20.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
Arete-27B27.4B params20.1 / 16.0 GBQ4Won’t fitQ8Won’t fit
gemma-2-27b-it27.2B params20.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
gemma-2-27b27.2B params20.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Qwen3.8-27B-Longbook27-v227.1B params19.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
medgemma-27b-text-it27.0B params19.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Hemmingway-1.526.9B params19.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
Trinity-Mini26.1B params19.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
diffusiongemma-26B-A4B-it25.8B params19.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Gemma 4 26B A4B25.8B params19.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
gemma-4-26B-A4B25.8B params19.0 / 16.0 GBQ4Won’t fitQ8Won’t fit
Voxtral-Small-24B-250724.3B params17.9 / 16.0 GBQ4Won’t fitQ8Won’t fit
Mistral-Small-3.2-24B-Instruct-250624.0B params17.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
Mistral-Small-3.1-24B-Instruct-250324.0B params17.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LFM2.5-24B-A2B24.0B params17.7 / 16.0 GBQ4Won’t fitQ8Won’t fit
LFM2-24B-A2B23.8B params17.6 / 16.0 GBQ4Won’t fitQ8Won’t fit
Mistral-Small-24B-Instruct-250123.6B params17.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
EuroLLM-22B-Instruct-251222.6B params16.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
EuroLLM-22B-251222.6B params16.8 / 16.0 GBQ4Won’t fitQ8Won’t fit
solar-pro-preview-instruct22.1B params16.4 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-21B-A3B-PT21.9B params16.3 / 16.0 GBQ4Won’t fitQ8Won’t fit
ERNIE-4.5-21B-A3B-Thinking21.8B params16.2 / 16.0 GBQ4Won’t fitQ8Won’t fit
gpt-oss-20b20.9B params15.6 / 16.0 GBQ4TightQ8Won’t fit
reka-flash-320.9B params15.6 / 16.0 GBQ4TightQ8Won’t fit
maple-preview20.2B params15.1 / 16.0 GBQ4TightQ8Won’t fit
internlm2_5-20b-chat19.9B params14.8 / 16.0 GBQ4TightQ8Won’t fit
Groot-19B-A3B-26102319.4B params14.5 / 16.0 GBQ4TightQ8Won’t fit
Ling-Coder-lite16.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ring-lite-distill-preview16.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ling-lite16.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ling-lite-1.5-250716.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ling-lite-1.516.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ring-lite16.8B params12.7 / 16.0 GBQ4FitsQ8Won’t fit
Ring-mini-linear-2.016.4B params12.5 / 16.0 GBQ4FitsQ8Won’t fit
Kimi-VL-A3B-Instruct16.4B params12.4 / 16.0 GBQ4FitsQ8Won’t fit
Kimi-VL-A3B-Thinking16.4B params12.4 / 16.0 GBQ4FitsQ8Won’t fit
Kimi-VL-A3B-Thinking-250616.4B params12.4 / 16.0 GBQ4FitsQ8Won’t fit
LLaDA2.1-mini16.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
LLaDA2.0-mini16.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
Ling-mini-2.016.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
Ring-mini-2.016.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
LLaDA2.2-mini16.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
LLaDA2.0-mini-preview16.3B params12.3 / 16.0 GBQ4FitsQ8Won’t fit
Moonlight-16B-A3B-Instruct16.0B params12.1 / 16.0 GBQ4FitsQ8Won’t fit
Moonlight-16B-A3B16.0B params12.1 / 16.0 GBQ4FitsQ8Won’t fit
Instella-MoE-16B-A3B-Think15.9B params12.1 / 16.0 GBQ4FitsQ8Won’t fit
DeepSeek-Coder-V2-Lite-Instruct15.7B params12.0 / 16.0 GBQ4FitsQ8Won’t fit
kanana-1.5-15.7b-a3b-instruct15.7B params11.9 / 16.0 GBQ4FitsQ8Won’t fit
Phi-4-reasoning-vision-15B15.1B params11.5 / 16.0 GBQ4FitsQ8Won’t fit
Apriel-Nemotron-15b-Thinker15.0B params11.4 / 16.0 GBQ4FitsQ8Won’t fit
Apriel-1.5-15b-Thinker14.9B params11.4 / 16.0 GBQ4FitsQ8Won’t fit
Apriel-1.6-15b-Thinker14.9B params11.4 / 16.0 GBQ4FitsQ8Won’t fit
Carbon-X (DNA)14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen2.5-Coder-14B-Instruct14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
DeepSeek-R1-Distill-Qwen-14B14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen2.5-14B-Instruct14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen2.5-14B14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen2.5-14B-Instruct-1M14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen2.5-Coder-14B14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
Qwen3-14B14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
JanusCoder-14B14.8B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
HyperCLOVAX-SEED-Think-14B14.7B params11.3 / 16.0 GBQ4FitsQ8Won’t fit
phi-414.7B params11.2 / 16.0 GBQ4FitsQ8Won’t fit
Phi-4-reasoning14.7B params11.2 / 16.0 GBQ4FitsQ8Won’t fit
Phi-4-reasoning-plus14.7B params11.2 / 16.0 GBQ4FitsQ8Won’t fit
OLMo-2-1124-13B-Instruct13.7B params10.6 / 16.0 GBQ4FitsQ8Won’t fit
Mistral-Nemo-Instruct-240712.2B params9.5 / 16.0 GBQ4FitsQ8Tight
gemma-3-12b-it12.2B params9.5 / 16.0 GBQ4FitsQ8Tight
gemma-3-12b-pt12.2B params9.5 / 16.0 GBQ4FitsQ8Tight
Mellum2.1-12B-A2.5B-Thinking12.1B params9.5 / 16.0 GBQ4FitsQ8Tight
Mellum2-12B-A2.5B-Thinking12.1B params9.5 / 16.0 GBQ4FitsQ8Tight
Mellum2-12B-A2.5B-Instruct12.1B params9.5 / 16.0 GBQ4FitsQ8Tight
Mellum2-12B-A2.5B-Thinking-SFT12.1B params9.5 / 16.0 GBQ4FitsQ8Tight
Mellum2.1-12B-A2.5B-Thinking-MTP12.1B params9.5 / 16.0 GBQ4FitsQ8Tight
Gemma 4 12B12.0B params9.3 / 16.0 GBQ4FitsQ8Tight
HyperCLOVAX-SEED-Omni-8B10.7B params8.5 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-11B-Vision-Instruct10.7B params8.4 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-11B-Vision10.6B params8.4 / 16.0 GBQ4FitsQ8Fits
Falcon3-10B-Instruct10.3B params8.2 / 16.0 GBQ4FitsQ8Fits
GLM-4.1V-9B-Thinking10.3B params8.2 / 16.0 GBQ4FitsQ8Fits
GLM-4.6V-Flash10.3B params8.2 / 16.0 GBQ4FitsQ8Fits
GLM-4.1V-9B-Base10.3B params8.2 / 16.0 GBQ4FitsQ8Fits
Glyph10.3B params8.2 / 16.0 GBQ4FitsQ8Fits
INTELLECT-1-Instruct10.2B params8.1 / 16.0 GBQ4FitsQ8Fits
INTELLECT-110.2B params8.1 / 16.0 GBQ4FitsQ8Fits
ZAYA1-VL-8B9.7B params7.8 / 16.0 GBQ4FitsQ8Fits
SenseNova-Loop-9B-Preview9.7B params7.7 / 16.0 GBQ4FitsQ8Fits
Ornith-1.5-9B9.7B params7.7 / 16.0 GBQ4FitsQ8Fits
Qwen3.5-9B9.7B params7.7 / 16.0 GBQ4FitsQ8Fits
MiniCPM-SALA9.5B params7.6 / 16.0 GBQ4FitsQ8Fits
NorthCUA-Nano9.4B params7.6 / 16.0 GBQ4FitsQ8Fits
GLM-4-9B-04149.4B params7.6 / 16.0 GBQ4FitsQ8Fits
GLM-Z1-9B-04149.4B params7.6 / 16.0 GBQ4FitsQ8Fits
codegeex4-all-9b9.4B params7.6 / 16.0 GBQ4FitsQ8Fits
glm-4-9b-chat-hf9.4B params7.6 / 16.0 GBQ4FitsQ8Fits
moondream3-preview9.3B params7.5 / 16.0 GBQ4FitsQ8Fits
gemma-2-9b-it9.2B params7.4 / 16.0 GBQ4FitsQ8Fits
gemma-2-9b9.2B params7.4 / 16.0 GBQ4FitsQ8Fits
Gemma-SEA-LION-v3-9B-IT9.2B params7.4 / 16.0 GBQ4FitsQ8Fits
EuroLLM-9B-Instruct-25129.2B params7.4 / 16.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-Nano-9B-v28.9B params7.2 / 16.0 GBQ4FitsQ8Fits
ZAYA1-base8.8B params7.2 / 16.0 GBQ4FitsQ8Fits
Yi-Coder-9B-Chat8.8B params7.2 / 16.0 GBQ4FitsQ8Fits
Yi-Coder-9B8.8B params7.2 / 16.0 GBQ4FitsQ8Fits
internlm3-8b-instruct8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
granite-4.1-8b8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
granite-4.2-8b8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
granite-switch-4.1-8b8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
Qwen3-VL-8B-Instruct8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
Qwen3-VL-8B-Thinking8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
UI-Venus-1.5-8B8.8B params7.1 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Nemotron-Nano-VL-8B-V18.7B params7.1 / 16.0 GBQ4FitsQ8Fits
MiniCPM-V-4_58.7B params7.1 / 16.0 GBQ4FitsQ8Fits
aya-vision-8b8.6B params7.0 / 16.0 GBQ4FitsQ8Fits
Intern-S1-mini8.5B params7.0 / 16.0 GBQ4FitsQ8Fits
Nemotron-Labs-Diffusion-8B-Base8.5B params6.9 / 16.0 GBQ4FitsQ8Fits
LFM2.5-8B-A1B8.5B params6.9 / 16.0 GBQ4FitsQ8Fits
Mistral-NeMo-Minitron-8B-Base8.4B params6.9 / 16.0 GBQ4FitsQ8Fits
LFM2-8B-A1B8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
rnj-1-instruct8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
MiMo-VL-7B-RL-25088.3B params6.8 / 16.0 GBQ4FitsQ8Fits
MiMo-VL-7B-RL8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
MiMo-Embodied-7B8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
MiMo-VL-7B-SFT8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-VL-7B-Instruct8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
UI-TARS-1.5-7B8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Fara-7B8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
UI-Venus-Ground-7B8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Qwen2-VL-7B-Instruct8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
UI-TARS-7B-DPO8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
UI-TARS-7B-SFT8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Seed-Coder-8B-Instruct8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Seed-Coder-8B-Reasoning8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Stable-DiffCoder-8B-Instruct8.3B params6.8 / 16.0 GBQ4FitsQ8Fits
Qwen3-8B8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
DeepSeek-R1-0528-Qwen3-8B8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
Qwen3-8B-Base8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
WeDLM-8B-Instruct8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
PaCoRe-8B8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
MiniCPM4-8B8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
MiniCPM4.1-8B8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
AgentCPM-Report8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
MiniCPM4-MCP8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
granite-3.3-8b-instruct8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
granite-3.1-8b-instruct8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
granite-3.2-8b-instruct8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
granite-3.0-8b-instruct8.2B params6.7 / 16.0 GBQ4FitsQ8Fits
Apertus-8B-Instruct-25098.1B params6.6 / 16.0 GBQ4FitsQ8Fits
Apertus-8B-25098.1B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3-8B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3-8B-DPO8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Tulu-3.1-8B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
kanana-1.5-8b-instruct-25058.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-8B-Instruct8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-8B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Llama-8B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Meta-Llama-3.1-8B-Instruct8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
L3-8B-Stheno-v3.28.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Nemotron-Nano-8B-v18.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Meta-Llama-3.1-8B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
LongWriter-llama3.1-8b8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Hy-MT2-7B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Nemotron 3.5 Bolt E4B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Gemma 4 E4B8.0B params6.6 / 16.0 GBQ4FitsQ8Fits
Ling-3.0-tiny7.9B params6.5 / 16.0 GBQ4FitsQ8Fits
MiMo-7B-RL7.8B params6.5 / 16.0 GBQ4FitsQ8Fits
MiMo-7B-Base7.8B params6.5 / 16.0 GBQ4FitsQ8Fits
MiMo-7B-RL-05307.8B params6.5 / 16.0 GBQ4FitsQ8Fits
MiMo-7B-SFT7.8B params6.5 / 16.0 GBQ4FitsQ8Fits
EXAONE-3.5-7.8B-Instruct7.8B params6.5 / 16.0 GBQ4FitsQ8Fits
EXAONE-Deep-7.8B7.8B params6.5 / 16.0 GBQ4FitsQ8Fits
salamandra-7b-instruct7.8B params6.4 / 16.0 GBQ4FitsQ8Fits
salamandra-7b7.8B params6.4 / 16.0 GBQ4FitsQ8Fits
salamandra-7b-instruct-26067.8B params6.4 / 16.0 GBQ4FitsQ8Fits
FastVLM-7B7.8B params6.4 / 16.0 GBQ4FitsQ8Fits
Molmo2-O-7B7.8B params6.4 / 16.0 GBQ4FitsQ8Fits
internlm2_5-7b-chat7.7B params6.4 / 16.0 GBQ4FitsQ8Fits
internlm2_5-7b-chat-1m7.7B params6.4 / 16.0 GBQ4FitsQ8Fits
Phi-mini-MoE-instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-7B-Instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-7B-Instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2-7B-Instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-7B-Instruct-1M7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Math-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Math-7B-Instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Marco-o17.6B params6.3 / 16.0 GBQ4FitsQ8Fits
SeaLLMs-v3-7B-Chat7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Arcee-Spark7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-7B-Instruct7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
Falcon-H1R-7B7.6B params6.3 / 16.0 GBQ4FitsQ8Fits
starvector-8b-im2svg7.5B params6.2 / 16.0 GBQ4FitsQ8Fits
Falcon3-7B-Instruct7.5B params6.2 / 16.0 GBQ4FitsQ8Fits
Olmo-Hybrid-7B7.4B params6.2 / 16.0 GBQ4FitsQ8Fits
LLaDA-MoE-7B-A1B-Base7.4B params6.1 / 16.0 GBQ4FitsQ8Fits
Zamba2-7B-Instruct7.4B params6.1 / 16.0 GBQ4FitsQ8Fits
OLMo-2-1124-7B-Instruct7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-1025-7B7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct-SFT7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think-SFT7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Instruct-DPO7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-RL-Zero-Math7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3.1-7B-RL-Zero-Math7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Olmo-3-7B-Think-DPO7.3B params6.1 / 16.0 GBQ4FitsQ8Fits
Mistral-7B-Instruct-v0.37.2B params6.1 / 16.0 GBQ4FitsQ8Fits
granite-4.0-h-tiny6.9B params5.8 / 16.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-0924-Instruct6.9B params5.8 / 16.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-09246.9B params5.8 / 16.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-0125-Instruct6.9B params5.8 / 16.0 GBQ4FitsQ8Fits
OLMoE-1B-7B-01256.9B params5.8 / 16.0 GBQ4FitsQ8Fits
OLMo-7B-0724-hf6.9B params5.8 / 16.0 GBQ4FitsQ8Fits
granite-4.0-tiny-preview6.7B params5.7 / 16.0 GBQ4FitsQ8Fits
YuE-s1-7B-anneal-en-cot6.2B params5.3 / 16.0 GBQ4FitsQ8Fits
Trinity-Nano-Preview6.1B params5.3 / 16.0 GBQ4FitsQ8Fits
Trinity-Nano-Base6.1B params5.3 / 16.0 GBQ4FitsQ8Fits
gemma-3n-E2B-it5.4B params4.8 / 16.0 GBQ4FitsQ8Fits
Youtu-VL-4B-Instruct5.3B params4.7 / 16.0 GBQ4FitsQ8Fits
Gemma 4 E2B5.1B params4.6 / 16.0 GBQ4FitsQ8Fits
Molmo2-ER4.9B params4.4 / 16.0 GBQ4FitsQ8Fits
Apriel-5B-Instruct4.8B params4.4 / 16.0 GBQ4FitsQ8Fits
WeVisOne-VL-4B4.8B params4.4 / 16.0 GBQ4FitsQ8Fits
Mage-VL4.7B params4.3 / 16.0 GBQ4FitsQ8Fits
Mage-VL-Cowork4.7B params4.3 / 16.0 GBQ4FitsQ8Fits
Qwen3.5-4B4.7B params4.2 / 16.0 GBQ4FitsQ8Fits
S5-nano4.7B params4.2 / 16.0 GBQ4FitsQ8Fits
AFM-4.5B4.6B params4.2 / 16.0 GBQ4FitsQ8Fits
Llama-3.1-Minitron-4B-Width-Base4.5B params4.1 / 16.0 GBQ4FitsQ8Fits
Qwen3-VL-4B-Instruct4.4B params4.1 / 16.0 GBQ4FitsQ8Fits
CapRL-Qwen3VL-4B4.4B params4.1 / 16.0 GBQ4FitsQ8Fits
glm-edge-4b-chat4.3B params4.0 / 16.0 GBQ4FitsQ8Fits
gemma-3-4b-it4.3B params4.0 / 16.0 GBQ4FitsQ8Fits
medgemma-4b-it4.3B params4.0 / 16.0 GBQ4FitsQ8Fits
medgemma-1.5-4b-it4.3B params4.0 / 16.0 GBQ4FitsQ8Fits
gemma-3-4b-pt4.3B params4.0 / 16.0 GBQ4FitsQ8Fits
Nanbeige4.2-3B4.2B params3.9 / 16.0 GBQ4FitsQ8Fits
Phi-3.5-vision-instruct4.1B params3.9 / 16.0 GBQ4FitsQ8Fits
Spark-X2.6-4B4.1B params3.9 / 16.0 GBQ4FitsQ8Fits
Spark-X2.5-4B4.1B params3.9 / 16.0 GBQ4FitsQ8Fits
POINTS-Reader4.1B params3.9 / 16.0 GBQ4FitsQ8Fits
MiniCPM-V-44.1B params3.8 / 16.0 GBQ4FitsQ8Fits
Qwen3-4B4.0B params3.8 / 16.0 GBQ4FitsQ8Fits
Qwen3-4B-Instruct-25074.0B params3.8 / 16.0 GBQ4FitsQ8Fits
Qwen3-4B-Thinking-25074.0B params3.8 / 16.0 GBQ4FitsQ8Fits
Mellum-4b-base4.0B params3.8 / 16.0 GBQ4FitsQ8Fits
granite-4.0-3b-vision4.0B params3.8 / 16.0 GBQ4FitsQ8Fits
granite-vision-4.1-4b4.0B params3.8 / 16.0 GBQ4FitsQ8Fits
Nemotron Labs Dory 1.0 4B4.0B params3.8 / 16.0 GBQ4FitsQ8Fits
NVIDIA-Nemotron-3-Nano-4B-BF164.0B params3.8 / 16.0 GBQ4FitsQ8Fits
Nanbeige4.1-3B3.9B params3.7 / 16.0 GBQ4FitsQ8Fits
Nanbeige4-3B-Thinking-25113.9B params3.7 / 16.0 GBQ4FitsQ8Fits
Phi-4-mini-flash-reasoning3.9B params3.7 / 16.0 GBQ4FitsQ8Fits
Phi-4-mini-instruct3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
Phi-4-mini-reasoning3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
Nemotron-Labs-Diffusion-3B3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
Phi-3.5-mini-instruct3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
MediPhi-Instruct3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
MediPhi3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
MediPhi-Clinical3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
MediPhi-PubMed3.8B params3.7 / 16.0 GBQ4FitsQ8Fits
HY-Embodied-0.53.8B params3.6 / 16.0 GBQ4FitsQ8Fits
Phi-tiny-MoE-instruct3.8B params3.6 / 16.0 GBQ4FitsQ8Fits
CapRL-3B3.8B params3.6 / 16.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Vision-Instruct-3B3.7B params3.6 / 16.0 GBQ4FitsQ8Fits
kanana-1.5-v-3b-instruct3.7B params3.6 / 16.0 GBQ4FitsQ8Fits
granite-4.2-3b3.7B params3.6 / 16.0 GBQ4FitsQ8Fits
kanana-2-3b-instruct3.5B params3.4 / 16.0 GBQ4FitsQ8Fits
granite-4.1-3b3.4B params3.4 / 16.0 GBQ4FitsQ8Fits
granite-4.0-micro3.4B params3.4 / 16.0 GBQ4FitsQ8Fits
granite-3.1-3b-a800m-instruct3.3B params3.3 / 16.0 GBQ4FitsQ8Fits
Falcon3-3B-Instruct3.2B params3.3 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-3B-Instruct3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-3B3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
Hermes-3-Llama-3.2-3B3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
DeepHermes-3-Llama-3-3B-Preview3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
AI21-Jamba-Reasoning-3B3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
granite-4.0-h-micro3.2B params3.2 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-3B-Instruct3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
d1-3B3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
LFM2.5-VL-3B3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-3B-Instruct3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-3B-Instruct3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-3B3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-3B3.1B params3.2 / 16.0 GBQ4FitsQ8Fits
SmolLM3-3B3.1B params3.1 / 16.0 GBQ4FitsQ8Fits
AI21-Jamba2-3B3.0B params3.1 / 16.0 GBQ4FitsQ8Fits
granite-swash-3b-a600m3.0B params3.1 / 16.0 GBQ4FitsQ8Fits
granite-vision-3.2-2b3.0B params3.1 / 16.0 GBQ4FitsQ8Fits
granite-vision-3.1-2b-preview3.0B params3.1 / 16.0 GBQ4FitsQ8Fits
granite-vision-3.3-2b-chart2csv-preview3.0B params3.1 / 16.0 GBQ4FitsQ8Fits
LFM2.5-2.6B2.7B params2.9 / 16.0 GBQ4FitsQ8Fits
Zamba2-2.7B2.7B params2.9 / 16.0 GBQ4FitsQ8Fits
Zamba2-2.7B-instruct2.7B params2.9 / 16.0 GBQ4FitsQ8Fits
granite-3.0-2b-instruct2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
gemma-2-2b-it2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
gemma-2-2b2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
EuroMoE-2.6B-A0.6B-Instruct-Preview2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
Motif-2.6B2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
LFM2-2.6B2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
LFM2-2.6B-Exp2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
LFM2-2.6B-Transcript2.6B params2.8 / 16.0 GBQ4FitsQ8Fits
granite-3.3-2b-instruct2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
granite-3.1-2b-instruct2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
granite-3.2-2b-instruct2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
MiniCPM5-2B2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
MiniCPM5-2B-SFT2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
MiniCPM5-2B-Midtrain2.5B params2.8 / 16.0 GBQ4FitsQ8Fits
sarvam-1-v0.52.5B params2.7 / 16.0 GBQ4FitsQ8Fits
North-Micro-Vision-Instruct2.5B params2.7 / 16.0 GBQ4FitsQ8Fits
UI-TARS-2B-SFT2.4B params2.7 / 16.0 GBQ4FitsQ8Fits
WeVisOne-VL-2B2.4B params2.7 / 16.0 GBQ4FitsQ8Fits
UI-Venus-1.5-2B2.4B params2.7 / 16.0 GBQ4FitsQ8Fits
bitnet-b1.58-2B-4T-bf162.4B params2.7 / 16.0 GBQ4FitsQ8Fits
EXAONE-3.5-2.4B-Instruct2.4B params2.7 / 16.0 GBQ4FitsQ8Fits
EXAONE-Deep-2.4B2.4B params2.7 / 16.0 GBQ4FitsQ8Fits
kanana-1.5-2.1b-instruct-25052.3B params2.6 / 16.0 GBQ4FitsQ8Fits
Qwen3.5-2B2.3B params2.6 / 16.0 GBQ4FitsQ8Fits
salamandra-2b2.3B params2.6 / 16.0 GBQ4FitsQ8Fits
salamandra-2b-instruct2.3B params2.6 / 16.0 GBQ4FitsQ8Fits
SmolVLM2-2.2B-Instruct2.2B params2.6 / 16.0 GBQ4FitsQ8Fits
SmolVLM-Base2.2B params2.6 / 16.0 GBQ4FitsQ8Fits
Qwen2-VL-2B-Instruct2.2B params2.5 / 16.0 GBQ4FitsQ8Fits
LFM3-Omni-2B2.2B params2.5 / 16.0 GBQ4FitsQ8Fits
granite-swash-2b2.1B params2.5 / 16.0 GBQ4FitsQ8Fits
Qwen3-VL-2B-Instruct2.1B params2.5 / 16.0 GBQ4FitsQ8Fits
kanana-nano-2.1b-instruct2.1B params2.5 / 16.0 GBQ4FitsQ8Fits
Hy-MT2-1.8B2.0B params2.4 / 16.0 GBQ4FitsQ8Fits
Qwen3-1.7B2.0B params2.4 / 16.0 GBQ4FitsQ8Fits
Youtu-LLM-2B2.0B params2.4 / 16.0 GBQ4FitsQ8Fits
LFM3-VL-2B1.9B params2.3 / 16.0 GBQ4FitsQ8Fits
d1-2B1.9B params2.3 / 16.0 GBQ4FitsQ8Fits
FastVLM-1.5B1.9B params2.3 / 16.0 GBQ4FitsQ8Fits
internlm2_5-1_8b-chat1.9B params2.3 / 16.0 GBQ4FitsQ8Fits
DeepSeek-R1-Distill-Qwen-1.5B1.8B params2.2 / 16.0 GBQ4FitsQ8Fits
LFM3-1.7B1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
Qwen3-1.7B-Base1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
SmolLM2-1.7B1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
SmolLM-1.7B1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
Spark-X2.6-1.7B1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
Falcon3-1B-Instruct1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
EuroLLM-1.7B-Instruct1.7B params2.2 / 16.0 GBQ4FitsQ8Fits
granite-4.0-1b1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
LFM2.5-VL-1.6B1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
glm-edge-1.5b-chat1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
LFM2-VL-1.6B1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-1.5B-Deep-Instruct1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-1.5B-Instruct1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
MiniCPM-V-4.7-1B1.6B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-1.5B-Instruct1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-1.5B1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-1.5B-Instruct1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-1.5B1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Math-1.5B1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2-1.5B-Instruct1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Math-1.5B-Instruct1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
ReaderLM-v21.5B params2.1 / 16.0 GBQ4FitsQ8Fits
reader-lm-1.5b1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
Hymba-1.5B-Instruct1.5B params2.1 / 16.0 GBQ4FitsQ8Fits
OLMo-2-0425-1B1.5B params2.0 / 16.0 GBQ4FitsQ8Fits
OLMo-2-0425-1B-Instruct1.5B params2.0 / 16.0 GBQ4FitsQ8Fits
Yi-Coder-1.5B-Chat1.5B params2.0 / 16.0 GBQ4FitsQ8Fits
Bolmo-1B1.5B params2.0 / 16.0 GBQ4FitsQ8Fits
granite-4.0-h-1b1.5B params2.0 / 16.0 GBQ4FitsQ8Fits
granite-3.1-1b-a400m-instruct1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
granite-3.0-1b-a400m-instruct1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
MiniCPM-V-4.61.3B params1.9 / 16.0 GBQ4FitsQ8Fits
MiniCPM-V-4.6-Thinking1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
kanana-2-1.3b-instruct1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
OLMo-1B-0724-hf1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
EXAONE-4.0-1.2B1.3B params1.9 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-1B-Instruct1.2B params1.9 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-1B1.2B params1.9 / 16.0 GBQ4FitsQ8Fits
Llama-3.2-1B1.2B params1.9 / 16.0 GBQ4FitsQ8Fits
Zamba2-1.2B-instruct1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
HRM-Text-1B1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
AMD-OLMo-1B1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-Instruct1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
LFM2-1.2B1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-Thinking1.2B params1.8 / 16.0 GBQ4FitsQ8Fits
LFM2.5-1.2B-JP-2026061.2B params1.8 / 16.0 GBQ4FitsQ8Fits
MiniCPM5-1B1.1B params1.8 / 16.0 GBQ4FitsQ8Fits
MiniCPM5-1B-SFT1.1B params1.8 / 16.0 GBQ4FitsQ8Fits
gemma-3-1b-it1.0B params1.7 / 16.0 GBQ4FitsQ8Fits
gemma-3-1b-pt1.0B params1.7 / 16.0 GBQ4FitsQ8Fits
Qwen3.5-0.8B0.9B params1.6 / 16.0 GBQ4FitsQ8Fits
Falcon-E-3B-Instruct0.9B params1.6 / 16.0 GBQ4FitsQ8Fits
bitnet-b1.58-2B-4T0.8B params1.6 / 16.0 GBQ4FitsQ8Fits
FastVLM-0.5B0.8B params1.5 / 16.0 GBQ4FitsQ8Fits
Qwen3-0.6B0.8B params1.5 / 16.0 GBQ4FitsQ8Fits
LFM2-700M0.7B params1.5 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-Tiny-R-0.6B0.6B params1.4 / 16.0 GBQ4FitsQ8Fits
Qwen3-0.6B-Base0.6B params1.4 / 16.0 GBQ4FitsQ8Fits
d1-omni-600M0.6B params1.4 / 16.0 GBQ4FitsQ8Fits
HyperCLOVAX-SEED-Text-Instruct-0.5B0.6B params1.4 / 16.0 GBQ4FitsQ8Fits
Falcon-H1-0.5B-Instruct0.5B params1.4 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-0.5B-Instruct0.5B params1.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-0.5B0.5B params1.3 / 16.0 GBQ4FitsQ8Fits
Qwen2-0.5B-Instruct0.5B params1.3 / 16.0 GBQ4FitsQ8Fits
Qwen2.5-Coder-0.5B-Instruct0.5B params1.3 / 16.0 GBQ4FitsQ8Fits
LFM2-VL-450M0.5B params1.3 / 16.0 GBQ4FitsQ8Fits
LFM2.5-VL-450M0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
Apertus-v1.1-0.5B0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
MiniCPM4-0.5B0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
circuit-sparsity0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
SmolLM2-360M0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
SmolLM-360M0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
ERNIE-4.5-0.3B-PT0.4B params1.3 / 16.0 GBQ4FitsQ8Fits
LFM2.5-350M0.4B params1.2 / 16.0 GBQ4FitsQ8Fits
LFM2-350M0.4B params1.2 / 16.0 GBQ4FitsQ8Fits
granite-4.0-350m0.4B params1.2 / 16.0 GBQ4FitsQ8Fits
granite-4.0-h-350m0.3B params1.2 / 16.0 GBQ4FitsQ8Fits

Quick answers

What is the biggest LLM that runs on GeForce RTX 5070 Ti?
Ling-Coder-lite (16.8B parameters, about 12.7 GB at Q4) is the largest model we track that runs fully in its 16 GB of usable memory.
How many local LLMs can GeForce RTX 5070 Ti run?
388 of the 656 open models we track run fully at Q4 on it.
How much memory can GeForce RTX 5070 Ti use for AI models?
It has 16 GB of VRAM, and we count 16 GB as usable for model weights and cache.

New models, and what they need to run

One short email a week: fresh open models with their VRAM numbers, plus hardware worth watching.

Or subscribe here