Best local LLMs for 4 GB of VRAM
Every open model we track that runs fully in 4 GB of GPU memory, biggest first. Checks are at Q4_K_M with about 4k context.
- Run fully at Q4
- 148 of 656
- Run at Q8 too
- 87
- Biggest at Q4
- HyperCLOVAX-SEED-Vision-Instruct-3B
With 4 GB, the biggest model you can run comfortably is HyperCLOVAX-SEED-Vision-Instruct-3B (~3.6 GB at Q4). If you want higher quality at Q8, Qwen3-VL-2B-Instruct is the biggest that fits.
Models that fit in 4 GB
Bar: Q4 memory needed vs. 4 GB. The tick marks 90%, our comfort line.FitsTight
| Model | Q4 need vs. 4 GB | Q4 / Q8 |
|---|---|---|
| Llama-3.2-3B-Instruct3.2B params, meta-llama | 3.2 / 4.0 GB | Q4FitsQ8Won’t fit |
| Llama-3.2-3B3.2B params, meta-llama | 3.2 / 4.0 GB | Q4FitsQ8Won’t fit |
| Qwen2.5-3B-Instruct3.1B params, Qwen | 3.2 / 4.0 GB | Q4FitsQ8Won’t fit |
| SmolLM3-3B3.1B params, HuggingFaceTB | 3.1 / 4.0 GB | Q4FitsQ8Won’t fit |
| LFM2.5-2.6B2.7B params, LiquidAI | 2.9 / 4.0 GB | Q4FitsQ8Won’t fit |
| gemma-2-2b-it2.6B params, google | 2.8 / 4.0 GB | Q4FitsQ8Won’t fit |
| gemma-2-2b2.6B params, google | 2.8 / 4.0 GB | Q4FitsQ8Won’t fit |
| MiniCPM5-2B2.5B params, openbmb | 2.8 / 4.0 GB | Q4FitsQ8Won’t fit |
| Qwen3.5-2B2.3B params, Qwen | 2.6 / 4.0 GB | Q4FitsQ8Tight |
| Qwen2-VL-2B-Instruct2.2B params, Qwen | 2.5 / 4.0 GB | Q4FitsQ8Tight |
| Qwen3-VL-2B-Instruct2.1B params, Qwen | 2.5 / 4.0 GB | Q4FitsQ8Fits |
| Qwen3-1.7B2.0B params, Qwen | 2.4 / 4.0 GB | Q4FitsQ8Fits |
| DeepSeek-R1-Distill-Qwen-1.5B1.8B params, deepseek-ai | 2.2 / 4.0 GB | Q4FitsQ8Fits |
| LFM3-1.7B1.7B params, LiquidAI | 2.2 / 4.0 GB | Q4FitsQ8Fits |
| Qwen2.5-1.5B-Instruct1.5B params, Qwen | 2.1 / 4.0 GB | Q4FitsQ8Fits |
| Qwen2.5-1.5B1.5B params, Qwen | 2.1 / 4.0 GB | Q4FitsQ8Fits |
| MiniCPM-V-4.61.3B params, openbmb | 1.9 / 4.0 GB | Q4FitsQ8Fits |
| Llama-3.2-1B-Instruct1.2B params, meta-llama | 1.9 / 4.0 GB | Q4FitsQ8Fits |
| Llama-3.2-1B1.2B params, meta-llama | 1.9 / 4.0 GB | Q4FitsQ8Fits |
| LFM2.5-1.2B-Instruct1.2B params, LiquidAI | 1.8 / 4.0 GB | Q4FitsQ8Fits |
| MiniCPM5-1B1.1B params, openbmb | 1.8 / 4.0 GB | Q4FitsQ8Fits |
| gemma-3-1b-it1.0B params, google | 1.7 / 4.0 GB | Q4FitsQ8Fits |
| Qwen3.5-0.8B0.9B params, Qwen | 1.6 / 4.0 GB | Q4FitsQ8Fits |
| Qwen3-0.6B0.8B params, Qwen | 1.5 / 4.0 GB | Q4FitsQ8Fits |
| Qwen3-0.6B-Base0.6B params, Qwen | 1.4 / 4.0 GB | Q4FitsQ8Fits |
| Qwen2.5-0.5B-Instruct0.5B params, Qwen | 1.3 / 4.0 GB | Q4FitsQ8Fits |
| Qwen2.5-0.5B0.5B params, Qwen | 1.3 / 4.0 GB | Q4FitsQ8Fits |
| gemma-3-4b-it4.3B params, google | 4.0 / 4.0 GB | Q4TightQ8Won’t fit |
| medgemma-4b-it4.3B params, google | 4.0 / 4.0 GB | Q4TightQ8Won’t fit |
| medgemma-1.5-4b-it4.3B params, google | 4.0 / 4.0 GB | Q4TightQ8Won’t fit |
| Nanbeige4.2-3B4.2B params, Nanbeige | 3.9 / 4.0 GB | Q4TightQ8Won’t fit |
| Phi-3.5-vision-instruct4.1B params, microsoft | 3.9 / 4.0 GB | Q4TightQ8Won’t fit |
| Spark-X2.6-4B4.1B params, XHToken | 3.9 / 4.0 GB | Q4TightQ8Won’t fit |
| Qwen3-4B4.0B params, Qwen | 3.8 / 4.0 GB | Q4TightQ8Won’t fit |
| Qwen3-4B-Instruct-25074.0B params, Qwen | 3.8 / 4.0 GB | Q4TightQ8Won’t fit |
| Qwen3-4B-Thinking-25074.0B params, Qwen | 3.8 / 4.0 GB | Q4TightQ8Won’t fit |
| Nemotron Labs Dory 1.0 4B4.0B params, nvidia | 3.8 / 4.0 GB | Q4TightQ8Won’t fit |
| NVIDIA-Nemotron-3-Nano-4B-BF164.0B params, nvidia | 3.8 / 4.0 GB | Q4TightQ8Won’t fit |
| Phi-4-mini-instruct3.8B params, microsoft | 3.7 / 4.0 GB | Q4TightQ8Won’t fit |
| Phi-3.5-mini-instruct3.8B params, microsoft | 3.7 / 4.0 GB | Q4TightQ8Won’t fit |
Quick answers
- What is the biggest LLM I can run with 4 GB of VRAM?
- HyperCLOVAX-SEED-Vision-Instruct-3B (3.7B parameters) is the biggest model we track that runs comfortably in 4 GB at Q4_K_M, needing about 3.6 GB with a ~4k context. At Q8 the biggest is Qwen3-VL-2B-Instruct.
- How many models fit in 4 GB of VRAM?
- 148 of the 656 open models we track run fully in 4 GB at Q4 with room to spare, 87 also fit at Q8, and 32 more fit only with a short context.
- Can I run a model that needs more than 4 GB?
- Yes, with llama.cpp or Ollama you can offload some layers to system RAM, but speed drops sharply. For full-speed local inference the whole model plus its context cache should fit in GPU memory; we use 90% of 4 GB as the comfort line.
Other memory sizes
6 GB · 8 GB · 10 GB · 12 GB · 16 GB · 20 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB