# Best local LLMs for 6 GB VRAM | What Can I Run AI? > 198 open models run fully in 6 GB of VRAM at Q4, up to YuE-s1-7B-anneal-en-cot. See every model that fits, with memory needs. Source: https://whatcanirunai.vercel.app/best-llm-for/6gb-vram/ # Best local LLMs for 6 GB of VRAM Every open model we track that runs fully in 6 GB of GPU memory, biggest first. Checks are at Q4\_K\_M with about 4k context. Run fully at Q4 198 of 656 Run at Q8 too 145 Biggest at Q4 YuE-s1-7B-anneal-en-cot With 6 GB, the biggest model you can run comfortably is **YuE-s1-7B-anneal-en-cot** (\~5.3 GB at Q4). If you want higher quality at Q8, **kanana-2-3b-instruct** is the biggest that fits. ## Models that fit in 6 GB Bar: Q4 memory needed vs. 6 GB. The tick marks 90%, our comfort line.FitsTight | Model | Q4 need vs. 6 GB | Q4 / Q8 | | ----------------------------------------------------------------------------------------------- | ---------------- | ----------------- | | [Gemma 4 E2B](https://whatcanirunai.vercel.app/models/gemma-4-e2b-it/)5.1B params, google | **4.6** / 6.0 GB | Q4FitsQ8Won’t fit | | [Qwen3.5-4B](https://whatcanirunai.vercel.app/models/qwen3.5-4b/)4.7B params, Qwen | **4.2** / 6.0 GB | Q4FitsQ8Won’t fit | | [Qwen3-VL-4B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-vl-4b-instruct/)4.4B params, Qwen | **4.1** / 6.0 GB | Q4FitsQ8Won’t fit | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)4.3B params, google | **4.0** / 6.0 GB | Q4FitsQ8Won’t fit | | [medgemma-4b-it](https://whatcanirunai.vercel.app/models/medgemma-4b-it/)4.3B params, google | **4.0** / 6.0 GB | Q4FitsQ8Won’t fit | | [medgemma-1.5-4b-it](https://whatcanirunai.vercel.app/models/medgemma-1.5-4b-it/)4.3B params, google | **4.0** / 6.0 GB | Q4FitsQ8Won’t fit | | [Nanbeige4.2-3B](https://whatcanirunai.vercel.app/models/nanbeige4.2-3b/)4.2B params, Nanbeige | **3.9** / 6.0 GB | Q4FitsQ8Won’t fit | | [Phi-3.5-vision-instruct](https://whatcanirunai.vercel.app/models/phi-3.5-vision-instruct/)4.1B params, microsoft | **3.9** / 6.0 GB | Q4FitsQ8Won’t fit | | [Spark-X2.6-4B](https://whatcanirunai.vercel.app/models/spark-x2.6-4b/)4.1B params, XHToken | **3.9** / 6.0 GB | Q4FitsQ8Won’t fit | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)4.0B params, Qwen | **3.8** / 6.0 GB | Q4FitsQ8Tight | | [Qwen3-4B-Instruct-2507](https://whatcanirunai.vercel.app/models/qwen3-4b-instruct-2507/)4.0B params, Qwen | **3.8** / 6.0 GB | Q4FitsQ8Tight | | [Qwen3-4B-Thinking-2507](https://whatcanirunai.vercel.app/models/qwen3-4b-thinking-2507/)4.0B params, Qwen | **3.8** / 6.0 GB | Q4FitsQ8Tight | | [Nemotron Labs Dory 1.0 4B](https://whatcanirunai.vercel.app/models/nemotron-labs-dory1.0-4b/)4.0B params, nvidia | **3.8** / 6.0 GB | Q4FitsQ8Tight | | [NVIDIA-Nemotron-3-Nano-4B-BF16](https://whatcanirunai.vercel.app/models/nvidia-nemotron-3-nano-4b-bf16/)4.0B params, nvidia | **3.8** / 6.0 GB | Q4FitsQ8Tight | | [Phi-4-mini-instruct](https://whatcanirunai.vercel.app/models/phi-4-mini-instruct/)3.8B params, microsoft | **3.7** / 6.0 GB | Q4FitsQ8Tight | | [Phi-3.5-mini-instruct](https://whatcanirunai.vercel.app/models/phi-3.5-mini-instruct/)3.8B params, microsoft | **3.7** / 6.0 GB | Q4FitsQ8Tight | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)3.2B params, meta-llama | **3.2** / 6.0 GB | Q4FitsQ8Fits | | [Llama-3.2-3B](https://whatcanirunai.vercel.app/models/llama-3.2-3b/)3.2B params, meta-llama | **3.2** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2.5-3B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-3b-instruct/)3.1B params, Qwen | **3.2** / 6.0 GB | Q4FitsQ8Fits | | [SmolLM3-3B](https://whatcanirunai.vercel.app/models/smollm3-3b/)3.1B params, HuggingFaceTB | **3.1** / 6.0 GB | Q4FitsQ8Fits | | [gemma-2-2b-it](https://whatcanirunai.vercel.app/models/gemma-2-2b-it/)2.6B params, google | **2.8** / 6.0 GB | Q4FitsQ8Fits | | [MiniCPM5-2B](https://whatcanirunai.vercel.app/models/minicpm5-2b/)2.5B params, openbmb | **2.8** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3.5-2B](https://whatcanirunai.vercel.app/models/qwen3.5-2b/)2.3B params, Qwen | **2.6** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2-VL-2B-Instruct](https://whatcanirunai.vercel.app/models/qwen2-vl-2b-instruct/)2.2B params, Qwen | **2.5** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3-VL-2B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-vl-2b-instruct/)2.1B params, Qwen | **2.5** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3-1.7B](https://whatcanirunai.vercel.app/models/qwen3-1.7b/)2.0B params, Qwen | **2.4** / 6.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-1.5B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-1.5b/)1.8B params, deepseek-ai | **2.2** / 6.0 GB | Q4FitsQ8Fits | | [LFM3-1.7B](https://whatcanirunai.vercel.app/models/lfm3-1.7b/)1.7B params, LiquidAI | **2.2** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2.5-1.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-1.5b-instruct/)1.5B params, Qwen | **2.1** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2.5-1.5B](https://whatcanirunai.vercel.app/models/qwen2.5-1.5b/)1.5B params, Qwen | **2.1** / 6.0 GB | Q4FitsQ8Fits | | [MiniCPM-V-4.6](https://whatcanirunai.vercel.app/models/minicpm-v-4.6/)1.3B params, openbmb | **1.9** / 6.0 GB | Q4FitsQ8Fits | | [Llama-3.2-1B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-1b-instruct/)1.2B params, meta-llama | **1.9** / 6.0 GB | Q4FitsQ8Fits | | [Llama-3.2-1B](https://whatcanirunai.vercel.app/models/llama-3.2-1b/)1.2B params, meta-llama | **1.9** / 6.0 GB | Q4FitsQ8Fits | | [MiniCPM5-1B](https://whatcanirunai.vercel.app/models/minicpm5-1b/)1.1B params, openbmb | **1.8** / 6.0 GB | Q4FitsQ8Fits | | [gemma-3-1b-it](https://whatcanirunai.vercel.app/models/gemma-3-1b-it/)1.0B params, google | **1.7** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3.5-0.8B](https://whatcanirunai.vercel.app/models/qwen3.5-0.8b/)0.9B params, Qwen | **1.6** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3-0.6B](https://whatcanirunai.vercel.app/models/qwen3-0.6b/)0.8B params, Qwen | **1.5** / 6.0 GB | Q4FitsQ8Fits | | [Qwen3-0.6B-Base](https://whatcanirunai.vercel.app/models/qwen3-0.6b-base/)0.6B params, Qwen | **1.4** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2.5-0.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-0.5b-instruct/)0.5B params, Qwen | **1.3** / 6.0 GB | Q4FitsQ8Fits | | [Qwen2.5-0.5B](https://whatcanirunai.vercel.app/models/qwen2.5-0.5b/)0.5B params, Qwen | **1.3** / 6.0 GB | Q4FitsQ8Fits | Showing the 40 most popular of 205 models that fit. [See all 205 in the model directory →](https://whatcanirunai.vercel.app/models/?gb=6) ## Hardware we track with 6–7 GB usable * [GeForce RTX 2060 6GB](https://whatcanirunai.vercel.app/what-can-i-run/rtx-2060/)6 GB usable, $349 Macs and unified-memory mini-PCs count about 75% of memory as usable ([how we calculate](https://whatcanirunai.vercel.app/method/)). ## Quick answers What is the biggest LLM I can run with 6 GB of VRAM? YuE-s1-7B-anneal-en-cot (6.2B parameters) is the biggest model we track that runs comfortably in 6 GB at Q4\_K\_M, needing about 5.3 GB with a \~4k context. At Q8 the biggest is kanana-2-3b-instruct. How many models fit in 6 GB of VRAM? 198 of the 656 open models we track run fully in 6 GB at Q4 with room to spare, 145 also fit at Q8, and 7 more fit only with a short context. Can I run a model that needs more than 6 GB? Yes, with llama.cpp or Ollama you can offload some layers to system RAM, but speed drops sharply. For full-speed local inference the whole model plus its context cache should fit in GPU memory; we use 90% of 6 GB as the comfort line. ## Other memory sizes [4 GB](https://whatcanirunai.vercel.app/best-llm-for/4gb-vram/) · [8 GB](https://whatcanirunai.vercel.app/best-llm-for/8gb-vram/) · [10 GB](https://whatcanirunai.vercel.app/best-llm-for/10gb-vram/) · [12 GB](https://whatcanirunai.vercel.app/best-llm-for/12gb-vram/) · [16 GB](https://whatcanirunai.vercel.app/best-llm-for/16gb-vram/) · [20 GB](https://whatcanirunai.vercel.app/best-llm-for/20gb-vram/) · [24 GB](https://whatcanirunai.vercel.app/best-llm-for/24gb-vram/) · [32 GB](https://whatcanirunai.vercel.app/best-llm-for/32gb-vram/) · [48 GB](https://whatcanirunai.vercel.app/best-llm-for/48gb-vram/) · [64 GB](https://whatcanirunai.vercel.app/best-llm-for/64gb-vram/) · [96 GB](https://whatcanirunai.vercel.app/best-llm-for/96gb-vram/) · [128 GB](https://whatcanirunai.vercel.app/best-llm-for/128gb-vram/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/