# Best local LLMs for 12 GB VRAM | What Can I Run AI? > 350 open models run fully in 12 GB of VRAM at Q4, up to OLMo-2-1124-13B-Instruct. See every model that fits, with memory needs. Source: https://whatcanirunai.vercel.app/best-llm-for/12gb-vram/ # Best local LLMs for 12 GB of VRAM Every open model we track that runs fully in 12 GB of GPU memory, biggest first. Checks are at Q4\_K\_M with about 4k context. Run fully at Q4 350 of 656 Run at Q8 too 253 Biggest at Q4 OLMo-2-1124-13B-Instruct With 12 GB, the biggest model you can run comfortably is **OLMo-2-1124-13B-Instruct** (\~10.6 GB at Q4). If you want higher quality at Q8, **Nemotron 3.5 Bolt E4B** is the biggest that fits. ## Models that fit in 12 GB Bar: Q4 memory needed vs. 12 GB. The tick marks 90%, our comfort line.FitsTight | Model | Q4 need vs. 12 GB | Q4 / Q8 | | ---------------------------------------------------------------------------------------------- | ------------------ | ------------------ | | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)13.7B params, allenai | **10.6** / 12.0 GB | Q4FitsQ8Won’t fit | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)12.2B params, mistralai | **9.5** / 12.0 GB | Q4FitsQ8Won’t fit | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/)12.2B params, google | **9.5** / 12.0 GB | Q4FitsQ8Won’t fit | | [Mellum2.1-12B-A2.5B-Thinking](https://whatcanirunai.vercel.app/models/mellum2.1-12b-a2.5b-thinking/)12.1B params, JetBrains | **9.5** / 12.0 GB | Q4FitsQ8Won’t fit | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/)12.0B params, google | **9.3** / 12.0 GB | Q4FitsQ8Won’t fit | | [Ornith-1.5-9B](https://whatcanirunai.vercel.app/models/ornith-1.5-9b/)9.7B params, ornith-ai | **7.7** / 12.0 GB | Q4FitsQ8Won’t fit | | [Qwen3.5-9B](https://whatcanirunai.vercel.app/models/qwen3.5-9b/)9.7B params, Qwen | **7.7** / 12.0 GB | Q4FitsQ8Won’t fit | | [Qwen3-VL-8B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-vl-8b-instruct/)8.8B params, Qwen | **7.1** / 12.0 GB | Q4FitsQ8Tight | | [Qwen2.5-VL-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-vl-7b-instruct/)8.3B params, Qwen | **6.8** / 12.0 GB | Q4FitsQ8Tight | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)8.2B params, Qwen | **6.7** / 12.0 GB | Q4FitsQ8Tight | | [granite-3.3-8b-instruct](https://whatcanirunai.vercel.app/models/granite-3.3-8b-instruct/)8.2B params, ibm-granite | **6.7** / 12.0 GB | Q4FitsQ8Tight | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)8.0B params, meta-llama | **6.6** / 12.0 GB | Q4FitsQ8Tight | | [Nemotron 3.5 Bolt E4B](https://whatcanirunai.vercel.app/models/nemotron-3.5-bolt-e4b/)8.0B params, nvidia | **6.6** / 12.0 GB | Q4FitsQ8Fits | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/)8.0B params, google | **6.6** / 12.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-7b-instruct/)7.6B params, Qwen | **6.3** / 12.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-7B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-7b/)7.6B params, deepseek-ai | **6.3** / 12.0 GB | Q4FitsQ8Fits | | [Qwen2.5-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-7b-instruct/)7.6B params, Qwen | **6.3** / 12.0 GB | Q4FitsQ8Fits | | [Mistral-7B-Instruct-v0.3](https://whatcanirunai.vercel.app/models/mistral-7b-instruct-v0.3/)7.2B params, mistralai | **6.1** / 12.0 GB | Q4FitsQ8Fits | | [Gemma 4 E2B](https://whatcanirunai.vercel.app/models/gemma-4-e2b-it/)5.1B params, google | **4.6** / 12.0 GB | Q4FitsQ8Fits | | [Qwen3.5-4B](https://whatcanirunai.vercel.app/models/qwen3.5-4b/)4.7B params, Qwen | **4.2** / 12.0 GB | Q4FitsQ8Fits | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)4.3B params, google | **4.0** / 12.0 GB | Q4FitsQ8Fits | | [Spark-X2.6-4B](https://whatcanirunai.vercel.app/models/spark-x2.6-4b/)4.1B params, XHToken | **3.9** / 12.0 GB | Q4FitsQ8Fits | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)4.0B params, Qwen | **3.8** / 12.0 GB | Q4FitsQ8Fits | | [Nemotron Labs Dory 1.0 4B](https://whatcanirunai.vercel.app/models/nemotron-labs-dory1.0-4b/)4.0B params, nvidia | **3.8** / 12.0 GB | Q4FitsQ8Fits | | [Phi-4-mini-instruct](https://whatcanirunai.vercel.app/models/phi-4-mini-instruct/)3.8B params, microsoft | **3.7** / 12.0 GB | Q4FitsQ8Fits | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)3.2B params, meta-llama | **3.2** / 12.0 GB | Q4FitsQ8Fits | | [MiniCPM5-2B](https://whatcanirunai.vercel.app/models/minicpm5-2b/)2.5B params, openbmb | **2.8** / 12.0 GB | Q4FitsQ8Fits | | [Qwen3-1.7B](https://whatcanirunai.vercel.app/models/qwen3-1.7b/)2.0B params, Qwen | **2.4** / 12.0 GB | Q4FitsQ8Fits | | [LFM3-1.7B](https://whatcanirunai.vercel.app/models/lfm3-1.7b/)1.7B params, LiquidAI | **2.2** / 12.0 GB | Q4FitsQ8Fits | | [Qwen2.5-1.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-1.5b-instruct/)1.5B params, Qwen | **2.1** / 12.0 GB | Q4FitsQ8Fits | | [Llama-3.2-1B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-1b-instruct/)1.2B params, meta-llama | **1.9** / 12.0 GB | Q4FitsQ8Fits | | [gemma-3-1b-it](https://whatcanirunai.vercel.app/models/gemma-3-1b-it/)1.0B params, google | **1.7** / 12.0 GB | Q4FitsQ8Fits | | [Qwen3.5-0.8B](https://whatcanirunai.vercel.app/models/qwen3.5-0.8b/)0.9B params, Qwen | **1.6** / 12.0 GB | Q4FitsQ8Fits | | [Qwen3-0.6B](https://whatcanirunai.vercel.app/models/qwen3-0.6b/)0.8B params, Qwen | **1.5** / 12.0 GB | Q4FitsQ8Fits | | [Qwen2.5-0.5B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-0.5b-instruct/)0.5B params, Qwen | **1.3** / 12.0 GB | Q4FitsQ8Fits | | [Carbon-X (DNA)](https://whatcanirunai.vercel.app/models/carbon-x/)14.8B params, HuggingFaceBio | **11.3** / 12.0 GB | Q4TightQ8Won’t fit | | [Qwen2.5-Coder-14B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-14b-instruct/)14.8B params, Qwen | **11.3** / 12.0 GB | Q4TightQ8Won’t fit | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/)14.8B params, deepseek-ai | **11.3** / 12.0 GB | Q4TightQ8Won’t fit | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)14.8B params, Qwen | **11.3** / 12.0 GB | Q4TightQ8Won’t fit | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/)14.7B params, microsoft | **11.2** / 12.0 GB | Q4TightQ8Won’t fit | Showing the 40 most popular of 369 models that fit. [See all 369 in the model directory →](https://whatcanirunai.vercel.app/models/?gb=12) ## Hardware we track with 12–15 GB usable * [Intel Arc B580](https://whatcanirunai.vercel.app/what-can-i-run/arc-b580/)12 GB usable, $249 * [GeForce RTX 3060 12GB](https://whatcanirunai.vercel.app/what-can-i-run/rtx-3060-12gb/)12 GB usable, $329 * [Radeon RX 7700 XT](https://whatcanirunai.vercel.app/what-can-i-run/rx-7700-xt/)12 GB usable, $449 * [Radeon RX 6700 XT](https://whatcanirunai.vercel.app/what-can-i-run/rx-6700-xt/)12 GB usable, $479 * [GeForce RTX 5070](https://whatcanirunai.vercel.app/what-can-i-run/rtx-5070/)12 GB usable, $549 * [Radeon RX 6750 XT](https://whatcanirunai.vercel.app/what-can-i-run/rx-6750-xt/)12 GB usable, $549 * [GeForce RTX 4070](https://whatcanirunai.vercel.app/what-can-i-run/rtx-4070/)12 GB usable, $599 * [GeForce RTX 4070 Super](https://whatcanirunai.vercel.app/what-can-i-run/rtx-4070-super/)12 GB usable, $599 * [Mac mini M4 16GB](https://whatcanirunai.vercel.app/what-can-i-run/mac-mini-m4-16/)12 GB usable, $789 * [GeForce RTX 3080 12GB](https://whatcanirunai.vercel.app/what-can-i-run/rtx-3080-12gb/)12 GB usable, $799 * [GeForce RTX 4070 Ti](https://whatcanirunai.vercel.app/what-can-i-run/rtx-4070-ti/)12 GB usable, $799 * [Mac mini M6 16GB](https://whatcanirunai.vercel.app/what-can-i-run/mac-mini-m6-16/)12 GB usable, $899 * [GeForce RTX 3080 Ti](https://whatcanirunai.vercel.app/what-can-i-run/rtx-3080-ti/)12 GB usable, $1199 * [GeForce RTX 2060 12GB](https://whatcanirunai.vercel.app/what-can-i-run/rtx-2060-12gb/)12 GB usable Macs and unified-memory mini-PCs count about 75% of memory as usable ([how we calculate](https://whatcanirunai.vercel.app/method/)). ## Quick answers What is the biggest LLM I can run with 12 GB of VRAM? OLMo-2-1124-13B-Instruct (13.7B parameters) is the biggest model we track that runs comfortably in 12 GB at Q4\_K\_M, needing about 10.6 GB with a \~4k context. At Q8 the biggest is Nemotron 3.5 Bolt E4B. How many models fit in 12 GB of VRAM? 350 of the 656 open models we track run fully in 12 GB at Q4 with room to spare, 253 also fit at Q8, and 19 more fit only with a short context. Can I run a model that needs more than 12 GB? Yes, with llama.cpp or Ollama you can offload some layers to system RAM, but speed drops sharply. For full-speed local inference the whole model plus its context cache should fit in GPU memory; we use 90% of 12 GB as the comfort line. Which hardware has about 12 GB of usable memory? The cheapest current device we track in this range is the Intel Arc B580 (12 GB usable, $249 list price as of our last snapshot). 14 tracked devices have 12–15 GB usable. ## Other memory sizes [4 GB](https://whatcanirunai.vercel.app/best-llm-for/4gb-vram/) · [6 GB](https://whatcanirunai.vercel.app/best-llm-for/6gb-vram/) · [8 GB](https://whatcanirunai.vercel.app/best-llm-for/8gb-vram/) · [10 GB](https://whatcanirunai.vercel.app/best-llm-for/10gb-vram/) · [16 GB](https://whatcanirunai.vercel.app/best-llm-for/16gb-vram/) · [20 GB](https://whatcanirunai.vercel.app/best-llm-for/20gb-vram/) · [24 GB](https://whatcanirunai.vercel.app/best-llm-for/24gb-vram/) · [32 GB](https://whatcanirunai.vercel.app/best-llm-for/32gb-vram/) · [48 GB](https://whatcanirunai.vercel.app/best-llm-for/48gb-vram/) · [64 GB](https://whatcanirunai.vercel.app/best-llm-for/64gb-vram/) · [96 GB](https://whatcanirunai.vercel.app/best-llm-for/96gb-vram/) · [128 GB](https://whatcanirunai.vercel.app/best-llm-for/128gb-vram/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/