# Best local LLMs for 128 GB VRAM | What Can I Run AI? > 554 open models run fully in 128 GB of VRAM at Q4, up to Mixtral-8x22B-Instruct-v0.1. See every model that fits, with memory needs. Source: https://whatcanirunai.vercel.app/best-llm-for/128gb-vram/ # Best local LLMs for 128 GB of VRAM Every open model we track that runs fully in 128 GB of GPU memory, biggest first. Checks are at Q4\_K\_M with about 4k context. Run fully at Q4 554 of 656 Run at Q8 too 528 Biggest at Q4 Mixtral-8x22B-Instruct-v0.1 With 128 GB, the biggest model you can run comfortably is **Mixtral-8x22B-Instruct-v0.1** (\~99.0 GB at Q4). If you want higher quality at Q8, **Qwen3-Next-80B-A3B-Instruct** is the biggest that fits. ## Models that fit in 128 GB Bar: Q4 memory needed vs. 128 GB. The tick marks 90%, our comfort line.FitsTight | Model | Q4 need vs. 128 GB | Q4 / Q8 | | --------------------------------------------------------------------------------------------------------------------- | -------------------- | ------------------ | | [Ling-3.0-flash-Sante](https://whatcanirunai.vercel.app/models/ling-3.0-flash-sante/)127.5B params, inclusionAI | **89.9** / 128.0 GB | Q4FitsQ8Won’t fit | | [Ornith-1.5-122B](https://whatcanirunai.vercel.app/models/ornith-1.5-122b/)125.1B params, ornith-ai | **88.2** / 128.0 GB | Q4FitsQ8Won’t fit | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/)116.8B params, openai | **82.5** / 128.0 GB | Q4FitsQ8Won’t fit | | [Kolibri-1](https://whatcanirunai.vercel.app/models/kolibri-1/)78.1B params, Aleph-Alpha | **55.5** / 128.0 GB | Q4FitsQ8Fits | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/)72.7B params, Qwen | **51.7** / 128.0 GB | Q4FitsQ8Fits | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)70.6B params, meta-llama | **50.2** / 128.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Llama-70B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-llama-70b/)70.6B params, deepseek-ai | **50.2** / 128.0 GB | Q4FitsQ8Fits | | [Recursive Flint](https://whatcanirunai.vercel.app/models/flint/)39.8B params, RecursiveSI | **28.7** / 128.0 GB | Q4FitsQ8Fits | | [Ornith-1.5-35B-A3B](https://whatcanirunai.vercel.app/models/ornith-1.5-35b-a3b/)36.0B params, ornith-ai | **26.1** / 128.0 GB | Q4FitsQ8Fits | | [Sage-A1.5](https://whatcanirunai.vercel.app/models/sage-a1.5/)36.0B params, InternScience | **26.1** / 128.0 GB | Q4FitsQ8Fits | | [MemT-1.0-Nano](https://whatcanirunai.vercel.app/models/memt-1.0-nano/)35.1B params, MemTensor | **25.5** / 128.0 GB | Q4FitsQ8Fits | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/)33.1B params, Nanbeige | **24.1** / 128.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/)32.8B params, Qwen | **23.8** / 128.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/)32.8B params, deepseek-ai | **23.8** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)32.8B params, Qwen | **23.8** / 128.0 GB | Q4FitsQ8Fits | | [NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16](https://whatcanirunai.vercel.app/models/nvidia-nemotron-3.5-lightning-30b-a3b-bf16/)31.6B params, nvidia | **23.0** / 128.0 GB | Q4FitsQ8Fits | | [Gemma 4 31B](https://whatcanirunai.vercel.app/models/gemma-4-31b-it/)31.3B params, google | **22.8** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)30.5B params, Qwen | **22.3** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-Coder-30B-A3B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-coder-30b-a3b-instruct/)30.5B params, Qwen | **22.3** / 128.0 GB | Q4FitsQ8Fits | | [Hy-Memory-30B-A3B](https://whatcanirunai.vercel.app/models/hy-memory-30b-a3b/)30.1B params, tencent | **22.0** / 128.0 GB | Q4FitsQ8Fits | | [Muse-Glimmer-30B](https://whatcanirunai.vercel.app/models/muse-glimmer-30b/)29.8B params, meta-models | **21.8** / 128.0 GB | Q4FitsQ8Fits | | [Raon-1 mini preview](https://whatcanirunai.vercel.app/models/raon-1-mini-preview/)28.8B params, KRAFTON | **21.1** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)27.8B params, Qwen | **20.4** / 128.0 GB | Q4FitsQ8Fits | | [Tess-4.5-27B](https://whatcanirunai.vercel.app/models/tess-4.5-27b/)27.8B params, migtissera | **20.4** / 128.0 GB | Q4FitsQ8Fits | | [gemma-3-27b-it](https://whatcanirunai.vercel.app/models/gemma-3-27b-it/)27.4B params, google | **20.1** / 128.0 GB | Q4FitsQ8Fits | | [Gemma 4 26B A4B](https://whatcanirunai.vercel.app/models/gemma-4-26b-a4b-it/)25.8B params, google | **19.0** / 128.0 GB | Q4FitsQ8Fits | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)24.0B params, mistralai | **17.7** / 128.0 GB | Q4FitsQ8Fits | | [LFM2.5-24B-A2B](https://whatcanirunai.vercel.app/models/lfm2.5-24b-a2b/)24.0B params, LiquidAI | **17.7** / 128.0 GB | Q4FitsQ8Fits | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)20.9B params, openai | **15.6** / 128.0 GB | Q4FitsQ8Fits | | [Carbon-X (DNA)](https://whatcanirunai.vercel.app/models/carbon-x/)14.8B params, HuggingFaceBio | **11.3** / 128.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-14B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-14b-instruct/)14.8B params, Qwen | **11.3** / 128.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/)14.8B params, deepseek-ai | **11.3** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)14.8B params, Qwen | **11.3** / 128.0 GB | Q4FitsQ8Fits | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/)14.7B params, microsoft | **11.2** / 128.0 GB | Q4FitsQ8Fits | | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)13.7B params, allenai | **10.6** / 128.0 GB | Q4FitsQ8Fits | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)12.2B params, mistralai | **9.5** / 128.0 GB | Q4FitsQ8Fits | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/)12.2B params, google | **9.5** / 128.0 GB | Q4FitsQ8Fits | | [Mellum2.1-12B-A2.5B-Thinking](https://whatcanirunai.vercel.app/models/mellum2.1-12b-a2.5b-thinking/)12.1B params, JetBrains | **9.5** / 128.0 GB | Q4FitsQ8Fits | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/)12.0B params, google | **9.3** / 128.0 GB | Q4FitsQ8Fits | | [Ornith-1.5-9B](https://whatcanirunai.vercel.app/models/ornith-1.5-9b/)9.7B params, ornith-ai | **7.7** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)8.2B params, Qwen | **6.7** / 128.0 GB | Q4FitsQ8Fits | | [granite-3.3-8b-instruct](https://whatcanirunai.vercel.app/models/granite-3.3-8b-instruct/)8.2B params, ibm-granite | **6.7** / 128.0 GB | Q4FitsQ8Fits | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)8.0B params, meta-llama | **6.6** / 128.0 GB | Q4FitsQ8Fits | | [Nemotron 3.5 Bolt E4B](https://whatcanirunai.vercel.app/models/nemotron-3.5-bolt-e4b/)8.0B params, nvidia | **6.6** / 128.0 GB | Q4FitsQ8Fits | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/)8.0B params, google | **6.6** / 128.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-7b-instruct/)7.6B params, Qwen | **6.3** / 128.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-7B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-7b/)7.6B params, deepseek-ai | **6.3** / 128.0 GB | Q4FitsQ8Fits | | [Mistral-7B-Instruct-v0.3](https://whatcanirunai.vercel.app/models/mistral-7b-instruct-v0.3/)7.2B params, mistralai | **6.1** / 128.0 GB | Q4FitsQ8Fits | | [Gemma 4 E2B](https://whatcanirunai.vercel.app/models/gemma-4-e2b-it/)5.1B params, google | **4.6** / 128.0 GB | Q4FitsQ8Fits | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)4.3B params, google | **4.0** / 128.0 GB | Q4FitsQ8Fits | | [Spark-X2.6-4B](https://whatcanirunai.vercel.app/models/spark-x2.6-4b/)4.1B params, XHToken | **3.9** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)4.0B params, Qwen | **3.8** / 128.0 GB | Q4FitsQ8Fits | | [Nemotron Labs Dory 1.0 4B](https://whatcanirunai.vercel.app/models/nemotron-labs-dory1.0-4b/)4.0B params, nvidia | **3.8** / 128.0 GB | Q4FitsQ8Fits | | [Phi-4-mini-instruct](https://whatcanirunai.vercel.app/models/phi-4-mini-instruct/)3.8B params, microsoft | **3.7** / 128.0 GB | Q4FitsQ8Fits | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)3.2B params, meta-llama | **3.2** / 128.0 GB | Q4FitsQ8Fits | | [MiniCPM5-2B](https://whatcanirunai.vercel.app/models/minicpm5-2b/)2.5B params, openbmb | **2.8** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-1.7B](https://whatcanirunai.vercel.app/models/qwen3-1.7b/)2.0B params, Qwen | **2.4** / 128.0 GB | Q4FitsQ8Fits | | [LFM3-1.7B](https://whatcanirunai.vercel.app/models/lfm3-1.7b/)1.7B params, LiquidAI | **2.2** / 128.0 GB | Q4FitsQ8Fits | | [Llama-3.2-1B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-1b-instruct/)1.2B params, meta-llama | **1.9** / 128.0 GB | Q4FitsQ8Fits | | [gemma-3-1b-it](https://whatcanirunai.vercel.app/models/gemma-3-1b-it/)1.0B params, google | **1.7** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3-0.6B](https://whatcanirunai.vercel.app/models/qwen3-0.6b/)0.8B params, Qwen | **1.5** / 128.0 GB | Q4FitsQ8Fits | | [Qwen3.8-Flash-Next](https://whatcanirunai.vercel.app/models/qwen3.8-flash-next/)180.0B params, Qwen | **126.5** / 128.0 GB | Q4TightQ8Won’t fit | Showing the 62 most popular of 555 models that fit. [See all 555 in the model directory →](https://whatcanirunai.vercel.app/models/?gb=128) ## Hardware we track with 128+ GB usable * [Mac Studio M5 Ultra 256GB](https://whatcanirunai.vercel.app/what-can-i-run/mac-studio-m5-ultra-256/)192 GB usable, $9499 * [Mac Studio M5 Ultra 512GB](https://whatcanirunai.vercel.app/what-can-i-run/mac-studio-m5-ultra-512/)384 GB usable, $22299 Macs and unified-memory mini-PCs count about 75% of memory as usable ([how we calculate](https://whatcanirunai.vercel.app/method/)). ## Quick answers What is the biggest LLM I can run with 128 GB of VRAM? Mixtral-8x22B-Instruct-v0.1 (140.6B parameters) is the biggest model we track that runs comfortably in 128 GB at Q4\_K\_M, needing about 99.0 GB with a \~4k context. At Q8 the biggest is Qwen3-Next-80B-A3B-Instruct. How many models fit in 128 GB of VRAM? 554 of the 656 open models we track run fully in 128 GB at Q4 with room to spare, 528 also fit at Q8, and 1 more fit only with a short context. Can I run a model that needs more than 128 GB? Yes, with llama.cpp or Ollama you can offload some layers to system RAM, but speed drops sharply. For full-speed local inference the whole model plus its context cache should fit in GPU memory; we use 90% of 128 GB as the comfort line. Which hardware has about 128 GB of usable memory? The cheapest current device we track in this range is the Mac Studio M5 Ultra 256GB (192 GB usable, $9,499 list price as of our last snapshot). 2 tracked devices have 128+ GB usable. ## Other memory sizes [4 GB](https://whatcanirunai.vercel.app/best-llm-for/4gb-vram/) · [6 GB](https://whatcanirunai.vercel.app/best-llm-for/6gb-vram/) · [8 GB](https://whatcanirunai.vercel.app/best-llm-for/8gb-vram/) · [10 GB](https://whatcanirunai.vercel.app/best-llm-for/10gb-vram/) · [12 GB](https://whatcanirunai.vercel.app/best-llm-for/12gb-vram/) · [16 GB](https://whatcanirunai.vercel.app/best-llm-for/16gb-vram/) · [20 GB](https://whatcanirunai.vercel.app/best-llm-for/20gb-vram/) · [24 GB](https://whatcanirunai.vercel.app/best-llm-for/24gb-vram/) · [32 GB](https://whatcanirunai.vercel.app/best-llm-for/32gb-vram/) · [48 GB](https://whatcanirunai.vercel.app/best-llm-for/48gb-vram/) · [64 GB](https://whatcanirunai.vercel.app/best-llm-for/64gb-vram/) · [96 GB](https://whatcanirunai.vercel.app/best-llm-for/96gb-vram/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/