# Best local LLMs for 96 GB VRAM | What Can I Run AI? > 545 open models run fully in 96 GB of VRAM at Q4, up to Mistral-Small-4-119B-2603. See every model that fits, with memory needs. Source: https://whatcanirunai.vercel.app/best-llm-for/96gb-vram/ # Best local LLMs for 96 GB of VRAM Every open model we track that runs fully in 96 GB of GPU memory, biggest first. Checks are at Q4\_K\_M with about 4k context. Run fully at Q4 545 of 656 Run at Q8 too 512 Biggest at Q4 Mistral-Small-4-119B-2603 With 96 GB, the biggest model you can run comfortably is **Mistral-Small-4-119B-2603** (\~84.2 GB at Q4). If you want higher quality at Q8, **LongCat-Flash-Lite** is the biggest that fits. ## Models that fit in 96 GB Bar: Q4 memory needed vs. 96 GB. The tick marks 90%, our comfort line.FitsTight | Model | Q4 need vs. 96 GB | Q4 / Q8 | | --------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------ | | [gpt-oss-120b](https://whatcanirunai.vercel.app/models/gpt-oss-120b/)116.8B params, openai | **82.5** / 96.0 GB | Q4FitsQ8Won’t fit | | [Kolibri-1](https://whatcanirunai.vercel.app/models/kolibri-1/)78.1B params, Aleph-Alpha | **55.5** / 96.0 GB | Q4FitsQ8Won’t fit | | [Qwen2.5-72B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-72b-instruct/)72.7B params, Qwen | **51.7** / 96.0 GB | Q4FitsQ8Tight | | [Llama-3.3-70B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.3-70b-instruct/)70.6B params, meta-llama | **50.2** / 96.0 GB | Q4FitsQ8Tight | | [DeepSeek-R1-Distill-Llama-70B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-llama-70b/)70.6B params, deepseek-ai | **50.2** / 96.0 GB | Q4FitsQ8Tight | | [Recursive Flint](https://whatcanirunai.vercel.app/models/flint/)39.8B params, RecursiveSI | **28.7** / 96.0 GB | Q4FitsQ8Fits | | [Ornith-1.5-35B-A3B](https://whatcanirunai.vercel.app/models/ornith-1.5-35b-a3b/)36.0B params, ornith-ai | **26.1** / 96.0 GB | Q4FitsQ8Fits | | [Sage-A1.5](https://whatcanirunai.vercel.app/models/sage-a1.5/)36.0B params, InternScience | **26.1** / 96.0 GB | Q4FitsQ8Fits | | [MemT-1.0-Nano](https://whatcanirunai.vercel.app/models/memt-1.0-nano/)35.1B params, MemTensor | **25.5** / 96.0 GB | Q4FitsQ8Fits | | [Nanbeige4.3-33B](https://whatcanirunai.vercel.app/models/nanbeige4.3-33b/)33.1B params, Nanbeige | **24.1** / 96.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-32B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-32b-instruct/)32.8B params, Qwen | **23.8** / 96.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-32B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-32b/)32.8B params, deepseek-ai | **23.8** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)32.8B params, Qwen | **23.8** / 96.0 GB | Q4FitsQ8Fits | | [NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16](https://whatcanirunai.vercel.app/models/nvidia-nemotron-3.5-lightning-30b-a3b-bf16/)31.6B params, nvidia | **23.0** / 96.0 GB | Q4FitsQ8Fits | | [Gemma 4 31B](https://whatcanirunai.vercel.app/models/gemma-4-31b-it/)31.3B params, google | **22.8** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)30.5B params, Qwen | **22.3** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-Coder-30B-A3B-Instruct](https://whatcanirunai.vercel.app/models/qwen3-coder-30b-a3b-instruct/)30.5B params, Qwen | **22.3** / 96.0 GB | Q4FitsQ8Fits | | [Hy-Memory-30B-A3B](https://whatcanirunai.vercel.app/models/hy-memory-30b-a3b/)30.1B params, tencent | **22.0** / 96.0 GB | Q4FitsQ8Fits | | [Muse-Glimmer-30B](https://whatcanirunai.vercel.app/models/muse-glimmer-30b/)29.8B params, meta-models | **21.8** / 96.0 GB | Q4FitsQ8Fits | | [Raon-1 mini preview](https://whatcanirunai.vercel.app/models/raon-1-mini-preview/)28.8B params, KRAFTON | **21.1** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)27.8B params, Qwen | **20.4** / 96.0 GB | Q4FitsQ8Fits | | [Tess-4.5-27B](https://whatcanirunai.vercel.app/models/tess-4.5-27b/)27.8B params, migtissera | **20.4** / 96.0 GB | Q4FitsQ8Fits | | [gemma-3-27b-it](https://whatcanirunai.vercel.app/models/gemma-3-27b-it/)27.4B params, google | **20.1** / 96.0 GB | Q4FitsQ8Fits | | [Gemma 4 26B A4B](https://whatcanirunai.vercel.app/models/gemma-4-26b-a4b-it/)25.8B params, google | **19.0** / 96.0 GB | Q4FitsQ8Fits | | [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/)24.0B params, mistralai | **17.7** / 96.0 GB | Q4FitsQ8Fits | | [LFM2.5-24B-A2B](https://whatcanirunai.vercel.app/models/lfm2.5-24b-a2b/)24.0B params, LiquidAI | **17.7** / 96.0 GB | Q4FitsQ8Fits | | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)20.9B params, openai | **15.6** / 96.0 GB | Q4FitsQ8Fits | | [Carbon-X (DNA)](https://whatcanirunai.vercel.app/models/carbon-x/)14.8B params, HuggingFaceBio | **11.3** / 96.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-14B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-14b-instruct/)14.8B params, Qwen | **11.3** / 96.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-14B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-14b/)14.8B params, deepseek-ai | **11.3** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)14.8B params, Qwen | **11.3** / 96.0 GB | Q4FitsQ8Fits | | [phi-4](https://whatcanirunai.vercel.app/models/phi-4/)14.7B params, microsoft | **11.2** / 96.0 GB | Q4FitsQ8Fits | | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)13.7B params, allenai | **10.6** / 96.0 GB | Q4FitsQ8Fits | | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)12.2B params, mistralai | **9.5** / 96.0 GB | Q4FitsQ8Fits | | [gemma-3-12b-it](https://whatcanirunai.vercel.app/models/gemma-3-12b-it/)12.2B params, google | **9.5** / 96.0 GB | Q4FitsQ8Fits | | [Mellum2.1-12B-A2.5B-Thinking](https://whatcanirunai.vercel.app/models/mellum2.1-12b-a2.5b-thinking/)12.1B params, JetBrains | **9.5** / 96.0 GB | Q4FitsQ8Fits | | [Gemma 4 12B](https://whatcanirunai.vercel.app/models/gemma-4-12b-it/)12.0B params, google | **9.3** / 96.0 GB | Q4FitsQ8Fits | | [Ornith-1.5-9B](https://whatcanirunai.vercel.app/models/ornith-1.5-9b/)9.7B params, ornith-ai | **7.7** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)8.2B params, Qwen | **6.7** / 96.0 GB | Q4FitsQ8Fits | | [granite-3.3-8b-instruct](https://whatcanirunai.vercel.app/models/granite-3.3-8b-instruct/)8.2B params, ibm-granite | **6.7** / 96.0 GB | Q4FitsQ8Fits | | [Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/)8.0B params, meta-llama | **6.6** / 96.0 GB | Q4FitsQ8Fits | | [Nemotron 3.5 Bolt E4B](https://whatcanirunai.vercel.app/models/nemotron-3.5-bolt-e4b/)8.0B params, nvidia | **6.6** / 96.0 GB | Q4FitsQ8Fits | | [Gemma 4 E4B](https://whatcanirunai.vercel.app/models/gemma-4-e4b-it/)8.0B params, google | **6.6** / 96.0 GB | Q4FitsQ8Fits | | [Qwen2.5-Coder-7B-Instruct](https://whatcanirunai.vercel.app/models/qwen2.5-coder-7b-instruct/)7.6B params, Qwen | **6.3** / 96.0 GB | Q4FitsQ8Fits | | [DeepSeek-R1-Distill-Qwen-7B](https://whatcanirunai.vercel.app/models/deepseek-r1-distill-qwen-7b/)7.6B params, deepseek-ai | **6.3** / 96.0 GB | Q4FitsQ8Fits | | [Mistral-7B-Instruct-v0.3](https://whatcanirunai.vercel.app/models/mistral-7b-instruct-v0.3/)7.2B params, mistralai | **6.1** / 96.0 GB | Q4FitsQ8Fits | | [Gemma 4 E2B](https://whatcanirunai.vercel.app/models/gemma-4-e2b-it/)5.1B params, google | **4.6** / 96.0 GB | Q4FitsQ8Fits | | [gemma-3-4b-it](https://whatcanirunai.vercel.app/models/gemma-3-4b-it/)4.3B params, google | **4.0** / 96.0 GB | Q4FitsQ8Fits | | [Spark-X2.6-4B](https://whatcanirunai.vercel.app/models/spark-x2.6-4b/)4.1B params, XHToken | **3.9** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-4B](https://whatcanirunai.vercel.app/models/qwen3-4b/)4.0B params, Qwen | **3.8** / 96.0 GB | Q4FitsQ8Fits | | [Nemotron Labs Dory 1.0 4B](https://whatcanirunai.vercel.app/models/nemotron-labs-dory1.0-4b/)4.0B params, nvidia | **3.8** / 96.0 GB | Q4FitsQ8Fits | | [Phi-4-mini-instruct](https://whatcanirunai.vercel.app/models/phi-4-mini-instruct/)3.8B params, microsoft | **3.7** / 96.0 GB | Q4FitsQ8Fits | | [Llama-3.2-3B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-3b-instruct/)3.2B params, meta-llama | **3.2** / 96.0 GB | Q4FitsQ8Fits | | [MiniCPM5-2B](https://whatcanirunai.vercel.app/models/minicpm5-2b/)2.5B params, openbmb | **2.8** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-1.7B](https://whatcanirunai.vercel.app/models/qwen3-1.7b/)2.0B params, Qwen | **2.4** / 96.0 GB | Q4FitsQ8Fits | | [LFM3-1.7B](https://whatcanirunai.vercel.app/models/lfm3-1.7b/)1.7B params, LiquidAI | **2.2** / 96.0 GB | Q4FitsQ8Fits | | [Llama-3.2-1B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.2-1b-instruct/)1.2B params, meta-llama | **1.9** / 96.0 GB | Q4FitsQ8Fits | | [gemma-3-1b-it](https://whatcanirunai.vercel.app/models/gemma-3-1b-it/)1.0B params, google | **1.7** / 96.0 GB | Q4FitsQ8Fits | | [Qwen3-0.6B](https://whatcanirunai.vercel.app/models/qwen3-0.6b/)0.8B params, Qwen | **1.5** / 96.0 GB | Q4FitsQ8Fits | | [Ling-3.0-flash-Sante](https://whatcanirunai.vercel.app/models/ling-3.0-flash-sante/)127.5B params, inclusionAI | **89.9** / 96.0 GB | Q4TightQ8Won’t fit | | [Ornith-1.5-122B](https://whatcanirunai.vercel.app/models/ornith-1.5-122b/)125.1B params, ornith-ai | **88.2** / 96.0 GB | Q4TightQ8Won’t fit | Showing the 61 most popular of 553 models that fit. [See all 553 in the model directory →](https://whatcanirunai.vercel.app/models/?gb=96) ## Hardware we track with 96–127 GB usable * [Framework Desktop (Ryzen AI Max+ 395) 128GB](https://whatcanirunai.vercel.app/what-can-i-run/framework-desktop-128/)96 GB usable, $3449 * [MacBook Pro M4 Max 128GB](https://whatcanirunai.vercel.app/what-can-i-run/macbook-pro-m4-max-128/)96 GB usable, $4999 * [MacBook Pro M5 Max 128GB](https://whatcanirunai.vercel.app/what-can-i-run/macbook-pro-m5-max-128/)96 GB usable, $6699 * [NVIDIA DGX Spark 128GB](https://whatcanirunai.vercel.app/what-can-i-run/dgx-spark-128/)96 GB usable, $6950 * [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/what-can-i-run/rtx-pro-6000-blackwell/)96 GB usable, $8565 Macs and unified-memory mini-PCs count about 75% of memory as usable ([how we calculate](https://whatcanirunai.vercel.app/method/)). ## Quick answers What is the biggest LLM I can run with 96 GB of VRAM? Mistral-Small-4-119B-2603 (119.4B parameters) is the biggest model we track that runs comfortably in 96 GB at Q4\_K\_M, needing about 84.2 GB with a \~4k context. At Q8 the biggest is LongCat-Flash-Lite. How many models fit in 96 GB of VRAM? 545 of the 656 open models we track run fully in 96 GB at Q4 with room to spare, 512 also fit at Q8, and 8 more fit only with a short context. Can I run a model that needs more than 96 GB? Yes, with llama.cpp or Ollama you can offload some layers to system RAM, but speed drops sharply. For full-speed local inference the whole model plus its context cache should fit in GPU memory; we use 90% of 96 GB as the comfort line. Which hardware has about 96 GB of usable memory? The cheapest current device we track in this range is the Framework Desktop (Ryzen AI Max+ 395) 128GB (96 GB usable, $3,449 list price as of our last snapshot). 5 tracked devices have 96–127 GB usable. ## Other memory sizes [4 GB](https://whatcanirunai.vercel.app/best-llm-for/4gb-vram/) · [6 GB](https://whatcanirunai.vercel.app/best-llm-for/6gb-vram/) · [8 GB](https://whatcanirunai.vercel.app/best-llm-for/8gb-vram/) · [10 GB](https://whatcanirunai.vercel.app/best-llm-for/10gb-vram/) · [12 GB](https://whatcanirunai.vercel.app/best-llm-for/12gb-vram/) · [16 GB](https://whatcanirunai.vercel.app/best-llm-for/16gb-vram/) · [20 GB](https://whatcanirunai.vercel.app/best-llm-for/20gb-vram/) · [24 GB](https://whatcanirunai.vercel.app/best-llm-for/24gb-vram/) · [32 GB](https://whatcanirunai.vercel.app/best-llm-for/32gb-vram/) · [48 GB](https://whatcanirunai.vercel.app/best-llm-for/48gb-vram/) · [64 GB](https://whatcanirunai.vercel.app/best-llm-for/64gb-vram/) · [128 GB](https://whatcanirunai.vercel.app/best-llm-for/128gb-vram/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/