# Best GPU for running local LLMs (2026) | What Can I Run AI? > Which graphics card to buy for local LLMs: every NVIDIA, AMD and Intel GPU we track ranked by VRAM, with prices, $ per GB and estimated tok/s for Qwen3 and gpt-oss. Source: https://whatcanirunai.vercel.app/best-gpu-for-local-llm/ # Best GPU for running local LLMs Every graphics card we track, ranked by VRAM, with prices, price per GB and estimated speed for six popular open models at Q4\_K\_M. VRAM decides which models fit; memory bandwidth decides how fast they run. GPUs tracked 64 VRAM range 6–96 GB 24 GB+ cards 11 Short answer: the cheapest 24 GB card we track, the **Intel Arc Pro B70 32GB** ($949), runs 27–32B models at Q4; on a budget, the **Radeon RX 7600 XT** ($329) gives 16 GB. ## Top picks Cheapest entry (12 GB) ### [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/) VRAM 12 GB Bandwidth 456 GB/s Run at Q4 350 / 656 The cheapest card we track with 12 GB, enough for 8B–14B models at Q4\. Biggest at Q4: [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/). **$249** list price (MSRP), [source](https://en.wikipedia.org/wiki/Intel%5FArc#Battlemage), 2026-10-22 Street prices may be higher than list. [Amazon](https://www.amazon.com/s?k=Intel%20Arc%20B580&tag=whatcanirunai-20)[eBay](https://www.ebay.com/sch/i.html?%5Fnkw=Intel%20Arc%20B580) Best budget 16 GB and most VRAM per dollar ### [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/) VRAM 16 GB Bandwidth 288 GB/s Run at Q4 388 / 656 The cheapest 16 GB card we track. Room for 14B models with longer context; gpt-oss-20b (\~15.6 GB) is tight at short context. It also has the lowest price per GB of VRAM among the cards we track. Biggest at Q4: [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/). **$329** list price (MSRP), [source](https://en.wikipedia.org/wiki/Radeon%5FRX%5F7000%5Fseries), 2026-10-22 Street prices may be higher than list. [Amazon](https://www.amazon.com/s?k=Radeon%20RX%207600%20XT&tag=whatcanirunai-20)[eBay](https://www.ebay.com/sch/i.html?%5Fnkw=Radeon%20RX%207600%20XT) Best for 27–32B models ### [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/) VRAM 32 GB Bandwidth 608 GB/s Run at Q4 504 / 656 The cheapest card we track that fits Qwen3.8-27B at Q4\. Biggest at Q4: [IQuest-Coder-V1-40B-Loop-Instruct](https://whatcanirunai.vercel.app/models/iquest-coder-v1-40b-loop-instruct/). **$949** list price (MSRP), [source](https://www.igorslab.de/en/intel-launches-the-arc-pro-b70-with-32-gb-of-vram-for-949-big-battlemage-is-coming-first-for-ai-and-workstations/), 2026-10-29 Street prices may be higher than list. [Amazon](https://www.amazon.com/s?k=Intel%20Arc%20Pro%20B70%2032GB&tag=whatcanirunai-20)[eBay](https://www.ebay.com/sch/i.html?%5Fnkw=Intel%20Arc%20Pro%20B70%2032GB) Fastest single card ### [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/) VRAM 96 GB Bandwidth 1792 GB/s Run at Q4 545 / 656 The most VRAM and memory bandwidth of any consumer card we track. Biggest at Q4: [Mistral-Small-4-119B-2603](https://whatcanirunai.vercel.app/models/mistral-small-4-119b-2603/). **$8565** list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 Street prices may be higher than list. [Amazon](https://www.amazon.com/s?k=NVIDIA%20RTX%20Pro%206000%20Blackwell%2096GB&tag=whatcanirunai-20)[eBay](https://www.ebay.com/sch/i.html?%5Fnkw=NVIDIA%20RTX%20Pro%206000%20Blackwell%2096GB) We may earn a commission when you buy through these links, at no extra cost to you. ## Every GPU, by VRAM Prices are dated list-price (MSRP) snapshots; street prices may be higher. Each row links its source. Board power is from the maker's spec page where we have it. __GPUs ranked by VRAM__ | GPU | Price | VRAM | Bandwidth | $ per GB | Board power | Biggest model at Q4 | Models that fit | | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | --------- | -------- | ----------- | ---------------------------------------------------------------------------------------------- | --------------- | | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)NVIDIA | $8,565list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 96 GB | 1792 GB/s | $89 | 600 W | [Mistral-Small-4-119B-2603](https://whatcanirunai.vercel.app/models/mistral-small-4-119b-2603/)\~84.2 GB at Q4 | 545 / 656 | | [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)NVIDIA | $4,649list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 48 GB | 768 GB/s | $97 | 300 W | [AI21-Jamba2-Mini](https://whatcanirunai.vercel.app/models/ai21-jamba2-mini/)\~37.0 GB at Q4 | 511 / 656 | | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)NVIDIA | $6,799list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 48 GB | 960 GB/s | $142 | 300 W | [AI21-Jamba2-Mini](https://whatcanirunai.vercel.app/models/ai21-jamba2-mini/)\~37.0 GB at Q4 | 511 / 656 | | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)Intel | $949list price (MSRP), [source](https://www.igorslab.de/en/intel-launches-the-arc-pro-b70-with-32-gb-of-vram-for-949-big-battlemage-is-coming-first-for-ai-and-workstations/), 2026-10-29 | 32 GB | 608 GB/s | $30 | 230 W | [IQuest-Coder-V1-40B-Loop-Instruct](https://whatcanirunai.vercel.app/models/iquest-coder-v1-40b-loop-instruct/)\~28.7 GB at Q4 | 504 / 656 | | [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)NVIDIA | $1,999list price (MSRP), [source](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/), 2026-10-22 | 32 GB | 1792 GB/s | $62 | 575 W | [IQuest-Coder-V1-40B-Loop-Instruct](https://whatcanirunai.vercel.app/models/iquest-coder-v1-40b-loop-instruct/)\~28.7 GB at Q4 | 504 / 656 | | [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)AMD | $999list price (MSRP), [source](https://en.wikipedia.org/wiki/Radeon%5FRX%5F7000%5Fseries), 2026-10-22 | 24 GB | 960 GB/s | $42 | 355 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)NVIDIA | $1,499list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 24 GB | 936 GB/s | $62 | 350 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)NVIDIA | $1,599list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 24 GB | 1008 GB/s | $67 | 450 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)NVIDIA | $1,999list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 24 GB | 1008 GB/s | $83 | 450 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)NVIDIA | $2,250list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 24 GB | 768 GB/s | $94 | 230 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)NVIDIA | $2,499list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 24 GB | 672 GB/s | $104 | 280 W | [ERNIE-4.5-VL-28B-A3B-PT](https://whatcanirunai.vercel.app/models/ernie-4.5-vl-28b-a3b-pt/)\~21.5 GB at Q4 | 423 / 656 | | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)AMD | $899list price (MSRP), [source](https://en.wikipedia.org/wiki/Radeon%5FRX%5F7000%5Fseries), 2026-10-22 | 20 GB | 800 GB/s | $45 | 315 W | [Voxtral-Small-24B-2507](https://whatcanirunai.vercel.app/models/voxtral-small-24b-2507/)\~17.9 GB at Q4 | 404 / 656 | | [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)NVIDIA | $1,250list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 20 GB | 360 GB/s | $63 | 130 W | [Voxtral-Small-24B-2507](https://whatcanirunai.vercel.app/models/voxtral-small-24b-2507/)\~17.9 GB at Q4 | 404 / 656 | | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)AMD | $329list price (MSRP), [source](https://en.wikipedia.org/wiki/Radeon%5FRX%5F7000%5Fseries), 2026-10-22 | 16 GB | 288 GB/s | $21 | 190 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 9060 XT 16GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-16gb/)AMD | $349list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 320 GB/s | $22 | 160 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Intel Arc A770 16GB](https://whatcanirunai.vercel.app/hardware/arc-a770-16gb/)Intel | $349list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/Intel%5FArc), 2026-10-28 | 16 GB | 560 GB/s | $22 | 225 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 5060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-16gb/)NVIDIA | $429list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 448 GB/s | $27 | 180 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-16gb/)NVIDIA | $499list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 16 GB | 288 GB/s | $31 | 165 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 7800 XT](https://whatcanirunai.vercel.app/hardware/rx-7800-xt/)AMD | $499list price (MSRP), [source](https://en.wikipedia.org/wiki/Radeon%5FRX%5F7000%5Fseries), 2026-10-22 | 16 GB | 624 GB/s | $31 | 263 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 7900 GRE](https://whatcanirunai.vercel.app/hardware/rx-7900-gre/)AMD | $549list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 576 GB/s | $34 | 260 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 9070](https://whatcanirunai.vercel.app/hardware/rx-9070/)AMD | $549list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 640 GB/s | $34 | 220 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 6800](https://whatcanirunai.vercel.app/hardware/rx-6800/)AMD | $579list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 512 GB/s | $36 | 250 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 9070 XT](https://whatcanirunai.vercel.app/hardware/rx-9070-xt/)AMD | $599list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 640 GB/s | $37 | 304 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 6800 XT](https://whatcanirunai.vercel.app/hardware/rx-6800-xt/)AMD | $649list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 512 GB/s | $41 | 300 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [NVIDIA RTX 2000 Ada 16GB](https://whatcanirunai.vercel.app/hardware/rtx-2000-ada/)NVIDIA | $649list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 224 GB/s | $41 | 70 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 5070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-5070-ti/)NVIDIA | $749list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 896 GB/s | $47 | 300 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 4070 Ti Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti-super/)NVIDIA | $799list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 16 GB | 672 GB/s | $50 | 285 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 4080 Super](https://whatcanirunai.vercel.app/hardware/rtx-4080-super/)NVIDIA | $999list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 16 GB | 736 GB/s | $62 | 320 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 5080](https://whatcanirunai.vercel.app/hardware/rtx-5080/)NVIDIA | $999list price (MSRP), [source](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080/), 2026-10-22 | 16 GB | 960 GB/s | $62 | 360 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 6900 XT](https://whatcanirunai.vercel.app/hardware/rx-6900-xt/)AMD | $999list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 512 GB/s | $62 | 300 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [NVIDIA RTX A4000](https://whatcanirunai.vercel.app/hardware/rtx-a4000/)NVIDIA | $1,000list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 448 GB/s | $63 | 140 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Radeon RX 6950 XT](https://whatcanirunai.vercel.app/hardware/rx-6950-xt/)AMD | $1,099list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 576 GB/s | $69 | 335 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [GeForce RTX 4080](https://whatcanirunai.vercel.app/hardware/rtx-4080/)NVIDIA | $1,199list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 16 GB | 717 GB/s | $75 | 320 W | [Ling-Coder-lite](https://whatcanirunai.vercel.app/models/ling-coder-lite/)\~12.7 GB at Q4 | 388 / 656 | | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)Intel | $249list price (MSRP), [source](https://en.wikipedia.org/wiki/Intel%5FArc#Battlemage), 2026-10-22 | 12 GB | 456 GB/s | $21 | — | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 3060 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3060-12gb/)NVIDIA | $329list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 12 GB | 360 GB/s | $27 | 170 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [Radeon RX 7700 XT](https://whatcanirunai.vercel.app/hardware/rx-7700-xt/)AMD | $449list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 12 GB | 432 GB/s | $37 | 245 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [Radeon RX 6700 XT](https://whatcanirunai.vercel.app/hardware/rx-6700-xt/)AMD | $479list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 12 GB | 384 GB/s | $40 | 230 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 5070](https://whatcanirunai.vercel.app/hardware/rtx-5070/)NVIDIA | $549list price (MSRP), [source](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5070-family/), 2026-10-22 | 12 GB | 672 GB/s | $46 | 250 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [Radeon RX 6750 XT](https://whatcanirunai.vercel.app/hardware/rx-6750-xt/)AMD | $549list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 12 GB | 432 GB/s | $46 | 250 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 4070](https://whatcanirunai.vercel.app/hardware/rtx-4070/)NVIDIA | $599list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 12 GB | 504 GB/s | $50 | 200 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 4070 Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-super/)NVIDIA | $599list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 12 GB | 504 GB/s | $50 | 220 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 3080 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-12gb/)NVIDIA | $799list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 12 GB | 912 GB/s | $67 | 350 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 4070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti/)NVIDIA | $799list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 12 GB | 504 GB/s | $67 | 285 W | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 3080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3080-ti/)NVIDIA | $1,199list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 12 GB | 912 GB/s | $100 | — | [OLMo-2-1124-13B-Instruct](https://whatcanirunai.vercel.app/models/olmo-2-1124-13b-instruct/)\~10.6 GB at Q4 | 350 / 656 | | [GeForce RTX 2080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-2080-ti/)NVIDIA | $999list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 11 GB | 616 GB/s | $91 | 250 W | [Mistral-Nemo-Instruct-2407](https://whatcanirunai.vercel.app/models/mistral-nemo-instruct-2407/)\~9.5 GB at Q4 | 349 / 656 | | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)Intel | $219list price (MSRP), [source](https://en.wikipedia.org/wiki/Intel%5FArc), 2026-10-28 | 10 GB | 380 GB/s | $22 | 150 W | [HyperCLOVAX-SEED-Omni-8B](https://whatcanirunai.vercel.app/models/hyperclovax-seed-omni-8b/)\~8.5 GB at Q4 | 340 / 656 | | [GeForce RTX 3080 10GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-10gb/)NVIDIA | $699list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 10 GB | 760 GB/s | $70 | 320 W | [HyperCLOVAX-SEED-Omni-8B](https://whatcanirunai.vercel.app/models/hyperclovax-seed-omni-8b/)\~8.5 GB at Q4 | 340 / 656 | | [GeForce RTX 3050 8GB](https://whatcanirunai.vercel.app/hardware/rtx-3050-8gb/)NVIDIA | $249list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 8 GB | 224 GB/s | $31 | 130 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [Radeon RX 7600](https://whatcanirunai.vercel.app/hardware/rx-7600/)AMD | $269list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 288 GB/s | $34 | 165 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [Intel Arc A750](https://whatcanirunai.vercel.app/hardware/arc-a750/)Intel | $289list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/Intel%5FArc), 2026-10-28 | 8 GB | 512 GB/s | $36 | 225 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 4060](https://whatcanirunai.vercel.app/hardware/rtx-4060/)NVIDIA | $299list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F40%5Fseries), 2026-10-22 | 8 GB | 272 GB/s | $37 | 115 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 5060](https://whatcanirunai.vercel.app/hardware/rtx-5060/)NVIDIA | $299list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $37 | 145 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [Radeon RX 9060 XT 8GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-8gb/)AMD | $299list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FAMD%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 320 GB/s | $37 | 150 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 5060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-8gb/)NVIDIA | $379list price (MSRP), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $47 | 180 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 3060 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3060-ti/)NVIDIA | $399list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 8 GB | 448 GB/s | $50 | 200 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2060 Super](https://whatcanirunai.vercel.app/hardware/rtx-2060-super/)NVIDIA | $399list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $50 | 175 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 4060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-8gb/)NVIDIA | $399list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 288 GB/s | $50 | 160 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 3070](https://whatcanirunai.vercel.app/hardware/rtx-3070/)NVIDIA | $499list price (MSRP), [source](https://en.wikipedia.org/wiki/GeForce%5F30%5Fseries), 2026-10-22 | 8 GB | 448 GB/s | $62 | 220 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2070](https://whatcanirunai.vercel.app/hardware/rtx-2070/)NVIDIA | $499list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $62 | 175 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2070 Super](https://whatcanirunai.vercel.app/hardware/rtx-2070-super/)NVIDIA | $499list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $62 | 215 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/)NVIDIA | $599list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 608 GB/s | $75 | 290 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2080](https://whatcanirunai.vercel.app/hardware/rtx-2080/)NVIDIA | $699list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 448 GB/s | $87 | 215 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2080 Super](https://whatcanirunai.vercel.app/hardware/rtx-2080-super/)NVIDIA | $699list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 8 GB | 496 GB/s | $87 | 250 W | [NVIDIA-Nemotron-Nano-9B-v2](https://whatcanirunai.vercel.app/models/nvidia-nemotron-nano-9b-v2/)\~7.2 GB at Q4 | 315 / 656 | | [GeForce RTX 2060 6GB](https://whatcanirunai.vercel.app/hardware/rtx-2060/)NVIDIA | $349list (launch MSRP; discontinued), [source](https://en.wikipedia.org/wiki/List%5Fof%5FNvidia%5Fgraphics%5Fprocessing%5Funits), 2026-10-28 | 6 GB | 336 GB/s | $58 | 160 W | [YuE-s1-7B-anneal-en-cot](https://whatcanirunai.vercel.app/models/yue-s1-7b-anneal-en-cot/)\~5.3 GB at Q4 | 198 / 656 | ## Which popular models run, and how fast Estimated decode speed at Q4 in tokens per second ([how we estimate speed](https://whatcanirunai.vercel.app/method/#speed)). MoE models show an upper bound only. "Tight" means it fits with short context only. __Estimated fit and decode speed at Q4 for popular models on each GPU__ | GPU | [Qwen3-8B](https://whatcanirunai.vercel.app/models/qwen3-8b/)\~6.7 GB | [Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/)\~11.3 GB | [gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/)\~15.6 GB | [Qwen3.8-27B](https://whatcanirunai.vercel.app/models/qwen3.8-27b/)\~20.4 GB | [Qwen3-30B-A3B](https://whatcanirunai.vercel.app/models/qwen3-30b-a3b/)\~22.3 GB | [Qwen3-32B](https://whatcanirunai.vercel.app/models/qwen3-32b/)\~23.8 GB | | ---------------------------------------------------------------------------- | ------------------------------------- | ---------------------------------------- | -------------------------------------------- | -------------------------------------------- | ------------------------------------------------ | ---------------------------------------- | | [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)96 GB | Fits217–307 tok/s | Fits120–170 tok/s | Fitsup to 698 tok/s | Fits64–90 tok/s | Fitsup to 837 tok/s | Fits54–77 tok/s | | [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)48 GB | Fits93–131 tok/s | Fits51–73 tok/s | Fitsup to 299 tok/s | Fits27–39 tok/s | Fitsup to 359 tok/s | Fits23–33 tok/s | | [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)48 GB | Fits116–164 tok/s | Fits64–91 tok/s | Fitsup to 374 tok/s | Fits34–48 tok/s | Fitsup to 449 tok/s | Fits29–41 tok/s | | [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)32 GB | Fits73–104 tok/s | Fits41–58 tok/s | Fitsup to 237 tok/s | Fits22–31 tok/s | Fitsup to 284 tok/s | Fits18–26 tok/s | | [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)32 GB | Fits217–307 tok/s | Fits120–170 tok/s | Fitsup to 698 tok/s | Fits64–90 tok/s | Fitsup to 837 tok/s | Fits54–77 tok/s | | [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)24 GB | Fits116–164 tok/s | Fits64–91 tok/s | Fitsup to 374 tok/s | Fits34–48 tok/s | Tightup to 449 tok/s | Tight29–41 tok/s | | [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)24 GB | Fits113–160 tok/s | Fits63–89 tok/s | Fitsup to 365 tok/s | Fits33–47 tok/s | Tightup to 437 tok/s | Tight28–40 tok/s | | [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)24 GB | Fits122–173 tok/s | Fits68–96 tok/s | Fitsup to 393 tok/s | Fits36–51 tok/s | Tightup to 471 tok/s | Tight30–43 tok/s | | [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)24 GB | Fits122–173 tok/s | Fits68–96 tok/s | Fitsup to 393 tok/s | Fits36–51 tok/s | Tightup to 471 tok/s | Tight30–43 tok/s | | [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)24 GB | Fits93–131 tok/s | Fits51–73 tok/s | Fitsup to 299 tok/s | Fits27–39 tok/s | Tightup to 359 tok/s | Tight23–33 tok/s | | [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)24 GB | Fits81–115 tok/s | Fits45–64 tok/s | Fitsup to 262 tok/s | Fits24–34 tok/s | Tightup to 314 tok/s | Tight20–29 tok/s | | [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)20 GB | Fits97–137 tok/s | Fits54–76 tok/s | Fitsup to 312 tok/s | Won’t fit | Won’t fit | Won’t fit | | [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)20 GB | Fits43–62 tok/s | Fits24–34 tok/s | Fitsup to 140 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)16 GB | Fits35–49 tok/s | Fits19–27 tok/s | Tightup to 112 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 9060 XT 16GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-16gb/)16 GB | Fits39–55 tok/s | Fits21–30 tok/s | Tightup to 125 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Intel Arc A770 16GB](https://whatcanirunai.vercel.app/hardware/arc-a770-16gb/)16 GB | Fits68–96 tok/s | Fits38–53 tok/s | Tightup to 218 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-16gb/)16 GB | Fits54–77 tok/s | Fits30–43 tok/s | Tightup to 174 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-16gb/)16 GB | Fits35–49 tok/s | Fits19–27 tok/s | Tightup to 112 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 7800 XT](https://whatcanirunai.vercel.app/hardware/rx-7800-xt/)16 GB | Fits75–107 tok/s | Fits42–59 tok/s | Tightup to 243 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 7900 GRE](https://whatcanirunai.vercel.app/hardware/rx-7900-gre/)16 GB | Fits70–99 tok/s | Fits39–55 tok/s | Tightup to 224 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 9070](https://whatcanirunai.vercel.app/hardware/rx-9070/)16 GB | Fits77–110 tok/s | Fits43–61 tok/s | Tightup to 249 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6800](https://whatcanirunai.vercel.app/hardware/rx-6800/)16 GB | Fits62–88 tok/s | Fits34–49 tok/s | Tightup to 199 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 9070 XT](https://whatcanirunai.vercel.app/hardware/rx-9070-xt/)16 GB | Fits77–110 tok/s | Fits43–61 tok/s | Tightup to 249 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6800 XT](https://whatcanirunai.vercel.app/hardware/rx-6800-xt/)16 GB | Fits62–88 tok/s | Fits34–49 tok/s | Tightup to 199 tok/s | Won’t fit | Won’t fit | Won’t fit | | [NVIDIA RTX 2000 Ada 16GB](https://whatcanirunai.vercel.app/hardware/rtx-2000-ada/)16 GB | Fits27–38 tok/s | Fits15–21 tok/s | Tightup to 87 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-5070-ti/)16 GB | Fits108–153 tok/s | Fits60–85 tok/s | Tightup to 349 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4070 Ti Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti-super/)16 GB | Fits81–115 tok/s | Fits45–64 tok/s | Tightup to 262 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4080 Super](https://whatcanirunai.vercel.app/hardware/rtx-4080-super/)16 GB | Fits89–126 tok/s | Fits49–70 tok/s | Tightup to 287 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5080](https://whatcanirunai.vercel.app/hardware/rtx-5080/)16 GB | Fits116–164 tok/s | Fits64–91 tok/s | Tightup to 374 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6900 XT](https://whatcanirunai.vercel.app/hardware/rx-6900-xt/)16 GB | Fits62–88 tok/s | Fits34–49 tok/s | Tightup to 199 tok/s | Won’t fit | Won’t fit | Won’t fit | | [NVIDIA RTX A4000](https://whatcanirunai.vercel.app/hardware/rtx-a4000/)16 GB | Fits54–77 tok/s | Fits30–43 tok/s | Tightup to 174 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6950 XT](https://whatcanirunai.vercel.app/hardware/rx-6950-xt/)16 GB | Fits70–99 tok/s | Fits39–55 tok/s | Tightup to 224 tok/s | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4080](https://whatcanirunai.vercel.app/hardware/rtx-4080/)16 GB | Fits87–123 tok/s | Fits48–68 tok/s | Tightup to 279 tok/s | Won’t fit | Won’t fit | Won’t fit | | [Intel Arc B580](https://whatcanirunai.vercel.app/hardware/arc-b580/)12 GB | Fits55–78 tok/s | Tight31–43 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3060 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3060-12gb/)12 GB | Fits43–62 tok/s | Tight24–34 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 7700 XT](https://whatcanirunai.vercel.app/hardware/rx-7700-xt/)12 GB | Fits52–74 tok/s | Tight29–41 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6700 XT](https://whatcanirunai.vercel.app/hardware/rx-6700-xt/)12 GB | Fits46–66 tok/s | Tight26–36 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5070](https://whatcanirunai.vercel.app/hardware/rtx-5070/)12 GB | Fits81–115 tok/s | Tight45–64 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 6750 XT](https://whatcanirunai.vercel.app/hardware/rx-6750-xt/)12 GB | Fits52–74 tok/s | Tight29–41 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4070](https://whatcanirunai.vercel.app/hardware/rtx-4070/)12 GB | Fits61–86 tok/s | Tight34–48 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4070 Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-super/)12 GB | Fits61–86 tok/s | Tight34–48 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3080 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-12gb/)12 GB | Fits110–156 tok/s | Tight61–87 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti/)12 GB | Fits61–86 tok/s | Tight34–48 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3080-ti/)12 GB | Fits110–156 tok/s | Tight61–87 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-2080-ti/)11 GB | Fits74–105 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)10 GB | Fits46–65 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3080 10GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-10gb/)10 GB | Fits92–130 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3050 8GB](https://whatcanirunai.vercel.app/hardware/rtx-3050-8gb/)8 GB | Fits27–38 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 7600](https://whatcanirunai.vercel.app/hardware/rx-7600/)8 GB | Fits35–49 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Intel Arc A750](https://whatcanirunai.vercel.app/hardware/arc-a750/)8 GB | Fits62–88 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4060](https://whatcanirunai.vercel.app/hardware/rtx-4060/)8 GB | Fits33–47 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5060](https://whatcanirunai.vercel.app/hardware/rtx-5060/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [Radeon RX 9060 XT 8GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-8gb/)8 GB | Fits39–55 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 5060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-8gb/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3060 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3060-ti/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2060 Super](https://whatcanirunai.vercel.app/hardware/rtx-2060-super/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 4060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-8gb/)8 GB | Fits35–49 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3070](https://whatcanirunai.vercel.app/hardware/rtx-3070/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2070](https://whatcanirunai.vercel.app/hardware/rtx-2070/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2070 Super](https://whatcanirunai.vercel.app/hardware/rtx-2070-super/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/)8 GB | Fits73–104 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2080](https://whatcanirunai.vercel.app/hardware/rtx-2080/)8 GB | Fits54–77 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2080 Super](https://whatcanirunai.vercel.app/hardware/rtx-2080-super/)8 GB | Fits60–85 tok/s | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | | [GeForce RTX 2060 6GB](https://whatcanirunai.vercel.app/hardware/rtx-2060/)6 GB | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | Won’t fit | ## Need more than 32 GB? No single consumer card we track holds a 70B model at Q4\. Two 24 GB cards do (see [multi-GPU for local LLMs](https://whatcanirunai.vercel.app/multi-gpu-for-local-llm/)): see the [ready-made builds](https://whatcanirunai.vercel.app/builds/) or plan one in the [planner](https://whatcanirunai.vercel.app/planner/). A 128 GB unified-memory machine also works: see the [best Mac for local LLMs](https://whatcanirunai.vercel.app/best-mac-for-local-llm/) or [Framework Desktop vs DGX Spark](https://whatcanirunai.vercel.app/compare/framework-desktop-128-vs-dgx-spark-128/). Card not listed? Enter its VRAM in the [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/). Have a budget in mind? See [best GPU by budget](https://whatcanirunai.vercel.app/best-gpu-under/). ## Quick answers What is the best GPU for running local LLMs? For most people it is a 24 GB card, because 24 GB fits 27B–32B models at Q4\. The cheapest 24 GB card we track is the Intel Arc Pro B70 32GB ($949). On a tighter budget, the Radeon RX 7600 XT ($329) gives 16 GB. For the most speed in one card, the NVIDIA RTX Pro 6000 Blackwell 96GB has 96 GB and 1792 GB/s. How much VRAM do I need for a local LLM? At Q4\_K\_M, plan on roughly 0.7 GB per billion parameters plus about 1 GB of overhead at \~4k context. An 8B model needs \~6.7 GB, a 14B \~11.3 GB, a 32B \~23.8 GB. Longer context needs more. What is the cheapest GPU for local AI? The Intel Arc B580 at $249 (2026-10-22) is the cheapest 12 GB card we track. It runs 350 of our 656 models at Q4\. 8 GB cards are cheaper but limit you to about 7–8B models. Can one GPU run a 70B model? Not at Q4 on any consumer card we track: Llama 3.3 70B needs about 50.2 GB, more than the 32 GB on an RTX 5090\. Use two 24 GB cards, a 128 GB unified-memory machine, or a smaller quant with less quality. NVIDIA, AMD or Intel for local LLMs? NVIDIA (CUDA) has the widest app support. AMD and Intel cards work in llama.cpp-based apps through ROCm, SYCL or Vulkan, often for less money per GB of VRAM, but some tools support them later or not at all. Our fit estimates are the same for all three; speed depends mostly on memory bandwidth. How are the speed estimates made? Memory bandwidth divided by the model's Q4 weight size, times an efficiency band calibrated on public llama.cpp CUDA results. MoE models show an upper bound only. Real speed depends on the app, context length and settings. [Buying guides](https://whatcanirunai.vercel.app/best-gpu-for/) · [Best GPU by budget](https://whatcanirunai.vercel.app/best-gpu-under/) · [Best used GPU](https://whatcanirunai.vercel.app/best-used-gpu-for-local-llm/) · [Best Mac](https://whatcanirunai.vercel.app/best-mac-for-local-llm/) · [NVIDIA vs AMD vs Mac](https://whatcanirunai.vercel.app/nvidia-vs-amd-vs-mac/) · [Tokens per second](https://whatcanirunai.vercel.app/tokens-per-second/) · [GPU vs GPU](https://whatcanirunai.vercel.app/compare/) --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/