# GPU power consumption for local LLMs: watts, tokens per watt and cost (2026)

> How much power and electricity does running an LLM locally use? Board power, est. tok/s per 100 W, Wh and cost per 1M tokens for 62 GPUs.

Source: https://whatcanirunai.vercel.app/gpu-power-consumption/

# GPU power consumption for local LLMs

How many watts does running an LLM at home take, and what does it cost? This page puts each GPU's rated board power next to our token-speed estimate, so you can see tokens per watt, energy per million tokens and electricity cost for 62 GPUs.

GPUs

62 with published board power

Board power

70–600 W

Example rate

$0.18/kWh not a sourced average

Short answer: electricity is a small cost per token (well under a dollar per million tokens on most cards for an 8B model). Board power matters more for your PSU, heat and noise than for your bill. The most efficient cards are the ones with the most memory bandwidth per watt.

## How we estimate it

Wh per 1M tokens = board power (W) ÷ tok/s × 1,000,000 ÷ 3,600 · cost = Wh ÷ 1,000 × $/kWh

Board power is the maker's rating for the reference card (NVIDIA Founders Edition, AMD reference). Partner cards can be higher. Real draw while generating tokens is often lower, so treat energy and cost as a worst case for the GPU alone. Speed is our bandwidth-based decode estimate ([how we estimate](https://whatcanirunai.vercel.app/method/#speed)). The $0.18/kWh rate is an example (the same default as the [planner](https://whatcanirunai.vercel.app/planner/)); multiply by your rate ÷ 0.18.

## Energy and cost per million tokens

### Llama-3.1-8B-Instruct at Q4

[Llama-3.1-8B-Instruct](https://whatcanirunai.vercel.app/models/llama-3.1-8b-instruct/) needs \~6.6 GB at Q4, so 61 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power.

__Llama-3.1-8B-Instruct: board power, estimated speed, energy and example electricity cost per million tokens by GPU__
| GPU                                                                                      | Board power | Est. tok/s | tok/s per 100 W | Wh per 1M tokens | Cost per 1M tokens | List price |
| ---------------------------------------------------------------------------------------- | ----------- | ---------- | --------------- | ---------------- | ------------------ | ---------- |
| [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)24 GB · 768 GB/s                                 | 230 W       | 95–134     | 49.7            | 476–675          | $0.09–$0.12        | $2,250     |
| [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)48 GB · 960 GB/s                      | 300 W       | 118–168    | 47.7            | 497–704          | $0.09–$0.13        | $6,799     |
| [NVIDIA RTX A4000](https://whatcanirunai.vercel.app/hardware/rtx-a4000/)16 GB · 448 GB/s                                 | 140 W       | 55–78      | 47.7            | 497–704          | $0.09–$0.13        | $1,000     |
| [NVIDIA RTX 2000 Ada 16GB](https://whatcanirunai.vercel.app/hardware/rtx-2000-ada/)16 GB · 224 GB/s                      | 70 W        | 28–39      | 47.7            | 497–704          | $0.09–$0.13        | $649       |
| [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)32 GB · 1792 GB/s                                 | 575 W       | 221–313    | 46.4            | 510–723          | $0.09–$0.13        | $1,999     |
| [GeForce RTX 5060](https://whatcanirunai.vercel.app/hardware/rtx-5060/)8 GB · 448 GB/s                                   | 145 W       | 55–78      | 46.0            | 515–729          | $0.09–$0.13        | $299       |
| [GeForce RTX 5070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-5070-ti/)16 GB · 896 GB/s                            | 300 W       | 110–156    | 44.5            | 533–755          | $0.10–$0.14        | $749       |
| [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)96 GB · 1792 GB/s | 600 W       | 221–313    | 44.5            | 533–755          | $0.10–$0.14        | $8,565     |
| [Radeon RX 9070](https://whatcanirunai.vercel.app/hardware/rx-9070/)16 GB · 640 GB/s                                     | 220 W       | 79–112     | 43.3            | 547–775          | $0.10–$0.14        | $549       |
| [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)20 GB · 360 GB/s                      | 130 W       | 44–63      | 41.2            | 575–814          | $0.10–$0.15        | $1,250     |
| [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)24 GB · 960 GB/s                             | 355 W       | 118–168    | 40.3            | 588–833          | $0.11–$0.15        | $999       |
| [GeForce RTX 5070](https://whatcanirunai.vercel.app/hardware/rtx-5070/)12 GB · 672 GB/s                                  | 250 W       | 83–117     | 40.0            | 592–838          | $0.11–$0.15        | $549       |
| [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)24 GB · 936 GB/s                                  | 350 W       | 115–163    | 39.8            | 595–843          | $0.11–$0.15        | $1,499     |
| [GeForce RTX 5080](https://whatcanirunai.vercel.app/hardware/rtx-5080/)16 GB · 960 GB/s                                  | 360 W       | 118–168    | 39.7            | 597–845          | $0.11–$0.15        | $999       |
| [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)32 GB · 608 GB/s                         | 230 W       | 75–106     | 39.4            | 602–853          | $0.11–$0.15        | $949       |
| [GeForce RTX 3080 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-12gb/)12 GB · 912 GB/s                        | 350 W       | 112–159    | 38.8            | 611–865          | $0.11–$0.16        | $799       |
| [GeForce RTX 2060 Super](https://whatcanirunai.vercel.app/hardware/rtx-2060-super/)8 GB · 448 GB/s                       | 175 W       | 55–78      | 38.1            | 621–880          | $0.11–$0.16        | $399       |
| [GeForce RTX 2070](https://whatcanirunai.vercel.app/hardware/rtx-2070/)8 GB · 448 GB/s                                   | 175 W       | 55–78      | 38.1            | 621–880          | $0.11–$0.16        | $499       |
| [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)48 GB · 768 GB/s                                 | 300 W       | 95–134     | 38.1            | 621–880          | $0.11–$0.16        | $4,649     |
| [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)20 GB · 800 GB/s                               | 315 W       | 99–140     | 37.8            | 626–887          | $0.11–$0.16        | $899       |
| [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)10 GB · 380 GB/s                                    | 150 W       | 47–66      | 37.7            | 628–890          | $0.11–$0.16        | $219       |
| [GeForce RTX 4070](https://whatcanirunai.vercel.app/hardware/rtx-4070/)12 GB · 504 GB/s                                  | 200 W       | 62–88      | 37.5            | 631–894          | $0.11–$0.16        | $599       |
| [GeForce RTX 5060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-8gb/)8 GB · 448 GB/s                     | 180 W       | 55–78      | 37.1            | 639–906          | $0.12–$0.16        | $379       |
| [GeForce RTX 5060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-16gb/)16 GB · 448 GB/s                  | 180 W       | 55–78      | 37.1            | 639–906          | $0.12–$0.16        | $429       |
| [Intel Arc A770 16GB](https://whatcanirunai.vercel.app/hardware/arc-a770-16gb/)16 GB · 560 GB/s                          | 225 W       | 69–98      | 37.1            | 639–906          | $0.12–$0.16        | $349       |
| [GeForce RTX 2080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-2080-ti/)11 GB · 616 GB/s                            | 250 W       | 76–108     | 36.7            | 646–915          | $0.12–$0.16        | $999       |
| [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)24 GB · 672 GB/s                                 | 280 W       | 83–117     | 35.7            | 663–939          | $0.12–$0.17        | $2,499     |
| [GeForce RTX 3080 10GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-10gb/)10 GB · 760 GB/s                        | 320 W       | 94–133     | 35.4            | 670–949          | $0.12–$0.17        | $699       |
| [Radeon RX 7800 XT](https://whatcanirunai.vercel.app/hardware/rx-7800-xt/)16 GB · 624 GB/s                               | 263 W       | 77–109     | 35.3            | 671–950          | $0.12–$0.17        | $499       |
| [GeForce RTX 4060](https://whatcanirunai.vercel.app/hardware/rtx-4060/)8 GB · 272 GB/s                                   | 115 W       | 34–47      | 35.2            | 673–953          | $0.12–$0.17        | $299       |
| [GeForce RTX 4070 Ti Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti-super/)16 GB · 672 GB/s                | 285 W       | 83–117     | 35.1            | 675–956          | $0.12–$0.17        | $799       |
| [GeForce RTX 4080 Super](https://whatcanirunai.vercel.app/hardware/rtx-4080-super/)16 GB · 736 GB/s                      | 320 W       | 91–129     | 34.3            | 692–980          | $0.12–$0.18        | $999       |
| [GeForce RTX 4070 Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-super/)12 GB · 504 GB/s                      | 220 W       | 62–88      | 34.1            | 694–984          | $0.13–$0.18        | $599       |
| [Intel Arc A750](https://whatcanirunai.vercel.app/hardware/arc-a750/)8 GB · 512 GB/s                                     | 225 W       | 63–89      | 33.9            | 699–990          | $0.13–$0.18        | $289       |
| [GeForce RTX 4080](https://whatcanirunai.vercel.app/hardware/rtx-4080/)16 GB · 717 GB/s                                  | 320 W       | 88–125     | 33.4            | 710–1,006        | $0.13–$0.18        | $1,199     |
| [GeForce RTX 3060 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3060-ti/)8 GB · 448 GB/s                             | 200 W       | 55–78      | 33.4            | 710–1,006        | $0.13–$0.18        | $399       |
| [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)24 GB · 1008 GB/s                           | 450 W       | 124–176    | 33.4            | 710–1,006        | $0.13–$0.18        | $1,999     |
| [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)24 GB · 1008 GB/s                                 | 450 W       | 124–176    | 33.4            | 710–1,006        | $0.13–$0.18        | $1,599     |
| [Radeon RX 7900 GRE](https://whatcanirunai.vercel.app/hardware/rx-7900-gre/)16 GB · 576 GB/s                             | 260 W       | 71–101     | 33.0            | 718–1,017        | $0.13–$0.18        | $549       |
| [Radeon RX 9060 XT 8GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-8gb/)8 GB · 320 GB/s                        | 150 W       | 39–56      | 31.8            | 746–1,056        | $0.13–$0.19        | $299       |
| [GeForce RTX 3060 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3060-12gb/)12 GB · 360 GB/s                        | 170 W       | 44–63      | 31.5            | 751–1,064        | $0.14–$0.19        | $329       |
| [Radeon RX 9070 XT](https://whatcanirunai.vercel.app/hardware/rx-9070-xt/)16 GB · 640 GB/s                               | 304 W       | 79–112     | 31.4            | 756–1,071        | $0.14–$0.19        | $599       |
| [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/)8 GB · 608 GB/s                             | 290 W       | 75–106     | 31.2            | 759–1,075        | $0.14–$0.19        | $599       |
| [GeForce RTX 2070 Super](https://whatcanirunai.vercel.app/hardware/rtx-2070-super/)8 GB · 448 GB/s                       | 215 W       | 55–78      | 31.0            | 764–1,082        | $0.14–$0.19        | $499       |
| [GeForce RTX 2080](https://whatcanirunai.vercel.app/hardware/rtx-2080/)8 GB · 448 GB/s                                   | 215 W       | 55–78      | 31.0            | 764–1,082        | $0.14–$0.19        | $699       |
| [Radeon RX 6800](https://whatcanirunai.vercel.app/hardware/rx-6800/)16 GB · 512 GB/s                                     | 250 W       | 63–89      | 30.5            | 777–1,101        | $0.14–$0.20        | $579       |
| [GeForce RTX 3070](https://whatcanirunai.vercel.app/hardware/rtx-3070/)8 GB · 448 GB/s                                   | 220 W       | 55–78      | 30.3            | 781–1,107        | $0.14–$0.20        | $499       |
| [Radeon RX 9060 XT 16GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-16gb/)16 GB · 320 GB/s                     | 160 W       | 39–56      | 29.8            | 795–1,127        | $0.14–$0.20        | $349       |
| [GeForce RTX 2080 Super](https://whatcanirunai.vercel.app/hardware/rtx-2080-super/)8 GB · 496 GB/s                       | 250 W       | 61–87      | 29.5            | 802–1,136        | $0.14–$0.20        | $699       |
| [GeForce RTX 4060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-8gb/)8 GB · 288 GB/s                     | 160 W       | 35–50      | 26.8            | 884–1,252        | $0.16–$0.23        | $399       |
| [GeForce RTX 4070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti/)12 GB · 504 GB/s                            | 285 W       | 62–88      | 26.3            | 900–1,275        | $0.16–$0.23        | $799       |
| [Radeon RX 7700 XT](https://whatcanirunai.vercel.app/hardware/rx-7700-xt/)12 GB · 432 GB/s                               | 245 W       | 53–75      | 26.3            | 902–1,278        | $0.16–$0.23        | $449       |
| [GeForce RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-16gb/)16 GB · 288 GB/s                  | 165 W       | 35–50      | 26.0            | 911–1,291        | $0.16–$0.23        | $499       |
| [Radeon RX 7600](https://whatcanirunai.vercel.app/hardware/rx-7600/)8 GB · 288 GB/s                                      | 165 W       | 35–50      | 26.0            | 911–1,291        | $0.16–$0.23        | $269       |
| [Radeon RX 6750 XT](https://whatcanirunai.vercel.app/hardware/rx-6750-xt/)12 GB · 432 GB/s                               | 250 W       | 53–75      | 25.7            | 921–1,304        | $0.17–$0.23        | $549       |
| [GeForce RTX 3050 8GB](https://whatcanirunai.vercel.app/hardware/rtx-3050-8gb/)8 GB · 224 GB/s                           | 130 W       | 28–39      | 25.7            | 923–1,308        | $0.17–$0.24        | $249       |
| [Radeon RX 6950 XT](https://whatcanirunai.vercel.app/hardware/rx-6950-xt/)16 GB · 576 GB/s                               | 335 W       | 71–101     | 25.6            | 925–1,311        | $0.17–$0.24        | $1,099     |
| [Radeon RX 6800 XT](https://whatcanirunai.vercel.app/hardware/rx-6800-xt/)16 GB · 512 GB/s                               | 300 W       | 63–89      | 25.4            | 932–1,321        | $0.17–$0.24        | $649       |
| [Radeon RX 6900 XT](https://whatcanirunai.vercel.app/hardware/rx-6900-xt/)16 GB · 512 GB/s                               | 300 W       | 63–89      | 25.4            | 932–1,321        | $0.17–$0.24        | $999       |
| [Radeon RX 6700 XT](https://whatcanirunai.vercel.app/hardware/rx-6700-xt/)12 GB · 384 GB/s                               | 230 W       | 47–67      | 24.9            | 953–1,350        | $0.17–$0.24        | $479       |
| [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)16 GB · 288 GB/s                               | 190 W       | 35–50      | 22.6            | 1,050–1,487      | $0.19–$0.27        | $329       |

### Qwen3-14B at Q4

[Qwen3-14B](https://whatcanirunai.vercel.app/models/qwen3-14b/) needs \~11.3 GB at Q4, so 33 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power.

__Qwen3-14B: board power, estimated speed, energy and example electricity cost per million tokens by GPU__
| GPU                                                                                      | Board power | Est. tok/s | tok/s per 100 W | Wh per 1M tokens | Cost per 1M tokens | List price |
| ---------------------------------------------------------------------------------------- | ----------- | ---------- | --------------- | ---------------- | ------------------ | ---------- |
| [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)24 GB · 768 GB/s                                 | 230 W       | 51–73      | 27.0            | 876–1,241        | $0.16–$0.22        | $2,250     |
| [NVIDIA RTX A4000](https://whatcanirunai.vercel.app/hardware/rtx-a4000/)16 GB · 448 GB/s                                 | 140 W       | 30–43      | 25.9            | 914–1,295        | $0.16–$0.23        | $1,000     |
| [NVIDIA RTX 2000 Ada 16GB](https://whatcanirunai.vercel.app/hardware/rtx-2000-ada/)16 GB · 224 GB/s                      | 70 W        | 15–21      | 25.9            | 914–1,295        | $0.16–$0.23        | $649       |
| [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)48 GB · 960 GB/s                      | 300 W       | 64–91      | 25.9            | 914–1,295        | $0.16–$0.23        | $6,799     |
| [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)32 GB · 1792 GB/s                                 | 575 W       | 120–170    | 25.2            | 939–1,330        | $0.17–$0.24        | $1,999     |
| [GeForce RTX 5070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-5070-ti/)16 GB · 896 GB/s                            | 300 W       | 60–85      | 24.2            | 980–1,388        | $0.18–$0.25        | $749       |
| [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)96 GB · 1792 GB/s | 600 W       | 120–170    | 24.2            | 980–1,388        | $0.18–$0.25        | $8,565     |
| [Radeon RX 9070](https://whatcanirunai.vercel.app/hardware/rx-9070/)16 GB · 640 GB/s                                     | 220 W       | 43–61      | 23.6            | 1,006–1,425      | $0.18–$0.26        | $549       |
| [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)20 GB · 360 GB/s                      | 130 W       | 24–34      | 22.4            | 1,057–1,497      | $0.19–$0.27        | $1,250     |
| [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)24 GB · 960 GB/s                             | 355 W       | 64–91      | 21.9            | 1,082–1,533      | $0.19–$0.28        | $999       |
| [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)24 GB · 936 GB/s                                  | 350 W       | 63–89      | 21.7            | 1,094–1,550      | $0.20–$0.28        | $1,499     |
| [GeForce RTX 5080](https://whatcanirunai.vercel.app/hardware/rtx-5080/)16 GB · 960 GB/s                                  | 360 W       | 64–91      | 21.6            | 1,097–1,554      | $0.20–$0.28        | $999       |
| [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)32 GB · 608 GB/s                         | 230 W       | 41–58      | 21.4            | 1,107–1,568      | $0.20–$0.28        | $949       |
| [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)48 GB · 768 GB/s                                 | 300 W       | 51–73      | 20.7            | 1,143–1,619      | $0.21–$0.29        | $4,649     |
| [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)20 GB · 800 GB/s                               | 315 W       | 54–76      | 20.6            | 1,152–1,632      | $0.21–$0.29        | $899       |
| [GeForce RTX 5060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-16gb/)16 GB · 448 GB/s                  | 180 W       | 30–43      | 20.2            | 1,176–1,665      | $0.21–$0.30        | $429       |
| [Intel Arc A770 16GB](https://whatcanirunai.vercel.app/hardware/arc-a770-16gb/)16 GB · 560 GB/s                          | 225 W       | 38–53      | 20.2            | 1,176–1,665      | $0.21–$0.30        | $349       |
| [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)24 GB · 672 GB/s                                 | 280 W       | 45–64      | 19.4            | 1,219–1,727      | $0.22–$0.31        | $2,499     |
| [Radeon RX 7800 XT](https://whatcanirunai.vercel.app/hardware/rx-7800-xt/)16 GB · 624 GB/s                               | 263 W       | 42–59      | 19.2            | 1,233–1,747      | $0.22–$0.31        | $499       |
| [GeForce RTX 4070 Ti Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti-super/)16 GB · 672 GB/s                | 285 W       | 45–64      | 19.1            | 1,241–1,758      | $0.22–$0.32        | $799       |
| [GeForce RTX 4080 Super](https://whatcanirunai.vercel.app/hardware/rtx-4080-super/)16 GB · 736 GB/s                      | 320 W       | 49–70      | 18.6            | 1,272–1,802      | $0.23–$0.32        | $999       |
| [GeForce RTX 4080](https://whatcanirunai.vercel.app/hardware/rtx-4080/)16 GB · 717 GB/s                                  | 320 W       | 48–68      | 18.1            | 1,306–1,850      | $0.24–$0.33        | $1,199     |
| [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)24 GB · 1008 GB/s                           | 450 W       | 68–96      | 18.1            | 1,306–1,850      | $0.24–$0.33        | $1,999     |
| [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)24 GB · 1008 GB/s                                 | 450 W       | 68–96      | 18.1            | 1,306–1,850      | $0.24–$0.33        | $1,599     |
| [Radeon RX 7900 GRE](https://whatcanirunai.vercel.app/hardware/rx-7900-gre/)16 GB · 576 GB/s                             | 260 W       | 39–55      | 17.9            | 1,321–1,871      | $0.24–$0.34        | $549       |
| [Radeon RX 9070 XT](https://whatcanirunai.vercel.app/hardware/rx-9070-xt/)16 GB · 640 GB/s                               | 304 W       | 43–61      | 17.0            | 1,390–1,969      | $0.25–$0.35        | $599       |
| [Radeon RX 6800](https://whatcanirunai.vercel.app/hardware/rx-6800/)16 GB · 512 GB/s                                     | 250 W       | 34–49      | 16.6            | 1,429–2,024      | $0.26–$0.36        | $579       |
| [Radeon RX 9060 XT 16GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-16gb/)16 GB · 320 GB/s                     | 160 W       | 21–30      | 16.2            | 1,463–2,073      | $0.26–$0.37        | $349       |
| [GeForce RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-16gb/)16 GB · 288 GB/s                  | 165 W       | 19–27      | 14.1            | 1,676–2,375      | $0.30–$0.43        | $499       |
| [Radeon RX 6950 XT](https://whatcanirunai.vercel.app/hardware/rx-6950-xt/)16 GB · 576 GB/s                               | 335 W       | 39–55      | 13.9            | 1,702–2,411      | $0.31–$0.43        | $1,099     |
| [Radeon RX 6800 XT](https://whatcanirunai.vercel.app/hardware/rx-6800-xt/)16 GB · 512 GB/s                               | 300 W       | 34–49      | 13.8            | 1,714–2,429      | $0.31–$0.44        | $649       |
| [Radeon RX 6900 XT](https://whatcanirunai.vercel.app/hardware/rx-6900-xt/)16 GB · 512 GB/s                               | 300 W       | 34–49      | 13.8            | 1,714–2,429      | $0.31–$0.44        | $999       |
| [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)16 GB · 288 GB/s                               | 190 W       | 19–27      | 12.3            | 1,930–2,735      | $0.35–$0.49        | $329       |

### gpt-oss-20b at Q4

[gpt-oss-20b](https://whatcanirunai.vercel.app/models/gpt-oss-20b/) needs \~15.6 GB at Q4, so 13 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power. MoE speeds are upper bounds.

__gpt-oss-20b: board power, estimated speed, energy and example electricity cost per million tokens by GPU__
| GPU                                                                                      | Board power | Est. tok/s | tok/s per 100 W | Wh per 1M tokens | Cost per 1M tokens | List price |
| ---------------------------------------------------------------------------------------- | ----------- | ---------- | --------------- | ---------------- | ------------------ | ---------- |
| [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)24 GB · 768 GB/s                                 | 230 W       | up to 299  | 130.0           | 214+             | $0.04+             | $2,250     |
| [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)48 GB · 960 GB/s                      | 300 W       | up to 374  | 124.6           | 223+             | $0.04+             | $6,799     |
| [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)32 GB · 1792 GB/s                                 | 575 W       | up to 698  | 121.4           | 229+             | $0.04+             | $1,999     |
| [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)96 GB · 1792 GB/s | 600 W       | up to 698  | 116.3           | 239+             | $0.04+             | $8,565     |
| [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)20 GB · 360 GB/s                      | 130 W       | up to 140  | 107.9           | 258+             | $0.05+             | $1,250     |
| [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)24 GB · 960 GB/s                             | 355 W       | up to 374  | 105.3           | 264+             | $0.05+             | $999       |
| [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)24 GB · 936 GB/s                                  | 350 W       | up to 365  | 104.2           | 267+             | $0.05+             | $1,499     |
| [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)32 GB · 608 GB/s                         | 230 W       | up to 237  | 103.0           | 270+             | $0.05+             | $949       |
| [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)48 GB · 768 GB/s                                 | 300 W       | up to 299  | 99.7            | 279+             | $0.05+             | $4,649     |
| [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)20 GB · 800 GB/s                               | 315 W       | up to 312  | 98.9            | 281+             | $0.05+             | $899       |
| [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)24 GB · 672 GB/s                                 | 280 W       | up to 262  | 93.5            | 297+             | $0.05+             | $2,499     |
| [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)24 GB · 1008 GB/s                           | 450 W       | up to 393  | 87.2            | 318+             | $0.06+             | $1,999     |
| [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)24 GB · 1008 GB/s                                 | 450 W       | up to 393  | 87.2            | 318+             | $0.06+             | $1,599     |

## Cost per hour at full board power

Worst case for the GPU alone, at $0.18/kWh. "Per year" assumes 8 hours a day at full board power every day.

__GPU board power and example electricity cost per hour, per 8-hour day and per year__
| GPU                                                                                   | Board power | Per hour | Per 8 h day | Per year (8 h/day) |
| ------------------------------------------------------------------------------------- | ----------- | -------- | ----------- | ------------------ |
| [NVIDIA RTX 2000 Ada 16GB](https://whatcanirunai.vercel.app/hardware/rtx-2000-ada/)16 GB · $649                       | 70 W        | $0.01    | $0.10       | $37                |
| [GeForce RTX 4060](https://whatcanirunai.vercel.app/hardware/rtx-4060/)8 GB · $299                                    | 115 W       | $0.02    | $0.17       | $60                |
| [GeForce RTX 3050 8GB](https://whatcanirunai.vercel.app/hardware/rtx-3050-8gb/)8 GB · $249                            | 130 W       | $0.02    | $0.19       | $68                |
| [NVIDIA RTX 4000 Ada 20GB](https://whatcanirunai.vercel.app/hardware/rtx-4000-ada/)20 GB · $1,250                     | 130 W       | $0.02    | $0.19       | $68                |
| [NVIDIA RTX A4000](https://whatcanirunai.vercel.app/hardware/rtx-a4000/)16 GB · $1,000                                | 140 W       | $0.03    | $0.20       | $74                |
| [GeForce RTX 5060](https://whatcanirunai.vercel.app/hardware/rtx-5060/)8 GB · $299                                    | 145 W       | $0.03    | $0.21       | $76                |
| [Radeon RX 9060 XT 8GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-8gb/)8 GB · $299                         | 150 W       | $0.03    | $0.22       | $79                |
| [Intel Arc B570](https://whatcanirunai.vercel.app/hardware/arc-b570/)10 GB · $219                                     | 150 W       | $0.03    | $0.22       | $79                |
| [GeForce RTX 2060 6GB](https://whatcanirunai.vercel.app/hardware/rtx-2060/)6 GB · $349                                | 160 W       | $0.03    | $0.23       | $84                |
| [GeForce RTX 4060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-8gb/)8 GB · $399                      | 160 W       | $0.03    | $0.23       | $84                |
| [Radeon RX 9060 XT 16GB](https://whatcanirunai.vercel.app/hardware/rx-9060-xt-16gb/)16 GB · $349                      | 160 W       | $0.03    | $0.23       | $84                |
| [GeForce RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-4060-ti-16gb/)16 GB · $499                   | 165 W       | $0.03    | $0.24       | $87                |
| [Radeon RX 7600](https://whatcanirunai.vercel.app/hardware/rx-7600/)8 GB · $269                                       | 165 W       | $0.03    | $0.24       | $87                |
| [GeForce RTX 3060 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3060-12gb/)12 GB · $329                         | 170 W       | $0.03    | $0.24       | $89                |
| [GeForce RTX 2060 Super](https://whatcanirunai.vercel.app/hardware/rtx-2060-super/)8 GB · $399                        | 175 W       | $0.03    | $0.25       | $92                |
| [GeForce RTX 2070](https://whatcanirunai.vercel.app/hardware/rtx-2070/)8 GB · $499                                    | 175 W       | $0.03    | $0.25       | $92                |
| [GeForce RTX 5060 Ti 8GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-8gb/)8 GB · $379                      | 180 W       | $0.03    | $0.26       | $95                |
| [GeForce RTX 5060 Ti 16GB](https://whatcanirunai.vercel.app/hardware/rtx-5060-ti-16gb/)16 GB · $429                   | 180 W       | $0.03    | $0.26       | $95                |
| [Radeon RX 7600 XT](https://whatcanirunai.vercel.app/hardware/rx-7600-xt/)16 GB · $329                                | 190 W       | $0.03    | $0.27       | $100               |
| [GeForce RTX 3060 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3060-ti/)8 GB · $399                              | 200 W       | $0.04    | $0.29       | $105               |
| [GeForce RTX 4070](https://whatcanirunai.vercel.app/hardware/rtx-4070/)12 GB · $599                                   | 200 W       | $0.04    | $0.29       | $105               |
| [GeForce RTX 2070 Super](https://whatcanirunai.vercel.app/hardware/rtx-2070-super/)8 GB · $499                        | 215 W       | $0.04    | $0.31       | $113               |
| [GeForce RTX 2080](https://whatcanirunai.vercel.app/hardware/rtx-2080/)8 GB · $699                                    | 215 W       | $0.04    | $0.31       | $113               |
| [GeForce RTX 3070](https://whatcanirunai.vercel.app/hardware/rtx-3070/)8 GB · $499                                    | 220 W       | $0.04    | $0.32       | $116               |
| [GeForce RTX 4070 Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-super/)12 GB · $599                       | 220 W       | $0.04    | $0.32       | $116               |
| [Radeon RX 9070](https://whatcanirunai.vercel.app/hardware/rx-9070/)16 GB · $549                                      | 220 W       | $0.04    | $0.32       | $116               |
| [Intel Arc A750](https://whatcanirunai.vercel.app/hardware/arc-a750/)8 GB · $289                                      | 225 W       | $0.04    | $0.32       | $118               |
| [Intel Arc A770 16GB](https://whatcanirunai.vercel.app/hardware/arc-a770-16gb/)16 GB · $349                           | 225 W       | $0.04    | $0.32       | $118               |
| [Intel Arc Pro B70 32GB](https://whatcanirunai.vercel.app/hardware/arc-pro-b70/)32 GB · $949                          | 230 W       | $0.04    | $0.33       | $121               |
| [Radeon RX 6700 XT](https://whatcanirunai.vercel.app/hardware/rx-6700-xt/)12 GB · $479                                | 230 W       | $0.04    | $0.33       | $121               |
| [NVIDIA RTX A5000](https://whatcanirunai.vercel.app/hardware/rtx-a5000/)24 GB · $2,250                                | 230 W       | $0.04    | $0.33       | $121               |
| [Radeon RX 7700 XT](https://whatcanirunai.vercel.app/hardware/rx-7700-xt/)12 GB · $449                                | 245 W       | $0.04    | $0.35       | $129               |
| [GeForce RTX 5070](https://whatcanirunai.vercel.app/hardware/rtx-5070/)12 GB · $549                                   | 250 W       | $0.04    | $0.36       | $131               |
| [GeForce RTX 2080 Super](https://whatcanirunai.vercel.app/hardware/rtx-2080-super/)8 GB · $699                        | 250 W       | $0.04    | $0.36       | $131               |
| [GeForce RTX 2080 Ti](https://whatcanirunai.vercel.app/hardware/rtx-2080-ti/)11 GB · $999                             | 250 W       | $0.04    | $0.36       | $131               |
| [Radeon RX 6750 XT](https://whatcanirunai.vercel.app/hardware/rx-6750-xt/)12 GB · $549                                | 250 W       | $0.04    | $0.36       | $131               |
| [Radeon RX 6800](https://whatcanirunai.vercel.app/hardware/rx-6800/)16 GB · $579                                      | 250 W       | $0.04    | $0.36       | $131               |
| [Radeon RX 7900 GRE](https://whatcanirunai.vercel.app/hardware/rx-7900-gre/)16 GB · $549                              | 260 W       | $0.05    | $0.37       | $137               |
| [Radeon RX 7800 XT](https://whatcanirunai.vercel.app/hardware/rx-7800-xt/)16 GB · $499                                | 263 W       | $0.05    | $0.38       | $138               |
| [NVIDIA Titan RTX](https://whatcanirunai.vercel.app/hardware/titan-rtx/)24 GB · $2,499                                | 280 W       | $0.05    | $0.40       | $147               |
| [GeForce RTX 4070 Ti Super](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti-super/)16 GB · $799                 | 285 W       | $0.05    | $0.41       | $150               |
| [GeForce RTX 4070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-4070-ti/)12 GB · $799                             | 285 W       | $0.05    | $0.41       | $150               |
| [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/)8 GB · $599                              | 290 W       | $0.05    | $0.42       | $152               |
| [GeForce RTX 5070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-5070-ti/)16 GB · $749                             | 300 W       | $0.05    | $0.43       | $158               |
| [Radeon RX 6800 XT](https://whatcanirunai.vercel.app/hardware/rx-6800-xt/)16 GB · $649                                | 300 W       | $0.05    | $0.43       | $158               |
| [Radeon RX 6900 XT](https://whatcanirunai.vercel.app/hardware/rx-6900-xt/)16 GB · $999                                | 300 W       | $0.05    | $0.43       | $158               |
| [NVIDIA RTX A6000](https://whatcanirunai.vercel.app/hardware/rtx-a6000/)48 GB · $4,649                                | 300 W       | $0.05    | $0.43       | $158               |
| [NVIDIA RTX 6000 Ada 48GB](https://whatcanirunai.vercel.app/hardware/rtx-6000-ada/)48 GB · $6,799                     | 300 W       | $0.05    | $0.43       | $158               |
| [Radeon RX 9070 XT](https://whatcanirunai.vercel.app/hardware/rx-9070-xt/)16 GB · $599                                | 304 W       | $0.05    | $0.44       | $160               |
| [Radeon RX 7900 XT](https://whatcanirunai.vercel.app/hardware/rx-7900-xt/)20 GB · $899                                | 315 W       | $0.06    | $0.45       | $166               |
| [GeForce RTX 3080 10GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-10gb/)10 GB · $699                         | 320 W       | $0.06    | $0.46       | $168               |
| [GeForce RTX 4080 Super](https://whatcanirunai.vercel.app/hardware/rtx-4080-super/)16 GB · $999                       | 320 W       | $0.06    | $0.46       | $168               |
| [GeForce RTX 4080](https://whatcanirunai.vercel.app/hardware/rtx-4080/)16 GB · $1,199                                 | 320 W       | $0.06    | $0.46       | $168               |
| [Radeon RX 6950 XT](https://whatcanirunai.vercel.app/hardware/rx-6950-xt/)16 GB · $1,099                              | 335 W       | $0.06    | $0.48       | $176               |
| [GeForce RTX 3090](https://whatcanirunai.vercel.app/hardware/rtx-3090/)24 GB · $1,499                                 | 350 W       | $0.06    | $0.50       | $184               |
| [GeForce RTX 3080 12GB](https://whatcanirunai.vercel.app/hardware/rtx-3080-12gb/)12 GB · $799                         | 350 W       | $0.06    | $0.50       | $184               |
| [Radeon RX 7900 XTX](https://whatcanirunai.vercel.app/hardware/rx-7900-xtx/)24 GB · $999                              | 355 W       | $0.06    | $0.51       | $187               |
| [GeForce RTX 5080](https://whatcanirunai.vercel.app/hardware/rtx-5080/)16 GB · $999                                   | 360 W       | $0.06    | $0.52       | $189               |
| [GeForce RTX 3090 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3090-ti/)24 GB · $1,999                           | 450 W       | $0.08    | $0.65       | $237               |
| [GeForce RTX 4090](https://whatcanirunai.vercel.app/hardware/rtx-4090/)24 GB · $1,599                                 | 450 W       | $0.08    | $0.65       | $237               |
| [GeForce RTX 5090](https://whatcanirunai.vercel.app/hardware/rtx-5090/)32 GB · $1,999                                 | 575 W       | $0.10    | $0.83       | $302               |
| [NVIDIA RTX Pro 6000 Blackwell 96GB](https://whatcanirunai.vercel.app/hardware/rtx-pro-6000-blackwell/)96 GB · $8,565 | 600 W       | $0.11    | $0.86       | $315               |

Running two or more cards? Multiply by the count: two GeForce RTX 5090s are 1150 W of board power. See [multi-GPU for local LLMs](https://whatcanirunai.vercel.app/multi-gpu-for-local-llm/) and the PSU math in the [planner](https://whatcanirunai.vercel.app/planner/).

## Quick answers

How much power does a GPU use to run a local LLM?

At most its rated board power: from 70 W (NVIDIA RTX 2000 Ada 16GB) to 600 W (NVIDIA RTX Pro 6000 Blackwell 96GB) for the cards we track. Token generation is limited by memory bandwidth rather than compute, so cards often draw less than that during chat. We use board power as a worst case. Add the rest of the PC (CPU, board, drives) on top.

How much does it cost in electricity to run a local LLM?

Very little per token. For Llama-3.1-8B-Instruct at Q4, our worst-case estimate runs from $0.09–$0.12 per million tokens on the NVIDIA RTX A5000 to $0.19–$0.27 on the Radeon RX 7600 XT, at an example $0.18/kWh. Scale linearly for your own rate.

Which GPU is the most power-efficient for local LLMs?

For an 8B model, the NVIDIA RTX A5000 gives the most estimated tokens per second per watt of board power (49.7 tok/s per 100 W). Efficiency follows memory bandwidth per watt of board power, not card age. For bigger models only cards with enough VRAM count, so check the 14B and gpt-oss-20b tables.

Is a used RTX 3090 power hungry?

Its board power is 350 W, about 3.0× an RTX 4060's 115 W. But it has 24 GB and 936 GB/s, so per token it is far more efficient than a small card on models only it can fit.

Can I power-limit my GPU for LLM inference?

Yes. On NVIDIA, nvidia-smi -pl sets a lower power limit; AMD cards can be limited in their driver software. Because decode is memory-bound, a lower limit may cost little speed, but results vary by card, so measure tok/s and draw before and after. We don't model power limits.

Why aren't Macs and mini PCs on this page?

We only list board power that the maker publishes for the card (NVIDIA Founders Edition and AMD reference specs). We haven't sourced comparable LLM-load power figures for Macs, Ryzen AI Max machines or DGX Spark yet, so they are left out rather than guessed. GeForce RTX 3080 Ti and Intel Arc B580 are also left out for now.

[Tokens per second](https://whatcanirunai.vercel.app/tokens-per-second/) · [Best GPU for local LLMs](https://whatcanirunai.vercel.app/best-gpu-for-local-llm/) · [Multi-GPU](https://whatcanirunai.vercel.app/multi-gpu-for-local-llm/) · [Q4 vs Q8 vs FP16](https://whatcanirunai.vercel.app/quantization/) · [MoE vs dense](https://whatcanirunai.vercel.app/moe-vs-dense/) · [Planner](https://whatcanirunai.vercel.app/planner/)

---
Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/