How much power does a GPU use to run a local LLM?

At most its rated board power: from 70 W (NVIDIA RTX 2000 Ada 16GB) to 600 W (NVIDIA RTX Pro 6000 Blackwell 96GB) for the cards we track. Token generation is limited by memory bandwidth rather than compute, so cards often draw less than that during chat. We use board power as a worst case. Add the rest of the PC (CPU, board, drives) on top.

How much does it cost in electricity to run a local LLM?

Very little per token. For Llama-3.1-8B-Instruct at Q4, our worst-case estimate runs from $0.09–$0.12 per million tokens on the NVIDIA RTX A5000 to $0.19–$0.27 on the Radeon RX 7600 XT, at an example $0.18/kWh. Scale linearly for your own rate.

Which GPU is the most power-efficient for local LLMs?

For an 8B model, the NVIDIA RTX A5000 gives the most estimated tokens per second per watt of board power (49.7 tok/s per 100 W). Efficiency follows memory bandwidth per watt of board power, not card age. For bigger models only cards with enough VRAM count, so check the 14B and gpt-oss-20b tables.

Is a used RTX 3090 power hungry?

Its board power is 350 W, about 3.0× an RTX 4060's 115 W. But it has 24 GB and 936 GB/s, so per token it is far more efficient than a small card on models only it can fit.

Can I power-limit my GPU for LLM inference?

Yes. On NVIDIA, nvidia-smi -pl sets a lower power limit; AMD cards can be limited in their driver software. Because decode is memory-bound, a lower limit may cost little speed, but results vary by card, so measure tok/s and draw before and after. We don't model power limits.

Why aren't Macs and mini PCs on this page?

We only list board power that the maker publishes for the card (NVIDIA Founders Edition and AMD reference specs). We haven't sourced comparable LLM-load power figures for Macs, Ryzen AI Max machines or DGX Spark yet, so they are left out rather than guessed. GeForce RTX 3080 Ti and Intel Arc B580 are also left out for now.

GPU power consumption for local LLMs

How many watts does running an LLM at home take, and what does it cost? This page puts each GPU's rated board power next to our token-speed estimate, so you can see tokens per watt, energy per million tokens and electricity cost for 62 GPUs.

GPUs: 62 with published board power
Board power: 70–600 W
Example rate: $0.18/kWh not a sourced average

Short answer: electricity is a small cost per token (well under a dollar per million tokens on most cards for an 8B model). Board power matters more for your PSU, heat and noise than for your bill. The most efficient cards are the ones with the most memory bandwidth per watt.

How we estimate it

Wh per 1M tokens = board power (W) ÷ tok/s × 1,000,000 ÷ 3,600 · cost = Wh ÷ 1,000 × $/kWh

Board power is the maker's rating for the reference card (NVIDIA Founders Edition, AMD reference). Partner cards can be higher. Real draw while generating tokens is often lower, so treat energy and cost as a worst case for the GPU alone. Speed is our bandwidth-based decode estimate (how we estimate). The $0.18/kWh rate is an example (the same default as the planner); multiply by your rate ÷ 0.18.

Energy and cost per million tokens

Llama-3.1-8B-Instruct at Q4

Llama-3.1-8B-Instruct needs ~6.6 GB at Q4, so 61 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power.

Llama-3.1-8B-Instruct: board power, estimated speed, energy and example electricity cost per million tokens by GPU
GPU	Board power	Est. tok/s	tok/s per 100 W	Wh per 1M tokens	Cost per 1M tokens	List price
NVIDIA RTX A500024 GB · 768 GB/s	230 W	95–134	49.7	476–675	$0.09–$0.12	$2,250
NVIDIA RTX 6000 Ada 48GB48 GB · 960 GB/s	300 W	118–168	47.7	497–704	$0.09–$0.13	$6,799
NVIDIA RTX A400016 GB · 448 GB/s	140 W	55–78	47.7	497–704	$0.09–$0.13	$1,000
NVIDIA RTX 2000 Ada 16GB16 GB · 224 GB/s	70 W	28–39	47.7	497–704	$0.09–$0.13	$649
GeForce RTX 509032 GB · 1792 GB/s	575 W	221–313	46.4	510–723	$0.09–$0.13	$1,999
GeForce RTX 50608 GB · 448 GB/s	145 W	55–78	46.0	515–729	$0.09–$0.13	$299
GeForce RTX 5070 Ti16 GB · 896 GB/s	300 W	110–156	44.5	533–755	$0.10–$0.14	$749
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB · 1792 GB/s	600 W	221–313	44.5	533–755	$0.10–$0.14	$8,565
Radeon RX 907016 GB · 640 GB/s	220 W	79–112	43.3	547–775	$0.10–$0.14	$549
NVIDIA RTX 4000 Ada 20GB20 GB · 360 GB/s	130 W	44–63	41.2	575–814	$0.10–$0.15	$1,250
Radeon RX 7900 XTX24 GB · 960 GB/s	355 W	118–168	40.3	588–833	$0.11–$0.15	$999
GeForce RTX 507012 GB · 672 GB/s	250 W	83–117	40.0	592–838	$0.11–$0.15	$549
GeForce RTX 309024 GB · 936 GB/s	350 W	115–163	39.8	595–843	$0.11–$0.15	$1,499
GeForce RTX 508016 GB · 960 GB/s	360 W	118–168	39.7	597–845	$0.11–$0.15	$999
Intel Arc Pro B70 32GB32 GB · 608 GB/s	230 W	75–106	39.4	602–853	$0.11–$0.15	$949
GeForce RTX 3080 12GB12 GB · 912 GB/s	350 W	112–159	38.8	611–865	$0.11–$0.16	$799
GeForce RTX 2060 Super8 GB · 448 GB/s	175 W	55–78	38.1	621–880	$0.11–$0.16	$399
GeForce RTX 20708 GB · 448 GB/s	175 W	55–78	38.1	621–880	$0.11–$0.16	$499
NVIDIA RTX A600048 GB · 768 GB/s	300 W	95–134	38.1	621–880	$0.11–$0.16	$4,649
Radeon RX 7900 XT20 GB · 800 GB/s	315 W	99–140	37.8	626–887	$0.11–$0.16	$899
Intel Arc B57010 GB · 380 GB/s	150 W	47–66	37.7	628–890	$0.11–$0.16	$219
GeForce RTX 407012 GB · 504 GB/s	200 W	62–88	37.5	631–894	$0.11–$0.16	$599
GeForce RTX 5060 Ti 8GB8 GB · 448 GB/s	180 W	55–78	37.1	639–906	$0.12–$0.16	$379
GeForce RTX 5060 Ti 16GB16 GB · 448 GB/s	180 W	55–78	37.1	639–906	$0.12–$0.16	$429
Intel Arc A770 16GB16 GB · 560 GB/s	225 W	69–98	37.1	639–906	$0.12–$0.16	$349
GeForce RTX 2080 Ti11 GB · 616 GB/s	250 W	76–108	36.7	646–915	$0.12–$0.16	$999
NVIDIA Titan RTX24 GB · 672 GB/s	280 W	83–117	35.7	663–939	$0.12–$0.17	$2,499
GeForce RTX 3080 10GB10 GB · 760 GB/s	320 W	94–133	35.4	670–949	$0.12–$0.17	$699
Radeon RX 7800 XT16 GB · 624 GB/s	263 W	77–109	35.3	671–950	$0.12–$0.17	$499
GeForce RTX 40608 GB · 272 GB/s	115 W	34–47	35.2	673–953	$0.12–$0.17	$299
GeForce RTX 4070 Ti Super16 GB · 672 GB/s	285 W	83–117	35.1	675–956	$0.12–$0.17	$799
GeForce RTX 4080 Super16 GB · 736 GB/s	320 W	91–129	34.3	692–980	$0.12–$0.18	$999
GeForce RTX 4070 Super12 GB · 504 GB/s	220 W	62–88	34.1	694–984	$0.13–$0.18	$599
Intel Arc A7508 GB · 512 GB/s	225 W	63–89	33.9	699–990	$0.13–$0.18	$289
GeForce RTX 408016 GB · 717 GB/s	320 W	88–125	33.4	710–1,006	$0.13–$0.18	$1,199
GeForce RTX 3060 Ti8 GB · 448 GB/s	200 W	55–78	33.4	710–1,006	$0.13–$0.18	$399
GeForce RTX 3090 Ti24 GB · 1008 GB/s	450 W	124–176	33.4	710–1,006	$0.13–$0.18	$1,999
GeForce RTX 409024 GB · 1008 GB/s	450 W	124–176	33.4	710–1,006	$0.13–$0.18	$1,599
Radeon RX 7900 GRE16 GB · 576 GB/s	260 W	71–101	33.0	718–1,017	$0.13–$0.18	$549
Radeon RX 9060 XT 8GB8 GB · 320 GB/s	150 W	39–56	31.8	746–1,056	$0.13–$0.19	$299
GeForce RTX 3060 12GB12 GB · 360 GB/s	170 W	44–63	31.5	751–1,064	$0.14–$0.19	$329
Radeon RX 9070 XT16 GB · 640 GB/s	304 W	79–112	31.4	756–1,071	$0.14–$0.19	$599
GeForce RTX 3070 Ti8 GB · 608 GB/s	290 W	75–106	31.2	759–1,075	$0.14–$0.19	$599
GeForce RTX 2070 Super8 GB · 448 GB/s	215 W	55–78	31.0	764–1,082	$0.14–$0.19	$499
GeForce RTX 20808 GB · 448 GB/s	215 W	55–78	31.0	764–1,082	$0.14–$0.19	$699
Radeon RX 680016 GB · 512 GB/s	250 W	63–89	30.5	777–1,101	$0.14–$0.20	$579
GeForce RTX 30708 GB · 448 GB/s	220 W	55–78	30.3	781–1,107	$0.14–$0.20	$499
Radeon RX 9060 XT 16GB16 GB · 320 GB/s	160 W	39–56	29.8	795–1,127	$0.14–$0.20	$349
GeForce RTX 2080 Super8 GB · 496 GB/s	250 W	61–87	29.5	802–1,136	$0.14–$0.20	$699
GeForce RTX 4060 Ti 8GB8 GB · 288 GB/s	160 W	35–50	26.8	884–1,252	$0.16–$0.23	$399
GeForce RTX 4070 Ti12 GB · 504 GB/s	285 W	62–88	26.3	900–1,275	$0.16–$0.23	$799
Radeon RX 7700 XT12 GB · 432 GB/s	245 W	53–75	26.3	902–1,278	$0.16–$0.23	$449
GeForce RTX 4060 Ti 16GB16 GB · 288 GB/s	165 W	35–50	26.0	911–1,291	$0.16–$0.23	$499
Radeon RX 76008 GB · 288 GB/s	165 W	35–50	26.0	911–1,291	$0.16–$0.23	$269
Radeon RX 6750 XT12 GB · 432 GB/s	250 W	53–75	25.7	921–1,304	$0.17–$0.23	$549
GeForce RTX 3050 8GB8 GB · 224 GB/s	130 W	28–39	25.7	923–1,308	$0.17–$0.24	$249
Radeon RX 6950 XT16 GB · 576 GB/s	335 W	71–101	25.6	925–1,311	$0.17–$0.24	$1,099
Radeon RX 6800 XT16 GB · 512 GB/s	300 W	63–89	25.4	932–1,321	$0.17–$0.24	$649
Radeon RX 6900 XT16 GB · 512 GB/s	300 W	63–89	25.4	932–1,321	$0.17–$0.24	$999
Radeon RX 6700 XT12 GB · 384 GB/s	230 W	47–67	24.9	953–1,350	$0.17–$0.24	$479
Radeon RX 7600 XT16 GB · 288 GB/s	190 W	35–50	22.6	1,050–1,487	$0.19–$0.27	$329

Qwen3-14B at Q4

Qwen3-14B needs ~11.3 GB at Q4, so 33 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power.

Qwen3-14B: board power, estimated speed, energy and example electricity cost per million tokens by GPU
GPU	Board power	Est. tok/s	tok/s per 100 W	Wh per 1M tokens	Cost per 1M tokens	List price
NVIDIA RTX A500024 GB · 768 GB/s	230 W	51–73	27.0	876–1,241	$0.16–$0.22	$2,250
NVIDIA RTX A400016 GB · 448 GB/s	140 W	30–43	25.9	914–1,295	$0.16–$0.23	$1,000
NVIDIA RTX 2000 Ada 16GB16 GB · 224 GB/s	70 W	15–21	25.9	914–1,295	$0.16–$0.23	$649
NVIDIA RTX 6000 Ada 48GB48 GB · 960 GB/s	300 W	64–91	25.9	914–1,295	$0.16–$0.23	$6,799
GeForce RTX 509032 GB · 1792 GB/s	575 W	120–170	25.2	939–1,330	$0.17–$0.24	$1,999
GeForce RTX 5070 Ti16 GB · 896 GB/s	300 W	60–85	24.2	980–1,388	$0.18–$0.25	$749
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB · 1792 GB/s	600 W	120–170	24.2	980–1,388	$0.18–$0.25	$8,565
Radeon RX 907016 GB · 640 GB/s	220 W	43–61	23.6	1,006–1,425	$0.18–$0.26	$549
NVIDIA RTX 4000 Ada 20GB20 GB · 360 GB/s	130 W	24–34	22.4	1,057–1,497	$0.19–$0.27	$1,250
Radeon RX 7900 XTX24 GB · 960 GB/s	355 W	64–91	21.9	1,082–1,533	$0.19–$0.28	$999
GeForce RTX 309024 GB · 936 GB/s	350 W	63–89	21.7	1,094–1,550	$0.20–$0.28	$1,499
GeForce RTX 508016 GB · 960 GB/s	360 W	64–91	21.6	1,097–1,554	$0.20–$0.28	$999
Intel Arc Pro B70 32GB32 GB · 608 GB/s	230 W	41–58	21.4	1,107–1,568	$0.20–$0.28	$949
NVIDIA RTX A600048 GB · 768 GB/s	300 W	51–73	20.7	1,143–1,619	$0.21–$0.29	$4,649
Radeon RX 7900 XT20 GB · 800 GB/s	315 W	54–76	20.6	1,152–1,632	$0.21–$0.29	$899
GeForce RTX 5060 Ti 16GB16 GB · 448 GB/s	180 W	30–43	20.2	1,176–1,665	$0.21–$0.30	$429
Intel Arc A770 16GB16 GB · 560 GB/s	225 W	38–53	20.2	1,176–1,665	$0.21–$0.30	$349
NVIDIA Titan RTX24 GB · 672 GB/s	280 W	45–64	19.4	1,219–1,727	$0.22–$0.31	$2,499
Radeon RX 7800 XT16 GB · 624 GB/s	263 W	42–59	19.2	1,233–1,747	$0.22–$0.31	$499
GeForce RTX 4070 Ti Super16 GB · 672 GB/s	285 W	45–64	19.1	1,241–1,758	$0.22–$0.32	$799
GeForce RTX 4080 Super16 GB · 736 GB/s	320 W	49–70	18.6	1,272–1,802	$0.23–$0.32	$999
GeForce RTX 408016 GB · 717 GB/s	320 W	48–68	18.1	1,306–1,850	$0.24–$0.33	$1,199
GeForce RTX 3090 Ti24 GB · 1008 GB/s	450 W	68–96	18.1	1,306–1,850	$0.24–$0.33	$1,999
GeForce RTX 409024 GB · 1008 GB/s	450 W	68–96	18.1	1,306–1,850	$0.24–$0.33	$1,599
Radeon RX 7900 GRE16 GB · 576 GB/s	260 W	39–55	17.9	1,321–1,871	$0.24–$0.34	$549
Radeon RX 9070 XT16 GB · 640 GB/s	304 W	43–61	17.0	1,390–1,969	$0.25–$0.35	$599
Radeon RX 680016 GB · 512 GB/s	250 W	34–49	16.6	1,429–2,024	$0.26–$0.36	$579
Radeon RX 9060 XT 16GB16 GB · 320 GB/s	160 W	21–30	16.2	1,463–2,073	$0.26–$0.37	$349
GeForce RTX 4060 Ti 16GB16 GB · 288 GB/s	165 W	19–27	14.1	1,676–2,375	$0.30–$0.43	$499
Radeon RX 6950 XT16 GB · 576 GB/s	335 W	39–55	13.9	1,702–2,411	$0.31–$0.43	$1,099
Radeon RX 6800 XT16 GB · 512 GB/s	300 W	34–49	13.8	1,714–2,429	$0.31–$0.44	$649
Radeon RX 6900 XT16 GB · 512 GB/s	300 W	34–49	13.8	1,714–2,429	$0.31–$0.44	$999
Radeon RX 7600 XT16 GB · 288 GB/s	190 W	19–27	12.3	1,930–2,735	$0.35–$0.49	$329

gpt-oss-20b at Q4

gpt-oss-20b needs ~15.6 GB at Q4, so 13 of 62 GPUs fit it. Sorted by tokens per second per 100 W of board power. MoE speeds are upper bounds.

gpt-oss-20b: board power, estimated speed, energy and example electricity cost per million tokens by GPU
GPU	Board power	Est. tok/s	tok/s per 100 W	Wh per 1M tokens	Cost per 1M tokens	List price
NVIDIA RTX A500024 GB · 768 GB/s	230 W	up to 299	130.0	214+	$0.04+	$2,250
NVIDIA RTX 6000 Ada 48GB48 GB · 960 GB/s	300 W	up to 374	124.6	223+	$0.04+	$6,799
GeForce RTX 509032 GB · 1792 GB/s	575 W	up to 698	121.4	229+	$0.04+	$1,999
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB · 1792 GB/s	600 W	up to 698	116.3	239+	$0.04+	$8,565
NVIDIA RTX 4000 Ada 20GB20 GB · 360 GB/s	130 W	up to 140	107.9	258+	$0.05+	$1,250
Radeon RX 7900 XTX24 GB · 960 GB/s	355 W	up to 374	105.3	264+	$0.05+	$999
GeForce RTX 309024 GB · 936 GB/s	350 W	up to 365	104.2	267+	$0.05+	$1,499
Intel Arc Pro B70 32GB32 GB · 608 GB/s	230 W	up to 237	103.0	270+	$0.05+	$949
NVIDIA RTX A600048 GB · 768 GB/s	300 W	up to 299	99.7	279+	$0.05+	$4,649
Radeon RX 7900 XT20 GB · 800 GB/s	315 W	up to 312	98.9	281+	$0.05+	$899
NVIDIA Titan RTX24 GB · 672 GB/s	280 W	up to 262	93.5	297+	$0.05+	$2,499
GeForce RTX 3090 Ti24 GB · 1008 GB/s	450 W	up to 393	87.2	318+	$0.06+	$1,999
GeForce RTX 409024 GB · 1008 GB/s	450 W	up to 393	87.2	318+	$0.06+	$1,599

Cost per hour at full board power

Worst case for the GPU alone, at $0.18/kWh. "Per year" assumes 8 hours a day at full board power every day.

GPU board power and example electricity cost per hour, per 8-hour day and per year
GPU	Board power	Per hour	Per 8 h day	Per year (8 h/day)
NVIDIA RTX 2000 Ada 16GB16 GB · $649	70 W	$0.01	$0.10	$37
GeForce RTX 40608 GB · $299	115 W	$0.02	$0.17	$60
GeForce RTX 3050 8GB8 GB · $249	130 W	$0.02	$0.19	$68
NVIDIA RTX 4000 Ada 20GB20 GB · $1,250	130 W	$0.02	$0.19	$68
NVIDIA RTX A400016 GB · $1,000	140 W	$0.03	$0.20	$74
GeForce RTX 50608 GB · $299	145 W	$0.03	$0.21	$76
Radeon RX 9060 XT 8GB8 GB · $299	150 W	$0.03	$0.22	$79
Intel Arc B57010 GB · $219	150 W	$0.03	$0.22	$79
GeForce RTX 2060 6GB6 GB · $349	160 W	$0.03	$0.23	$84
GeForce RTX 4060 Ti 8GB8 GB · $399	160 W	$0.03	$0.23	$84
Radeon RX 9060 XT 16GB16 GB · $349	160 W	$0.03	$0.23	$84
GeForce RTX 4060 Ti 16GB16 GB · $499	165 W	$0.03	$0.24	$87
Radeon RX 76008 GB · $269	165 W	$0.03	$0.24	$87
GeForce RTX 3060 12GB12 GB · $329	170 W	$0.03	$0.24	$89
GeForce RTX 2060 Super8 GB · $399	175 W	$0.03	$0.25	$92
GeForce RTX 20708 GB · $499	175 W	$0.03	$0.25	$92
GeForce RTX 5060 Ti 8GB8 GB · $379	180 W	$0.03	$0.26	$95
GeForce RTX 5060 Ti 16GB16 GB · $429	180 W	$0.03	$0.26	$95
Radeon RX 7600 XT16 GB · $329	190 W	$0.03	$0.27	$100
GeForce RTX 3060 Ti8 GB · $399	200 W	$0.04	$0.29	$105
GeForce RTX 407012 GB · $599	200 W	$0.04	$0.29	$105
GeForce RTX 2070 Super8 GB · $499	215 W	$0.04	$0.31	$113
GeForce RTX 20808 GB · $699	215 W	$0.04	$0.31	$113
GeForce RTX 30708 GB · $499	220 W	$0.04	$0.32	$116
GeForce RTX 4070 Super12 GB · $599	220 W	$0.04	$0.32	$116
Radeon RX 907016 GB · $549	220 W	$0.04	$0.32	$116
Intel Arc A7508 GB · $289	225 W	$0.04	$0.32	$118
Intel Arc A770 16GB16 GB · $349	225 W	$0.04	$0.32	$118
Intel Arc Pro B70 32GB32 GB · $949	230 W	$0.04	$0.33	$121
Radeon RX 6700 XT12 GB · $479	230 W	$0.04	$0.33	$121
NVIDIA RTX A500024 GB · $2,250	230 W	$0.04	$0.33	$121
Radeon RX 7700 XT12 GB · $449	245 W	$0.04	$0.35	$129
GeForce RTX 507012 GB · $549	250 W	$0.04	$0.36	$131
GeForce RTX 2080 Super8 GB · $699	250 W	$0.04	$0.36	$131
GeForce RTX 2080 Ti11 GB · $999	250 W	$0.04	$0.36	$131
Radeon RX 6750 XT12 GB · $549	250 W	$0.04	$0.36	$131
Radeon RX 680016 GB · $579	250 W	$0.04	$0.36	$131
Radeon RX 7900 GRE16 GB · $549	260 W	$0.05	$0.37	$137
Radeon RX 7800 XT16 GB · $499	263 W	$0.05	$0.38	$138
NVIDIA Titan RTX24 GB · $2,499	280 W	$0.05	$0.40	$147
GeForce RTX 4070 Ti Super16 GB · $799	285 W	$0.05	$0.41	$150
GeForce RTX 4070 Ti12 GB · $799	285 W	$0.05	$0.41	$150
GeForce RTX 3070 Ti8 GB · $599	290 W	$0.05	$0.42	$152
GeForce RTX 5070 Ti16 GB · $749	300 W	$0.05	$0.43	$158
Radeon RX 6800 XT16 GB · $649	300 W	$0.05	$0.43	$158
Radeon RX 6900 XT16 GB · $999	300 W	$0.05	$0.43	$158
NVIDIA RTX A600048 GB · $4,649	300 W	$0.05	$0.43	$158
NVIDIA RTX 6000 Ada 48GB48 GB · $6,799	300 W	$0.05	$0.43	$158
Radeon RX 9070 XT16 GB · $599	304 W	$0.05	$0.44	$160
Radeon RX 7900 XT20 GB · $899	315 W	$0.06	$0.45	$166
GeForce RTX 3080 10GB10 GB · $699	320 W	$0.06	$0.46	$168
GeForce RTX 4080 Super16 GB · $999	320 W	$0.06	$0.46	$168
GeForce RTX 408016 GB · $1,199	320 W	$0.06	$0.46	$168
Radeon RX 6950 XT16 GB · $1,099	335 W	$0.06	$0.48	$176
GeForce RTX 309024 GB · $1,499	350 W	$0.06	$0.50	$184
GeForce RTX 3080 12GB12 GB · $799	350 W	$0.06	$0.50	$184
Radeon RX 7900 XTX24 GB · $999	355 W	$0.06	$0.51	$187
GeForce RTX 508016 GB · $999	360 W	$0.06	$0.52	$189
GeForce RTX 3090 Ti24 GB · $1,999	450 W	$0.08	$0.65	$237
GeForce RTX 409024 GB · $1,599	450 W	$0.08	$0.65	$237
GeForce RTX 509032 GB · $1,999	575 W	$0.10	$0.83	$302
NVIDIA RTX Pro 6000 Blackwell 96GB96 GB · $8,565	600 W	$0.11	$0.86	$315

Running two or more cards? Multiply by the count: two GeForce RTX 5090s are 1150 W of board power. See multi-GPU for local LLMs and the PSU math in the planner.

Quick answers

How much power does a GPU use to run a local LLM?: At most its rated board power: from 70 W (NVIDIA RTX 2000 Ada 16GB) to 600 W (NVIDIA RTX Pro 6000 Blackwell 96GB) for the cards we track. Token generation is limited by memory bandwidth rather than compute, so cards often draw less than that during chat. We use board power as a worst case. Add the rest of the PC (CPU, board, drives) on top.
How much does it cost in electricity to run a local LLM?: Very little per token. For Llama-3.1-8B-Instruct at Q4, our worst-case estimate runs from $0.09–$0.12 per million tokens on the NVIDIA RTX A5000 to $0.19–$0.27 on the Radeon RX 7600 XT, at an example $0.18/kWh. Scale linearly for your own rate.
Which GPU is the most power-efficient for local LLMs?: For an 8B model, the NVIDIA RTX A5000 gives the most estimated tokens per second per watt of board power (49.7 tok/s per 100 W). Efficiency follows memory bandwidth per watt of board power, not card age. For bigger models only cards with enough VRAM count, so check the 14B and gpt-oss-20b tables.
Is a used RTX 3090 power hungry?: Its board power is 350 W, about 3.0× an RTX 4060's 115 W. But it has 24 GB and 936 GB/s, so per token it is far more efficient than a small card on models only it can fit.
Can I power-limit my GPU for LLM inference?: Yes. On NVIDIA, nvidia-smi -pl sets a lower power limit; AMD cards can be limited in their driver software. Because decode is memory-bound, a lower limit may cost little speed, but results vary by card, so measure tok/s and draw before and after. We don't model power limits.
Why aren't Macs and mini PCs on this page?: We only list board power that the maker publishes for the card (NVIDIA Founders Edition and AMD reference specs). We haven't sourced comparable LLM-load power figures for Macs, Ryzen AI Max machines or DGX Spark yet, so they are left out rather than guessed. GeForce RTX 3080 Ti and Intel Arc B580 are also left out for now.

Tokens per second · Best GPU for local LLMs · Multi-GPU · Q4 vs Q8 vs FP16 · MoE vs dense · Planner