Multi-GPU for local LLMs: two cards or one big one?

Quick answers

Can two GPUs share VRAM to run a bigger LLM?

Yes. llama.cpp-based apps (LM Studio, Ollama) and vLLM can split a model's layers across cards, so two 24 GB cards hold models up to about 48 GB. The memory adds up; the speed does not (see below). Each card also keeps some memory for its own buffers, so leave a little headroom.

Do two GPUs make a local LLM run faster?

Usually not for one chat. With the common layer split, the cards take turns: each processes its share of layers, so decode speed is about what one card would do if it had all the memory. Our estimates use one card's bandwidth for multi-GPU setups. Tensor parallelism (vLLM, ExLlama) can speed things up, but needs a fast link between the cards and we don't model it.

Can dual RTX 3090s run a 70B model?

At Q4_K_M with ~4k context our formula puts Llama-3.3-70B-Instruct at about 50.2 GB, so two 24 GB cards (48 GB) are just short. People do run 70B on two 3090s with smaller quants (Q3/IQ4) or less context. The cheapest multi-GPU setup we track that fits it at Q4 with ~4k context is 4× Radeon RX 7600 XT ($1,316 for the cards); the cheapest pair is 2× Intel Arc Pro B70 32GB ($1,898).

What is the cheapest way to get 48 GB of VRAM?

From list prices in our data: 3× Radeon RX 7600 XT for $987 (GPU cost only, 288 GB/s per card). The cheapest pair is 2× Intel Arc Pro B70 32GB for $1,898 (608 GB/s per card, so much faster). The cheapest unified-memory machine with at least 48 GB usable is the Mac mini M5 Pro 64GB at $3,199, a complete computer. Used cards can be cheaper or dearer; prices swing a lot.

How many GPUs do I need for gpt-oss-120b?

It needs about 82.5 GB at Q4. The cheapest multi-GPU fit we track is 3× Intel Arc Pro B70 32GB ($2,847 for the cards). A single Framework Desktop (Ryzen AI Max+ 395) 128GB ($3,449) also holds it.

What else does a multi-GPU PC need?

A board with enough PCIe slots and spacing (x8/x8 or better helps, but layer split works over x4), a PSU sized for every card's board power plus the CPU, the right power connectors, and a case with room and airflow. Three or four cards usually need a workstation board, risers or an open frame. The planner checks PSU and case fit for 1–4 cards.