HomeCan I run it?Mixtral 8x22BMixtral 8x22B on dual RTX 3090s
No. Mixtral 8x22B at Q4_K_M needs about 99.0 GB, and dual RTX 3090s have 48.0 GB usable at a ~4k context.
Can I run Mixtral 8x22B on dual RTX 3090s?
Mixtral 8x22B at Q4_K_M needs about 99.0 GB, and dual RTX 3090s have 48.0 GB usable. Two cards add memory, not decode speed; the speed estimate uses one RTX 3090’s 936 GB/s. Mixtral 8x22B, a large Mixture-of-Experts. Total weights need a high-memory box; only a fraction is active per token. These are estimates from our formula, not measured benchmarks.
- Quant that fits
- None of Q4, Q8 or FP16 needs 99.0 GB
- Memory needed
- 99.0 GB at Q4_K_M, ~4k context
- Usable VRAM
- 48.0 GB of 48 GB VRAM
- Est. tokens/sec
- n/a does not fit
Memory at each quant
| Quant | Need vs 48.0 GB | Verdict |
|---|---|---|
| Q4_K_M | 99.0 GB | Q4Won’t fit |
| Q8_0 | 172.8 GB | Q8Won’t fit |
| FP16 | 324.4 GB | FP16Won’t fit |
Estimated tokens per second
No speed estimate: Mixtral 8x22B does not fit in the 48.0 GB this machine can use, so we do not publish a tokens/sec figure for CPU offload.
Context headroom
Even at a short ~4k context, the weights do not fit. Extending context would only increase the gap.
| Context | KV cache (FP16) | Total at Q4_K_M | Verdict |
|---|---|---|---|
| 4k | 0.9 GB | 99.0 GB | Won’t fit |
| 8k | 1.9 GB | 100.0 GB | Won’t fit |
| 16k | 3.8 GB | 101.9 GB | Won’t fit |
| 32k | 7.5 GB | 105.6 GB | Won’t fit |
| 64k | 15.0 GB | 113.1 GB | Won’t fit |
| 128k | 30.1 GB | 128.2 GB | Won’t fit |
What to do instead
- Run R1 Distill 32B instead (23.8 GB at Q4). It is a different model, not a drop-in, but it does fit this machine.
- To hold Mixtral 8x22B at Q4, you need about 99.0 GB. The Mac Studio M5 Ultra 256GB has 192.0 GB usable ($9499 Apple list price, M5 Ultra 30C CPU/64C GPU, 256GB/1TB).
- CPU offload can load the weights anyway, but it is slow. We do not publish an offload tokens/sec figure.
- Full hardware list: Mixtral-8x22B-Instruct-v0.1. Specs for this machine: GeForce RTX 3090 (used as a pair).
Other hardware for Mixtral 8x22B
- Mixtral 8x22B on an RTX 4090Won’t fit
- Mixtral 8x22B on an RTX 5090Won’t fit
- Mixtral 8x22B on a Mac Studio 256GBTight
- Mixtral 8x22B on a Mac mini 64GBWon’t fit
Other models on dual RTX 3090s
- Llama 3.3 70B on dual RTX 3090sWon’t fit
- R1 Distill 70B on dual RTX 3090sWon’t fit
- gpt-oss-120b on dual RTX 3090sWon’t fit
- Qwen3-235B on dual RTX 3090sWon’t fit
- DeepSeek-R1 on dual RTX 3090sWon’t fit
- GLM-4.5-Air on dual RTX 3090sWon’t fit
Quick answers
- Can I run Mixtral 8x22B on dual RTX 3090s?
- No. Mixtral 8x22B at Q4_K_M needs about 99.0 GB, and dual RTX 3090s have 48.0 GB usable.
- How much VRAM does Mixtral 8x22B need?
- About 99.0 GB at Q4_K_M, 172.8 GB at Q8_0 and 324.4 GB at FP16, estimated for a ~4k context.
- How fast is Mixtral 8x22B on dual RTX 3090s?
- We do not estimate tokens/sec when the model does not fit. CPU offload is possible but slow.
- What context length can I use?
- The weights already do not fit, so longer context is not the limiting factor.