# Can I run Mistral Small 24B on an RTX 3070 Ti? > No. Mistral Small 24B needs about 17.7 GB at Q4_K_M, and RTX 3070 Ti has 8.0 GB usable. What to run instead. Source: https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-3070-ti/ [Home](https://whatcanirunai.vercel.app/) / [Can I run it?](https://whatcanirunai.vercel.app/can-i-run/) / [Mistral Small 24B](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/) / Mistral Small 24B on an RTX 3070 Ti **No.** Mistral Small 24B at Q4\_K\_M needs about 17.7 GB, and the GeForce RTX 3070 Ti has 8.0 GB usable at a \~4k context. # Can I run Mistral Small 24B on an RTX 3070 Ti? Mistral Small 24B at Q4\_K\_M needs about 17.7 GB, and the GeForce RTX 3070 Ti has 8.0 GB usable. That figure is 8 GB of VRAM on the spec sheet, of which 8.0 GB is usable for weights and cache, at 608 GB/s memory bandwidth. Mistral’s 24B Small instruct model, a dense mid-size alternative to the 27B–32B class. These are estimates from [our formula](https://whatcanirunai.vercel.app/method/), not measured benchmarks. Quant that fits None of Q4, Q8 or FP16 needs 17.7 GB Memory needed 17.7 GB at Q4\_K\_M, \~4k context Usable VRAM 8.0 GB of 8 GB VRAM Est. tokens/sec n/a does not fit [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/) · [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/) · [Everything this machine runs](https://whatcanirunai.vercel.app/what-can-i-run/rtx-3070-ti/) · [Q4 vs Q8 vs FP16](https://whatcanirunai.vercel.app/quantization/) ## Memory at each quant | Quant | Need vs 8.0 GB | Verdict | | -------- | -------------- | ------------- | | Q4\_K\_M | 17.7 GB | Q4Won’t fit | | Q8\_0 | 30.3 GB | Q8Won’t fit | | FP16 | 56.2 GB | FP16Won’t fit | ## Estimated tokens per second No speed estimate: Mistral Small 24B does not fit in the 8.0 GB this machine can use, so we do not publish a tokens/sec figure for CPU offload. ## Context headroom Even at a short \~4k context, the weights do not fit. Extending context would only increase the gap. __Context headroom for Mistral Small 24B on an RTX 3070 Ti at Q4\_K\_M__ | Context | KV cache (FP16) | Total at Q4\_K\_M | Verdict | | ------- | --------------- | ----------------- | --------- | | 4k | 0.7 GB | 17.7 GB | Won’t fit | | 8k | 1.3 GB | 18.4 GB | Won’t fit | | 16k | 2.7 GB | 19.8 GB | Won’t fit | | 32k | 5.4 GB | 22.4 GB | Won’t fit | | 64k | 10.7 GB | 27.8 GB | Won’t fit | | 128k | 21.5 GB | 38.5 GB | Won’t fit | From the model’s config. Try other sizes in the [VRAM calculator](https://whatcanirunai.vercel.app/vram-calculator/) or see the [context length guide](https://whatcanirunai.vercel.app/context-length/). ## What to do instead * Run [Qwen3-8B](https://whatcanirunai.vercel.app/can-i-run/qwen3-8b/rtx-3070-ti/) instead (6.7 GB at Q4). It is a different model, not a drop-in, but it does fit this machine. * To hold Mistral Small 24B at Q4, you need about 17.7 GB. The [Mac mini M6 24GB](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/mac-mini-m6-24/) has 18.0 GB usable ($1299 Apple list price, M6 12C CPU/12C GPU, 24GB/512GB (MHQM4LL/A)). * CPU offload can load the weights anyway, but it is slow. We do not publish an offload tokens/sec figure. * Full hardware list: [Mistral-Small-3.2-24B-Instruct-2506](https://whatcanirunai.vercel.app/models/mistral-small-3.2-24b-instruct-2506/). Specs for this machine: [GeForce RTX 3070 Ti](https://whatcanirunai.vercel.app/hardware/rtx-3070-ti/). ## Other hardware for Mistral Small 24B * [Mistral Small 24B on an RTX 3060 12GB](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-3060-12gb/)Won’t fit * [Mistral Small 24B on an RTX 3080](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-3080-10gb/)Won’t fit * [Mistral Small 24B on an RTX 3090](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-3090/)Fits * [Mistral Small 24B on an RTX 4060 Ti 16GB](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-4060-ti-16gb/)Won’t fit * [Mistral Small 24B on an RTX 4070 Ti Super](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-4070-ti-super/)Won’t fit * [Mistral Small 24B on an RTX 4090](https://whatcanirunai.vercel.app/can-i-run/mistral-small-3.2-24b-instruct-2506/rtx-4090/)Fits ## Other models on an RTX 3070 Ti * [Qwen3-32B on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/qwen3-32b/rtx-3070-ti/)Won’t fit * [Qwen3-30B-A3B on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/qwen3-30b-a3b/rtx-3070-ti/)Won’t fit * [Qwen3.8-27B on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/qwen3.8-27b/rtx-3070-ti/)Won’t fit * [Gemma 3 27B on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/gemma-3-27b-it/rtx-3070-ti/)Won’t fit * [Gemma 4 31B on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/gemma-4-31b-it/rtx-3070-ti/)Won’t fit * [gpt-oss-20b on an RTX 3070 Ti](https://whatcanirunai.vercel.app/can-i-run/gpt-oss-20b/rtx-3070-ti/)Won’t fit ## Quick answers Can I run Mistral Small 24B on an RTX 3070 Ti? No. Mistral Small 24B at Q4\_K\_M needs about 17.7 GB, and the GeForce RTX 3070 Ti has 8.0 GB usable. How much VRAM does Mistral Small 24B need? About 17.7 GB at Q4\_K\_M, 30.3 GB at Q8\_0 and 56.2 GB at FP16, estimated for a \~4k context. How fast is Mistral Small 24B on an RTX 3070 Ti? We do not estimate tokens/sec when the model does not fit. CPU offload is possible but slow. What context length can I use? The weights already do not fit, so longer context is not the limiting factor. --- Agent guide: https://whatcanirunai.vercel.app/llms.txt · JSON API: https://whatcanirunai.vercel.app/developers/