Best LLMs for 24GB VRAM
With 24 GB of VRAM you can run models up to Qwen 3.5 35B-A3B (35B) at Q4_K_M with an 8K context. 41 of 58 tracked models fit fully, and 1 more fit at a lower quantization.
Models that fit in 24 GB
| Model | VRAM at 8K | Best quant that fits | Max context at Q4 |
|---|---|---|---|
| Qwen 3.5 35B-A3B | 21.0 GB | Q4_K_M | 88K |
| Qwen 3.6 35B-A3B | 21.0 GB | Q4_K_M | 88K |
| Qwen 2.5 32B | 21.4 GB | Q4_K_M | 12K |
| Qwen 2.5 Coder 32B | 21.4 GB | Q4_K_M | 12K |
| Qwen 3 32B | 21.4 GB | Q4_K_M | 12K |
| DeepSeek R1 Distill Qwen 32B | 21.4 GB | Q4_K_M | 12K |
| Gemma 4 31B | 19.8 GB | Q4_K_M | 43K |
| Qwen 3 30B-A3B | 18.8 GB | Q5_K_M | 40K |
| Qwen3-Coder 30B-A3B | 18.8 GB | Q5_K_M | 48K |
| Gemma 3 27B | 17.6 GB | Q5_K_M | 71K |
| Qwen 3.5 27B | 16.9 GB | Q6_K | 98K |
| Qwen 3.6 27B | 16.9 GB | Q6_K | 98K |
| Qwen 3.8 27B | 16.9 GB | Q6_K | 98K |
| Gemma 4 26B-A4B | 15.6 GB | Q6_K | 256K |
| Mistral Small 3.1 24B | 15.6 GB | Q6_K | 52K |
| gpt-oss-20b | 13.9 GB | MXFP4 (official) | 128K |
| Qwen 3 14B | 10.5 GB | Q8_0 | 40K |
| Qwen 2.5 14B | 10.7 GB | Q8_0 | 32K |
| Phi-4 14B | 10.6 GB | Q8_0 | 16K |
| DeepSeek R1 Distill Qwen 14B | 10.7 GB | Q8_0 | 71K |
| Ministral 3 14B | 9.7 GB | Q8_0 | 90K |
| Mistral Nemo 12B | 9.0 GB | Q8_0 | 94K |
| Gemma 3 12B | 8.8 GB | Q8_0 | 128K |
| Qwen 3.5 9B | 6.4 GB | FP16 / BF16 | 256K |
| Ministral 3 8B | 6.7 GB | FP16 / BF16 | 127K |
| Qwen 3 8B | 6.6 GB | FP16 / BF16 | 40K |
| Llama 3.1 8B | 6.3 GB | FP16 / BF16 | 128K |
| DeepSeek R1 Distill Llama 8B | 6.3 GB | FP16 / BF16 | 128K |
| Gemma 4 E4B | 5.7 GB | FP16 / BF16 | 128K |
| Qwen 2.5 7B | 5.6 GB | FP16 / BF16 | 32K |
| DeepSeek R1 Distill Qwen 7B | 5.6 GB | FP16 / BF16 | 128K |
| Mistral 7B v0.3 | 5.7 GB | FP16 / BF16 | 32K |
| Gemma 4 E2B | 4.0 GB | FP16 / BF16 | 128K |
| Gemma 3 4B | 3.7 GB | FP16 / BF16 | 128K |
| Qwen 3.5 4B | 3.7 GB | FP16 / BF16 | 256K |
| Qwen 3 4B | 4.2 GB | FP16 / BF16 | 40K |
| Phi-4 Mini | 4.1 GB | FP16 / BF16 | 128K |
| Ministral 3 3B | 3.5 GB | FP16 / BF16 | 195K |
| Llama 3.2 3B | 3.5 GB | FP16 / BF16 | 128K |
| Llama 3.2 1B | 1.7 GB | FP16 / BF16 | 128K |
| Gemma 3 1B | 1.6 GB | FP16 / BF16 | 32K |
Fits at a lower quantization
These need more than 24 GB at Q4_K_M but fit at a smaller format, with more quality loss.
| Model | VRAM at Q4_K_M | Fits at |
|---|---|---|
| Mixtral 8x7B | 28.0 GB | Q2_K |
Image and video models
Best weight precision that fits 24 GB with model swapping.
| Model | Best precision |
|---|---|
| Stable Diffusion 1.5 | FP16 / BF16 |
| SDXL 1.0 | FP16 / BF16 |
| Stable Diffusion 3.5 Medium | FP16 / BF16 |
| Stable Diffusion 3.5 Large | FP16 / BF16 |
| FLUX.1 [dev] | FP8 |
| FLUX.1 [schnell] | FP8 |
| Qwen-Image | FP8 |
| Wan 2.2 A14B (T2V / I2V) | FP8 |
| Wan 2.2 TI2V 5B | FP16 / BF16 |
| HunyuanVideo | FP8 |
| FLUX.1 Kontext [dev] | FP8 |
| FLUX.2 [dev] | Needs offloading or more VRAM |
| Z-Image Turbo | FP16 / BF16 |
| Qwen-Image-Edit 2509 | FP8 |
| LTX-2 | FP8 |
GPUs with 24 GB
- RTX 3090 24GB
- RTX 3090 Ti 24GB
- RTX 4090 24GB
- Radeon RX 7900 XTX 24GB
- Intel Arc Pro B60 24GB
- NVIDIA L4 24GB
- Tesla P40 24GB
- Apple M4 Max 36GB
- Apple M4 Pro 24GB
24 GB VRAM: common questions
Is 24 GB of VRAM enough for local LLMs?
Yes for models up to about 35B at Q4_K_M with 8K context. 24 GB fits 41 of the 58 models tracked here, including Qwen 3.5 35B-A3B, Qwen 3.6 35B-A3B, Qwen 2.5 32B.
What is the biggest model 24 GB can run?
Qwen 3.5 35B-A3B (35B, MoE) at Q4_K_M with 8K context, using about 21.0 GB. Mixtral 8x7B also fits at Q2_K, with some quality loss.
Is 24 GB enough for Flux and SDXL?
SDXL: FP16 / BF16. FLUX.1 dev: FP8 (best precision that fits with model swapping).