Best LLMs for 96GB VRAM

With 96 GB of VRAM you can run models up to Qwen 3.5 122B-A10B (122B) at Q4_K_M with an 8K context. 50 of 58 tracked models fit fully, and 3 more fit at a lower quantization.

Updated October 14, 2026 · 6% safety margin · FP16 KV cache

Models that fit in 96 GB

ModelVRAM at 8KBest quant that fitsMax context at Q4
Qwen 3.5 122B-A10B70.2 GBQ5_K_M256K
gpt-oss-120b62.0 GBMXFP4 (official)128K
Llama 4 Scout64.0 GBQ6_K128K
GLM-4.5-Air62.1 GBQ6_K128K
Qwen3-Next 80B-A3B46.3 GBQ8_0256K
Qwen 2.5 72B44.5 GBQ8_032K
Llama 3.3 70B43.2 GBQ8_0128K
DeepSeek R1 Distill Llama 70B43.2 GBQ8_0128K
Mixtral 8x7B28.0 GBFP16 / BF1632K
Qwen 3.5 35B-A3B21.0 GBFP16 / BF16256K
Qwen 3.6 35B-A3B21.0 GBFP16 / BF16256K
Qwen 2.5 32B21.4 GBFP16 / BF1632K
Qwen 2.5 Coder 32B21.4 GBFP16 / BF1632K
Qwen 3 32B21.4 GBFP16 / BF1640K
DeepSeek R1 Distill Qwen 32B21.4 GBFP16 / BF16128K
Gemma 4 31B19.8 GBFP16 / BF16256K
Qwen 3 30B-A3B18.8 GBFP16 / BF1640K
Qwen3-Coder 30B-A3B18.8 GBFP16 / BF16256K
Gemma 3 27B17.6 GBFP16 / BF16128K
Qwen 3.5 27B16.9 GBFP16 / BF16256K
Qwen 3.6 27B16.9 GBFP16 / BF16256K
Qwen 3.8 27B16.9 GBFP16 / BF16256K
Gemma 4 26B-A4B15.6 GBFP16 / BF16256K
Mistral Small 3.1 24B15.6 GBFP16 / BF16128K
gpt-oss-20b13.9 GBMXFP4 (official)128K
Qwen 3 14B10.5 GBFP16 / BF1640K
Qwen 2.5 14B10.7 GBFP16 / BF1632K
Phi-4 14B10.6 GBFP16 / BF1616K
DeepSeek R1 Distill Qwen 14B10.7 GBFP16 / BF16128K
Ministral 3 14B9.7 GBFP16 / BF16256K
Mistral Nemo 12B9.0 GBFP16 / BF16128K
Gemma 3 12B8.8 GBFP16 / BF16128K
Qwen 3.5 9B6.4 GBFP16 / BF16256K
Ministral 3 8B6.7 GBFP16 / BF16256K
Qwen 3 8B6.6 GBFP16 / BF1640K
Llama 3.1 8B6.3 GBFP16 / BF16128K
DeepSeek R1 Distill Llama 8B6.3 GBFP16 / BF16128K
Gemma 4 E4B5.7 GBFP16 / BF16128K
Qwen 2.5 7B5.6 GBFP16 / BF1632K
DeepSeek R1 Distill Qwen 7B5.6 GBFP16 / BF16128K
Mistral 7B v0.35.7 GBFP16 / BF1632K
Gemma 4 E2B4.0 GBFP16 / BF16128K
Gemma 3 4B3.7 GBFP16 / BF16128K
Qwen 3.5 4B3.7 GBFP16 / BF16256K
Qwen 3 4B4.2 GBFP16 / BF1640K
Phi-4 Mini4.1 GBFP16 / BF16128K
Ministral 3 3B3.5 GBFP16 / BF16256K
Llama 3.2 3B3.5 GBFP16 / BF16128K
Llama 3.2 1B1.7 GBFP16 / BF16128K
Gemma 3 1B1.6 GBFP16 / BF1632K

Fits at a lower quantization

These need more than 96 GB at Q4_K_M but fit at a smaller format, with more quality loss.

ModelVRAM at Q4_K_MFits at
Qwen 3 235B-A22B135 GBQ2_K
MiniMax-M2218 GBQ2_K
Mistral Small 4 119B112 GBQ5_K_M

Image and video models

Best weight precision that fits 96 GB with model swapping.

ModelBest precision
Stable Diffusion 1.5FP16 / BF16
SDXL 1.0FP16 / BF16
Stable Diffusion 3.5 MediumFP16 / BF16
Stable Diffusion 3.5 LargeFP16 / BF16
FLUX.1 [dev]FP16 / BF16
FLUX.1 [schnell]FP16 / BF16
Qwen-ImageFP16 / BF16
Wan 2.2 A14B (T2V / I2V)FP16 / BF16
Wan 2.2 TI2V 5BFP16 / BF16
HunyuanVideoFP16 / BF16
FLUX.1 Kontext [dev]FP16 / BF16
FLUX.2 [dev]FP16 / BF16
Z-Image TurboFP16 / BF16
Qwen-Image-Edit 2509FP16 / BF16
LTX-2FP16 / BF16

GPUs with 96 GB

96 GB VRAM: common questions

Is 96 GB of VRAM enough for local LLMs?

Yes for models up to about 122B at Q4_K_M with 8K context. 96 GB fits 50 of the 58 models tracked here, including Qwen 3.5 122B-A10B, gpt-oss-120b, Llama 4 Scout.

What is the biggest model 96 GB can run?

Qwen 3.5 122B-A10B (122B, MoE) at Q4_K_M with 8K context, using about 70.2 GB. Qwen 3 235B-A22B also fits at Q2_K, with some quality loss.

Is 96 GB enough for Flux and SDXL?

SDXL: FP16 / BF16. FLUX.1 dev: FP16 / BF16 (best precision that fits with model swapping).

Other VRAM sizes