What LLMs Can the RTX PRO 6000 Blackwell 96GB Run?

The RTX PRO 6000 Blackwell 96GB has 96 GB of VRAM and 1792 GB/s of bandwidth. It runs 50 of 58 models fully at Q4_K_M with 8K context, up to Qwen 3.5 122B-A10B, and generates about 202 tokens/s on an 8B model.

NVIDIA · Blackwell · Released 2025-04 · Updated October 14, 2026

51 of 58 models fit

RTX PRO 6000 Blackwell 96GB (96.0 GB) at Q4_K_M with 8K context. Largest that fits fully: Qwen 3.5 122B-A10B.

ModelNeedsStatusSpeedBest quant that fits
Qwen 3.5 122B-A10B70.2 GBFits~98 tok/sQ5_K_M
Mistral Small 4 119B68.2 GBFits~142 tok/sQ5_K_M
gpt-oss-120b67.2 GBFits~157 tok/sMXFP4 (official)
Llama 4 Scout64.0 GBFits~70 tok/sQ6_K
GLM-4.5-Air62.1 GBFits~87 tok/sQ6_K
Qwen3-Next 80B-A3B46.3 GBFits~153 tok/sQ8_0
Qwen 2.5 72B44.5 GBFits~25 tok/sQ8_0
Llama 3.3 70B43.2 GBFits~26 tok/sQ8_0
DeepSeek R1 Distill Llama 70B43.2 GBFits~26 tok/sQ8_0
Mixtral 8x7B28.0 GBFits~96 tok/sFP16 / BF16
Qwen 2.5 32B21.4 GBFits~54 tok/sFP16 / BF16
Qwen 2.5 Coder 32B21.4 GBFits~54 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 32B21.4 GBFits~54 tok/sFP16 / BF16
Qwen 3 32B21.4 GBFits~54 tok/sFP16 / BF16
Qwen 3.5 35B-A3B21.0 GBFits~176 tok/sFP16 / BF16
Qwen 3.6 35B-A3B21.0 GBFits~176 tok/sFP16 / BF16
Gemma 4 31B19.8 GBFits~57 tok/sFP16 / BF16
Qwen 3 30B-A3B18.8 GBFits~146 tok/sFP16 / BF16
Qwen3-Coder 30B-A3B18.8 GBFits~146 tok/sFP16 / BF16
Gemma 3 27B17.6 GBFits~64 tok/sFP16 / BF16
Qwen 3.5 27B16.9 GBFits~67 tok/sFP16 / BF16
Qwen 3.6 27B16.9 GBFits~67 tok/sFP16 / BF16
Qwen 3.8 27B16.9 GBFits~67 tok/sFP16 / BF16
Mistral Small 3.1 24B15.6 GBFits~74 tok/sFP16 / BF16
Gemma 4 26B-A4B15.6 GBFits~191 tok/sFP16 / BF16
gpt-oss-20b12.9 GBFits~229 tok/sMXFP4 (official)
Qwen 2.5 14B10.7 GBFits~113 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 14B10.7 GBFits~113 tok/sFP16 / BF16
Phi-4 14B10.6 GBFits~114 tok/sFP16 / BF16
Qwen 3 14B10.5 GBFits~115 tok/sFP16 / BF16
Ministral 3 14B9.7 GBFits~125 tok/sFP16 / BF16
Mistral Nemo 12B9.0 GBFits~137 tok/sFP16 / BF16
Gemma 3 12B8.8 GBFits~136 tok/sFP16 / BF16
Ministral 3 8B6.7 GBFits~191 tok/sFP16 / BF16
Qwen 3 8B6.6 GBFits~195 tok/sFP16 / BF16
Qwen 3.5 9B6.4 GBFits~195 tok/sFP16 / BF16
Llama 3.1 8B6.3 GBFits~202 tok/sFP16 / BF16
DeepSeek R1 Distill Llama 8B6.3 GBFits~202 tok/sFP16 / BF16
Mistral 7B v0.35.7 GBFits~220 tok/sFP16 / BF16
Gemma 4 E4B5.7 GBFits~215 tok/sFP16 / BF16
Qwen 2.5 7B5.6 GBFits~225 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 7B5.6 GBFits~225 tok/sFP16 / BF16
Qwen 3 4B4.2 GBFits~337 tok/sFP16 / BF16
Phi-4 Mini4.1 GBFits~360 tok/sFP16 / BF16
Gemma 4 E2B4.0 GBFits~330 tok/sFP16 / BF16
Gemma 3 4B3.7 GBFits~363 tok/sFP16 / BF16
Qwen 3.5 4B3.7 GBFits~375 tok/sFP16 / BF16
Ministral 3 3B3.5 GBFits~413 tok/sFP16 / BF16
Llama 3.2 3B3.5 GBFits~425 tok/sFP16 / BF16
Llama 3.2 1B1.7 GBFits~1086 tok/sFP16 / BF16
Gemma 3 1B1.6 GBFits~1243 tok/sFP16 / BF16
Llama 3.1 405B234 GB39% on GPU~0.27 tok/s–
Qwen 3.5 397B-A17B226 GB40% on GPU~6.2 tok/s–
DeepSeek V4 Flash161 GB56% on GPU~11 tok/s–
Qwen 3 235B-A22B135 GB67% on GPU~7.7 tok/sQ2_K
MiniMax-M2133 GB68% on GPU~16 tok/sQ2_K
Kimi K2.6566 GBToo small––
DeepSeek R1 671B380 GBToo small––

RTX PRO 6000 Blackwell 96GB for local AI

96 GB on one workstation card with RTX 5090-class bandwidth. It runs 70B models at Q8_0 or gpt-oss-120b natively on one card.

Among the 5 devices we track with about 96 GB of usable memory, the RTX PRO 6000 Blackwell 96GB is the fastest at generating text, because generation speed follows memory bandwidth. Others in the same class: Apple M5 Max 128GB, Apple M4 Max 128GB, Apple M3 Max 128GB, Ryzen AI Max+ 395 128GB. They all hold the same models; the faster ones simply answer sooner.

Our picks for the RTX PRO 6000 Blackwell 96GB

The model we would run first for each job, chosen from our recommendations by what fits fully with an 8K context. "Best quant" is the highest quality that fits; "max context" is at Q4_K_M or the native format.

TaskModelBest quantMax contextSpeed
Best all-rounderQwen 3.5 122B-A10BQ5_K_M256K~98 tok/s
CodingGLM-4.5-AirQ6_K128K~87 tok/s
Fastest repliesQwen 3.5 122B-A10BQ5_K_M256K~98 tok/s
Long documents (64K+)Qwen 3.5 122B-A10BQ5_K_M256K~98 tok/s
Images in, text outGemma 4 31BFP16 / BF16256K~57 tok/s
Image generationFLUX.2 [dev]FP8––
SpecValue
Memory96 GB
Memory bandwidth1792 GB/s
ArchitectureBlackwell
Released2025-04
Llama 3.1 8B Q4_K_M speed~202 tok/s
Largest model that fitsQwen 3.5 122B-A10B

Image and video models on the RTX PRO 6000 Blackwell 96GB

Peak VRAM with model swapping (ComfyUI default on smaller cards).

ModelFP8FP8 fitGGUF Q4Q4 fit
Stable Diffusion 1.51.8 GBFits1.7 GBFits
SDXL 1.04.1 GBFits4.1 GBFits
Stable Diffusion 3.5 Medium6.4 GBFits6.4 GBFits
Stable Diffusion 3.5 Large8.6 GBFits6.4 GBFits
FLUX.1 [dev]12.1 GBFits7.3 GBFits
FLUX.1 [schnell]12.1 GBFits7.3 GBFits
Qwen-Image20.4 GBFits12.3 GBFits
Wan 2.2 A14B (T2V / I2V)18.8 GBFits13.2 GBFits
Wan 2.2 TI2V 5B8.0 GBFits6.5 GBFits
HunyuanVideo18.3 GBFits13.1 GBFits
FLUX.1 Kontext [dev]12.3 GBFits7.5 GBFits
FLUX.2 [dev]31.0 GBFits23.6 GBFits
Z-Image Turbo6.8 GBFits4.9 GBFits
Qwen-Image-Edit 250920.5 GBFits12.4 GBFits
LTX-219.2 GBFits12.6 GBFits

Comparisons

RTX PRO 6000 Blackwell 96GB for LLMs: common questions

Can the RTX PRO 6000 Blackwell 96GB run Llama 3.3 70B?

Yes. Llama 3.3 70B needs about 43.2 GB at Q4_K_M with 8K context, and the best quality that fits is Q8_0.

Can the RTX PRO 6000 Blackwell 96GB run Qwen 3 32B?

Yes. Qwen 3 32B needs about 21.4 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

Can the RTX PRO 6000 Blackwell 96GB run Qwen 3 14B?

Yes. Qwen 3 14B needs about 10.5 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

How fast is the RTX PRO 6000 Blackwell 96GB for LLMs?

About 202 tokens per second for an 8B model at Q4_K_M, based on its 1792 GB/s memory bandwidth. Larger models scale down roughly in proportion to their size.

Related