What LLMs Can the RTX 6000 Ada 48GB Run?

The RTX 6000 Ada 48GB has 48 GB of VRAM and 960 GB/s of bandwidth. It runs 46 of 58 models fully at Q4_K_M with 8K context, up to Qwen3-Next 80B-A3B, and generates about 111 tokens/s on an 8B model.

NVIDIA · Ada Lovelace · Released 2022-12 · Updated October 14, 2026

46 of 58 models fit

RTX 6000 Ada 48GB (48.0 GB) at Q4_K_M with 8K context. Largest that fits fully: Qwen3-Next 80B-A3B.

ModelNeedsStatusSpeedBest quant that fits
Qwen 2.5 72B44.5 GBFits~14 tok/sQ4_K_M
Llama 3.3 70B43.2 GBFits~14 tok/sQ4_K_M
DeepSeek R1 Distill Llama 70B43.2 GBFits~14 tok/sQ4_K_M
Mixtral 8x7B28.0 GBFits~60 tok/sQ6_K
Qwen 2.5 32B21.4 GBFits~29 tok/sQ8_0
Qwen 2.5 Coder 32B21.4 GBFits~29 tok/sQ8_0
DeepSeek R1 Distill Qwen 32B21.4 GBFits~29 tok/sQ8_0
Qwen 3 32B21.4 GBFits~29 tok/sQ8_0
Qwen 3.5 35B-A3B21.0 GBFits~140 tok/sQ8_0
Qwen 3.6 35B-A3B21.0 GBFits~140 tok/sQ8_0
Gemma 4 31B19.8 GBFits~31 tok/sQ8_0
Qwen 3 30B-A3B18.8 GBFits~116 tok/sQ8_0
Qwen3-Coder 30B-A3B18.8 GBFits~116 tok/sQ8_0
Gemma 3 27B17.6 GBFits~35 tok/sQ8_0
Qwen 3.5 27B16.9 GBFits~36 tok/sQ8_0
Qwen 3.6 27B16.9 GBFits~36 tok/sQ8_0
Qwen 3.8 27B16.9 GBFits~36 tok/sQ8_0
Mistral Small 3.1 24B15.6 GBFits~40 tok/sQ8_0
Gemma 4 26B-A4B15.6 GBFits~140 tok/sQ8_0
gpt-oss-20b12.9 GBFits~165 tok/sMXFP4 (official)
Qwen 2.5 14B10.7 GBFits~62 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 14B10.7 GBFits~62 tok/sFP16 / BF16
Phi-4 14B10.6 GBFits~62 tok/sFP16 / BF16
Qwen 3 14B10.5 GBFits~63 tok/sFP16 / BF16
Ministral 3 14B9.7 GBFits~69 tok/sFP16 / BF16
Mistral Nemo 12B9.0 GBFits~75 tok/sFP16 / BF16
Gemma 3 12B8.8 GBFits~75 tok/sFP16 / BF16
Ministral 3 8B6.7 GBFits~105 tok/sFP16 / BF16
Qwen 3 8B6.6 GBFits~108 tok/sFP16 / BF16
Qwen 3.5 9B6.4 GBFits~107 tok/sFP16 / BF16
Llama 3.1 8B6.3 GBFits~111 tok/sFP16 / BF16
DeepSeek R1 Distill Llama 8B6.3 GBFits~111 tok/sFP16 / BF16
Mistral 7B v0.35.7 GBFits~122 tok/sFP16 / BF16
Gemma 4 E4B5.7 GBFits~120 tok/sFP16 / BF16
Qwen 2.5 7B5.6 GBFits~124 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 7B5.6 GBFits~124 tok/sFP16 / BF16
Qwen 3 4B4.2 GBFits~191 tok/sFP16 / BF16
Phi-4 Mini4.1 GBFits~204 tok/sFP16 / BF16
Gemma 4 E2B4.0 GBFits~187 tok/sFP16 / BF16
Gemma 3 4B3.7 GBFits~206 tok/sFP16 / BF16
Qwen 3.5 4B3.7 GBFits~213 tok/sFP16 / BF16
Ministral 3 3B3.5 GBFits~233 tok/sFP16 / BF16
Llama 3.2 3B3.5 GBFits~241 tok/sFP16 / BF16
Llama 3.2 1B1.7 GBFits~633 tok/sFP16 / BF16
Gemma 3 1B1.6 GBFits~783 tok/sFP16 / BF16
Qwen3-Next 80B-A3B46.3 GBTight fit~125 tok/sIQ4_XS
Qwen 3 235B-A22B135 GB33% on GPU~4.2 tok/s–
MiniMax-M2133 GB34% on GPU~8.2 tok/s–
Qwen 3.5 122B-A10B70.2 GB64% on GPU~15 tok/sQ2_K
Mistral Small 4 119B68.2 GB66% on GPU~25 tok/sQ2_K
gpt-oss-120b67.2 GB67% on GPU~31 tok/sQ2_K
Llama 4 Scout64.0 GB70% on GPU~10 tok/sQ2_K
GLM-4.5-Air62.1 GB73% on GPU~15 tok/sQ2_K
Kimi K2.6566 GBToo small––
DeepSeek R1 671B380 GBToo small––
Llama 3.1 405B234 GBToo small––
Qwen 3.5 397B-A17B226 GBToo small––
DeepSeek V4 Flash161 GBToo small––

RTX 6000 Ada 48GB for local AI

The Ada-generation 48 GB workstation card. Holds 70B models at Q4_K_M on a single card with ECC memory and a 300 W blower design.

Among the 6 devices we track with about 48 GB of usable memory, the RTX 6000 Ada 48GB is the fastest at generating text, because generation speed follows memory bandwidth. Others in the same class: L40S 48GB, RTX A6000 48GB, Apple M4 Max 64GB, Apple M1 Max 64GB. They all hold the same models; the faster ones simply answer sooner.

Our picks for the RTX 6000 Ada 48GB

The model we would run first for each job, chosen from our recommendations by what fits fully with an 8K context. "Best quant" is the highest quality that fits; "max context" is at Q4_K_M or the native format.

TaskModelBest quantMax contextSpeed
Best all-rounderLlama 3.3 70BQ4_K_M14K~14 tok/s
CodingQwen 3.6 27BQ8_0256K~36 tok/s
Fastest repliesQwen 3.6 35B-A3BQ8_0256K~140 tok/s
Long documents (64K+)Qwen 3.8 27BQ8_0256K~36 tok/s
Images in, text outGemma 4 31BQ8_0256K~31 tok/s
Image generationFLUX.2 [dev]FP8––
SpecValue
Memory48 GB
Memory bandwidth960 GB/s
ArchitectureAda Lovelace
Released2022-12
Llama 3.1 8B Q4_K_M speed~111 tok/s
Largest model that fitsQwen3-Next 80B-A3B

Image and video models on the RTX 6000 Ada 48GB

Peak VRAM with model swapping (ComfyUI default on smaller cards).

ModelFP8FP8 fitGGUF Q4Q4 fit
Stable Diffusion 1.51.8 GBFits1.7 GBFits
SDXL 1.04.1 GBFits4.1 GBFits
Stable Diffusion 3.5 Medium6.4 GBFits6.4 GBFits
Stable Diffusion 3.5 Large8.6 GBFits6.4 GBFits
FLUX.1 [dev]12.1 GBFits7.3 GBFits
FLUX.1 [schnell]12.1 GBFits7.3 GBFits
Qwen-Image20.4 GBFits12.3 GBFits
Wan 2.2 A14B (T2V / I2V)18.8 GBFits13.2 GBFits
Wan 2.2 TI2V 5B8.0 GBFits6.5 GBFits
HunyuanVideo18.3 GBFits13.1 GBFits
FLUX.1 Kontext [dev]12.3 GBFits7.5 GBFits
FLUX.2 [dev]31.0 GBFits23.6 GBFits
Z-Image Turbo6.8 GBFits4.9 GBFits
Qwen-Image-Edit 250920.5 GBFits12.4 GBFits
LTX-219.2 GBFits12.6 GBFits

Upgrade path

The next step up is the RTX PRO 6000 Blackwell 96GB with 96 GB and 87% more bandwidth.

RTX 6000 Ada 48GB for LLMs: common questions

Can the RTX 6000 Ada 48GB run Llama 3.3 70B?

Yes. Llama 3.3 70B needs about 43.2 GB at Q4_K_M with 8K context, and the best quality that fits is Q4_K_M.

Can the RTX 6000 Ada 48GB run Qwen 3 32B?

Yes. Qwen 3 32B needs about 21.4 GB at Q4_K_M with 8K context, and the best quality that fits is Q8_0.

Can the RTX 6000 Ada 48GB run Qwen 3 14B?

Yes. Qwen 3 14B needs about 10.5 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

How fast is the RTX 6000 Ada 48GB for LLMs?

About 111 tokens per second for an 8B model at Q4_K_M, based on its 960 GB/s memory bandwidth. Larger models scale down roughly in proportion to their size.

Related