What LLMs Can the Radeon AI PRO R9700 32GB Run?

The Radeon AI PRO R9700 32GB has 32 GB of VRAM and 640 GB/s of bandwidth. It runs 42 of 58 models fully at Q4_K_M with 8K context, up to Mixtral 8x7B, and generates about 64 tokens/s on an 8B model.

AMD · RDNA 4 · Released 2025-07 · Updated October 14, 2026

42 of 58 models fit

Radeon AI PRO R9700 32GB (32.0 GB) at Q4_K_M with 8K context. Largest that fits fully: Mixtral 8x7B.

ModelNeedsStatusSpeedBest quant that fits
Mixtral 8x7B28.0 GBFits~37 tok/sQ4_K_M
Qwen 2.5 32B21.4 GBFits~17 tok/sQ6_K
Qwen 2.5 Coder 32B21.4 GBFits~17 tok/sQ6_K
DeepSeek R1 Distill Qwen 32B21.4 GBFits~17 tok/sQ6_K
Qwen 3 32B21.4 GBFits~17 tok/sQ6_K
Qwen 3.5 35B-A3B21.0 GBFits~104 tok/sQ6_K
Qwen 3.6 35B-A3B21.0 GBFits~104 tok/sQ6_K
Gemma 4 31B19.8 GBFits~18 tok/sQ6_K
Qwen 3 30B-A3B18.8 GBFits~86 tok/sQ6_K
Qwen3-Coder 30B-A3B18.8 GBFits~86 tok/sQ6_K
Gemma 3 27B17.6 GBFits~20 tok/sQ8_0
Qwen 3.5 27B16.9 GBFits~21 tok/sQ8_0
Qwen 3.6 27B16.9 GBFits~21 tok/sQ8_0
Qwen 3.8 27B16.9 GBFits~21 tok/sQ8_0
Mistral Small 3.1 24B15.6 GBFits~23 tok/sQ8_0
Gemma 4 26B-A4B15.6 GBFits~96 tok/sQ8_0
gpt-oss-20b12.9 GBFits~112 tok/sMXFP4 (official)
Qwen 2.5 14B10.7 GBFits~36 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 14B10.7 GBFits~36 tok/sFP16 / BF16
Phi-4 14B10.6 GBFits~36 tok/sFP16 / BF16
Qwen 3 14B10.5 GBFits~36 tok/sFP16 / BF16
Ministral 3 14B9.7 GBFits~39 tok/sFP16 / BF16
Mistral Nemo 12B9.0 GBFits~43 tok/sFP16 / BF16
Gemma 3 12B8.8 GBFits~43 tok/sFP16 / BF16
Ministral 3 8B6.7 GBFits~60 tok/sFP16 / BF16
Qwen 3 8B6.6 GBFits~62 tok/sFP16 / BF16
Qwen 3.5 9B6.4 GBFits~61 tok/sFP16 / BF16
Llama 3.1 8B6.3 GBFits~64 tok/sFP16 / BF16
DeepSeek R1 Distill Llama 8B6.3 GBFits~64 tok/sFP16 / BF16
Mistral 7B v0.35.7 GBFits~70 tok/sFP16 / BF16
Gemma 4 E4B5.7 GBFits~69 tok/sFP16 / BF16
Qwen 2.5 7B5.6 GBFits~71 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 7B5.6 GBFits~71 tok/sFP16 / BF16
Qwen 3 4B4.2 GBFits~111 tok/sFP16 / BF16
Phi-4 Mini4.1 GBFits~118 tok/sFP16 / BF16
Gemma 4 E2B4.0 GBFits~108 tok/sFP16 / BF16
Gemma 3 4B3.7 GBFits~120 tok/sFP16 / BF16
Qwen 3.5 4B3.7 GBFits~124 tok/sFP16 / BF16
Ministral 3 3B3.5 GBFits~135 tok/sFP16 / BF16
Llama 3.2 3B3.5 GBFits~140 tok/sFP16 / BF16
Llama 3.2 1B1.7 GBFits~374 tok/sFP16 / BF16
Gemma 3 1B1.6 GBFits~485 tok/sFP16 / BF16
Qwen 3.5 122B-A10B70.2 GB43% on GPU~10 tok/s–
Mistral Small 4 119B68.2 GB44% on GPU~16 tok/s–
gpt-oss-120b67.2 GB45% on GPU~20 tok/s–
Llama 4 Scout64.0 GB47% on GPU~6.3 tok/s–
GLM-4.5-Air62.1 GB48% on GPU~8.7 tok/s–
Qwen3-Next 80B-A3B46.3 GB65% on GPU~40 tok/sQ2_K
Qwen 2.5 72B44.5 GB68% on GPU~2.3 tok/sQ2_K
Llama 3.3 70B43.2 GB70% on GPU~2.5 tok/sQ2_K
DeepSeek R1 Distill Llama 70B43.2 GB70% on GPU~2.5 tok/sQ2_K
Kimi K2.6566 GBToo small––
DeepSeek R1 671B380 GBToo small––
Llama 3.1 405B234 GBToo small––
Qwen 3.5 397B-A17B226 GBToo small––
DeepSeek V4 Flash161 GBToo small––
Qwen 3 235B-A22B135 GBToo small––
MiniMax-M2133 GBToo small––

Radeon AI PRO R9700 32GB for local AI

A 32 GB workstation version of the RX 9070 XT chip, priced well below NVIDIA's 32 GB options. It holds 32B models at Q6_K or with long contexts and pairs well for 64 GB multi-GPU setups. ROCm supports it officially.

Among the 2 devices we track with about 32 GB of usable memory, the Radeon AI PRO R9700 32GB is the slowest at generating text, because generation speed follows memory bandwidth. Others in the same class: RTX 5090 32GB. They all hold the same models; the faster ones simply answer sooner.

Our picks for the Radeon AI PRO R9700 32GB

The model we would run first for each job, chosen from our recommendations by what fits fully with an 8K context. "Best quant" is the highest quality that fits; "max context" is at Q4_K_M or the native format.

TaskModelBest quantMax contextSpeed
Best all-rounderQwen 3.6 27BQ8_0219K~21 tok/s
CodingQwen 3.6 27BQ8_0219K~21 tok/s
Fastest repliesQwen 3.6 35B-A3BQ6_K256K~104 tok/s
Long documents (64K+)Qwen 3.8 27BQ8_0219K~21 tok/s
Images in, text outGemma 4 31BQ6_K139K~18 tok/s
Image generationFLUX.2 [dev]GGUF Q6_K––
SpecValue
Memory32 GB
Memory bandwidth640 GB/s
ArchitectureRDNA 4
Released2025-07
Llama 3.1 8B Q4_K_M speed~64 tok/s
Largest model that fitsMixtral 8x7B

Image and video models on the Radeon AI PRO R9700 32GB

Peak VRAM with model swapping (ComfyUI default on smaller cards).

ModelFP8FP8 fitGGUF Q4Q4 fit
Stable Diffusion 1.51.8 GBFits1.7 GBFits
SDXL 1.04.1 GBFits4.1 GBFits
Stable Diffusion 3.5 Medium6.4 GBFits6.4 GBFits
Stable Diffusion 3.5 Large8.6 GBFits6.4 GBFits
FLUX.1 [dev]12.1 GBFits7.3 GBFits
FLUX.1 [schnell]12.1 GBFits7.3 GBFits
Qwen-Image20.4 GBFits12.3 GBFits
Wan 2.2 A14B (T2V / I2V)18.8 GBFits13.2 GBFits
Wan 2.2 TI2V 5B8.0 GBFits6.5 GBFits
HunyuanVideo18.3 GBFits13.1 GBFits
FLUX.1 Kontext [dev]12.3 GBFits7.5 GBFits
FLUX.2 [dev]31.0 GBTight fit23.6 GBFits
Z-Image Turbo6.8 GBFits4.9 GBFits
Qwen-Image-Edit 250920.5 GBFits12.4 GBFits
LTX-219.2 GBFits12.6 GBFits

Upgrade path

The next step up is the RTX A6000 48GB with 48 GB and 20% more bandwidth.

Radeon AI PRO R9700 32GB for LLMs: common questions

Can the Radeon AI PRO R9700 32GB run Llama 3.3 70B?

Only partly. It needs about 43.2 GB at Q4_K_M, so about 70% fits in VRAM and the rest runs from system RAM, which is slow. Q2_K fits fully.

Can the Radeon AI PRO R9700 32GB run Qwen 3 32B?

Yes. Qwen 3 32B needs about 21.4 GB at Q4_K_M with 8K context, and the best quality that fits is Q6_K.

Can the Radeon AI PRO R9700 32GB run Qwen 3 14B?

Yes. Qwen 3 14B needs about 10.5 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

How fast is the Radeon AI PRO R9700 32GB for LLMs?

About 64 tokens per second for an 8B model at Q4_K_M, based on its 640 GB/s memory bandwidth. Larger models scale down roughly in proportion to their size.

Related