What LLMs Can the AMD Instinct MI300X 192GB Run?

The AMD Instinct MI300X 192GB has 192 GB of VRAM and 5300 GB/s of bandwidth. It runs 53 of 58 models fully at Q4_K_M with 8K context, up to DeepSeek V4 Flash, and generates about 460 tokens/s on an 8B model.

AMD · CDNA 3 · Released 2023-12 · Updated October 14, 2026

54 of 58 models fit

AMD Instinct MI300X 192GB (192 GB) at Q4_K_M with 8K context. Largest that fits fully: DeepSeek V4 Flash.

ModelNeedsStatusSpeedBest quant that fits
DeepSeek V4 Flash161 GBFits~143 tok/sFP4 + FP8 (official)
Qwen 3 235B-A22B135 GBFits~70 tok/sQ5_K_M
MiniMax-M2133 GBFits~116 tok/sQ6_K
Qwen 3.5 122B-A10B70.2 GBFits~144 tok/sQ8_0
Mistral Small 4 119B68.2 GBFits~201 tok/sFP8 (official)
gpt-oss-120b67.2 GBFits~212 tok/sMXFP4 (official)
Llama 4 Scout64.0 GBFits~116 tok/sQ8_0
GLM-4.5-Air62.1 GBFits~136 tok/sQ8_0
Qwen3-Next 80B-A3B46.3 GBFits~182 tok/sFP16 / BF16
Qwen 2.5 72B44.5 GBFits~61 tok/sFP16 / BF16
Llama 3.3 70B43.2 GBFits~63 tok/sFP16 / BF16
DeepSeek R1 Distill Llama 70B43.2 GBFits~63 tok/sFP16 / BF16
Mixtral 8x7B28.0 GBFits~165 tok/sFP16 / BF16
Qwen 2.5 32B21.4 GBFits~128 tok/sFP16 / BF16
Qwen 2.5 Coder 32B21.4 GBFits~128 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 32B21.4 GBFits~128 tok/sFP16 / BF16
Qwen 3 32B21.4 GBFits~128 tok/sFP16 / BF16
Qwen 3.5 35B-A3B21.0 GBFits~214 tok/sFP16 / BF16
Qwen 3.6 35B-A3B21.0 GBFits~214 tok/sFP16 / BF16
Gemma 4 31B19.8 GBFits~135 tok/sFP16 / BF16
Qwen 3 30B-A3B18.8 GBFits~178 tok/sFP16 / BF16
Qwen3-Coder 30B-A3B18.8 GBFits~178 tok/sFP16 / BF16
Gemma 3 27B17.6 GBFits~152 tok/sFP16 / BF16
Qwen 3.5 27B16.9 GBFits~157 tok/sFP16 / BF16
Qwen 3.6 27B16.9 GBFits~157 tok/sFP16 / BF16
Qwen 3.8 27B16.9 GBFits~157 tok/sFP16 / BF16
Mistral Small 3.1 24B15.6 GBFits~178 tok/sFP16 / BF16
Gemma 4 26B-A4B15.6 GBFits~257 tok/sFP16 / BF16
gpt-oss-20b12.9 GBFits~314 tok/sMXFP4 (official)
Qwen 2.5 14B10.7 GBFits~262 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 14B10.7 GBFits~262 tok/sFP16 / BF16
Phi-4 14B10.6 GBFits~267 tok/sFP16 / BF16
Qwen 3 14B10.5 GBFits~270 tok/sFP16 / BF16
Ministral 3 14B9.7 GBFits~291 tok/sFP16 / BF16
Mistral Nemo 12B9.0 GBFits~316 tok/sFP16 / BF16
Gemma 3 12B8.8 GBFits~310 tok/sFP16 / BF16
Ministral 3 8B6.7 GBFits~435 tok/sFP16 / BF16
Qwen 3 8B6.6 GBFits~440 tok/sFP16 / BF16
Qwen 3.5 9B6.4 GBFits~445 tok/sFP16 / BF16
Llama 3.1 8B6.3 GBFits~460 tok/sFP16 / BF16
DeepSeek R1 Distill Llama 8B6.3 GBFits~460 tok/sFP16 / BF16
Mistral 7B v0.35.7 GBFits~497 tok/sFP16 / BF16
Gemma 4 E4B5.7 GBFits~473 tok/sFP16 / BF16
Qwen 2.5 7B5.6 GBFits~514 tok/sFP16 / BF16
DeepSeek R1 Distill Qwen 7B5.6 GBFits~514 tok/sFP16 / BF16
Qwen 3 4B4.2 GBFits~713 tok/sFP16 / BF16
Phi-4 Mini4.1 GBFits~768 tok/sFP16 / BF16
Gemma 4 E2B4.0 GBFits~703 tok/sFP16 / BF16
Gemma 3 4B3.7 GBFits~766 tok/sFP16 / BF16
Qwen 3.5 4B3.7 GBFits~795 tok/sFP16 / BF16
Ministral 3 3B3.5 GBFits~890 tok/sFP16 / BF16
Llama 3.2 3B3.5 GBFits~902 tok/sFP16 / BF16
Llama 3.2 1B1.7 GBFits~2155 tok/sFP16 / BF16
Gemma 3 1B1.6 GBFits~2093 tok/sFP16 / BF16
Kimi K2.6566 GB32% on GPU~3.1 tok/s–
DeepSeek R1 671B380 GB47% on GPU~3.4 tok/s–
Llama 3.1 405B234 GB77% on GPU~0.70 tok/sQ2_K
Qwen 3.5 397B-A17B226 GB80% on GPU~17 tok/sQ2_K

AMD Instinct MI300X 192GB for local AI

AMD's datacenter GPU with 192 GB of HBM3, more than an H100 or H200. Single-card capacity is the main advantage: Llama 3.1 405B at Q2–Q3 or 200B-class MoE models at Q4 fit on a single card. vLLM and SGLang support it through ROCm.

Our picks for the AMD Instinct MI300X 192GB

The model we would run first for each job, chosen from our recommendations by what fits fully with an 8K context. "Best quant" is the highest quality that fits; "max context" is at Q4_K_M or the native format.

TaskModelBest quantMax contextSpeed
Best all-rounderQwen 3.5 122B-A10BQ8_0256K~144 tok/s
CodingGLM-4.5-AirQ8_0128K~136 tok/s
Fastest repliesQwen 3.5 122B-A10BQ8_0256K~144 tok/s
Long documents (64K+)Qwen 3.5 122B-A10BQ8_0256K~144 tok/s
Images in, text outGemma 4 31BFP16 / BF16256K~135 tok/s
Image generationFLUX.2 [dev]FP8––
SpecValue
Memory192 GB
Memory bandwidth5300 GB/s
ArchitectureCDNA 3
Released2023-12
Llama 3.1 8B Q4_K_M speed~460 tok/s
Largest model that fitsDeepSeek V4 Flash

Image and video models on the AMD Instinct MI300X 192GB

Peak VRAM with model swapping (ComfyUI default on smaller cards).

ModelFP8FP8 fitGGUF Q4Q4 fit
Stable Diffusion 1.51.8 GBFits1.7 GBFits
SDXL 1.04.1 GBFits4.1 GBFits
Stable Diffusion 3.5 Medium6.4 GBFits6.4 GBFits
Stable Diffusion 3.5 Large8.6 GBFits6.4 GBFits
FLUX.1 [dev]12.1 GBFits7.3 GBFits
FLUX.1 [schnell]12.1 GBFits7.3 GBFits
Qwen-Image20.4 GBFits12.3 GBFits
Wan 2.2 A14B (T2V / I2V)18.8 GBFits13.2 GBFits
Wan 2.2 TI2V 5B8.0 GBFits6.5 GBFits
HunyuanVideo18.3 GBFits13.1 GBFits
FLUX.1 Kontext [dev]12.3 GBFits7.5 GBFits
FLUX.2 [dev]31.0 GBFits23.6 GBFits
Z-Image Turbo6.8 GBFits4.9 GBFits
Qwen-Image-Edit 250920.5 GBFits12.4 GBFits
LTX-219.2 GBFits12.6 GBFits

AMD Instinct MI300X 192GB for LLMs: common questions

Can the AMD Instinct MI300X 192GB run Llama 3.3 70B?

Yes. Llama 3.3 70B needs about 43.2 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

Can the AMD Instinct MI300X 192GB run Qwen 3 32B?

Yes. Qwen 3 32B needs about 21.4 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

Can the AMD Instinct MI300X 192GB run Qwen 3 14B?

Yes. Qwen 3 14B needs about 10.5 GB at Q4_K_M with 8K context, and the best quality that fits is FP16 / BF16.

How fast is the AMD Instinct MI300X 192GB for LLMs?

About 460 tokens per second for an 8B model at Q4_K_M, based on its 5300 GB/s memory bandwidth. Larger models scale down roughly in proportion to their size.

Related