Qwen 3 235B-A22B VRAM Requirements
Qwen 3 235B-A22B needs about 135 GB of VRAM at Q4_K_M with an 8K context: 133 GB for weights, 1.5 GB for the KV cache and 0.9 GB of runtime overhead. At Q8_0 it needs 235 GB and at full FP16 precision 440 GB. The smallest GPU that runs it fully is the NVIDIA H200 141GB.
135 GB VRAM
Qwen 3 235B-A22B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 133 GB
- KV cache 1.5 GB
- Runtime overhead 0.9 GB
Mixture of experts: all 235.1B parameters need memory, but only 22B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GBToo small
- RTX 3090 24GBToo small
- Radeon RX 7900 XTX 24GBToo small
- RTX 4090 24GBToo small
- RTX 5090 32GBToo small
- RTX 6000 Ada 48GB33% on GPU~4.2 tok/s
- A100 80GB56% on GPU~6.1 tok/s
- H100 80GB SXM56% on GPU~6.2 tok/s
- Ryzen AI Max+ 395 128GBToo small
- Apple M4 Max 128GBToo small
- Apple M5 Max 128GBToo small
- RTX PRO 6000 Blackwell 96GB67% on GPU~7.7 tok/s
- NVIDIA DGX Spark 128GBToo small
- Apple M3 Ultra 512GBFits~28 tok/s
Multi-GPU: about 7× 24 GB cards (RTX 3090 / 4090) with the model split across them.
Qwen 3 235B-A22B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 133 GB | 135 GB | 139 GB | Multi-GPU / server |
| Q5_K_M | 5.69 | 156 GB | 158 GB | 162 GB | Multi-GPU / server |
| Q6_K | 6.59 | 180 GB | 183 GB | 187 GB | Multi-GPU / server |
| Q8_0 | 8.50 | 233 GB | 235 GB | 239 GB | Multi-GPU / server |
| FP16 / BF16 | 16.00 | 438 GB | 440 GB | 445 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 107 GB | 109 GB | 114 GB | 128 GB |
| Q2_K | 3.00 | 82.1 GB | 84.4 GB | 88.8 GB | 96 GB |
VRAM by context length
At Q4_K_M. The KV cache grows linearly with context: 0.18 GB per 1,000 tokens in FP16.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.7 GB | 0.4 GB | 134 GB | 134 GB |
| 8K tokens | 1.5 GB | 0.8 GB | 135 GB | 134 GB |
| 32K tokens | 5.9 GB | 3.1 GB | 139 GB | 137 GB |
| 40K tokens | 7.3 GB | 3.9 GB | 141 GB | 137 GB |
GPUs that can run Qwen 3 235B-A22B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| NVIDIA H200 141GB | 141 GB | Tight fit | IQ4_XS | – | ~72 tok/s |
| Apple M2 Ultra 192GB | 144 GB | Fits | Q4_K_M | 9K | ~27 tok/s |
| AMD Instinct MI300X 192GB | 192 GB | Fits | Q5_K_M | 40K | ~70 tok/s |
| Apple M3 Ultra 512GB | 384 GB | Fits | Q8_0 | 40K | ~28 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | 67% on GPU | Q2_K | – | ~7.7 tok/s |
| A100 80GB | 80.0 GB | 56% on GPU | – | – | ~6.1 tok/s |
| H100 80GB SXM | 80.0 GB | 56% on GPU | – | – | ~6.2 tok/s |
| RTX 6000 Ada 48GB | 48.0 GB | 33% on GPU | – | – | ~4.2 tok/s |
| RTX A6000 48GB | 48.0 GB | 33% on GPU | – | – | ~4.1 tok/s |
| L40S 48GB | 48.0 GB | 33% on GPU | – | – | ~4.1 tok/s |
Multi-GPU: about 7× 24 GB cards (RTX 3090 or 4090) run it at Q4_K_M, using tensor or layer splitting in llama.cpp, vLLM or ExLlamaV2.
About Qwen 3 235B-A22B
Qwen's flagship mixture-of-experts model. All 235B parameters must be in memory (about 133 GB at Q4_K_M), but only 22B are read per token. It suits 192 GB+ Macs, multi-GPU servers, or hybrid setups that keep the experts in system RAM.
| Spec | Value |
|---|---|
| Parameters | 235.1B total, 22B active (MoE) |
| Layers | 94 |
| Hidden size | 4096 |
| Attention heads / KV heads | 64 / 4 |
| Head dimension | 128 |
| Vocabulary | 151,936 |
| Max context (used here) | 40K tokens |
| KV cache per token (FP16) | 188 KB |
| Released | 2025-04 |
| License | Apache 2.0 |
Qwen 3 235B-A22B VRAM: common questions
What hardware runs Qwen 3 235B-A22B locally?
A 256 GB or 512 GB Mac Studio, four or more 48 GB GPUs, or one large GPU plus 192 GB+ of fast system RAM using expert offload in llama.cpp.
How much VRAM does Qwen 3 235B-A22B need?
About 135 GB at Q4_K_M with an 8K context, 235 GB at Q8_0 and 440 GB at FP16. Longer contexts add 0.18 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 3 235B-A22B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the NVIDIA H200 141GB (141 GB usable). With 24 GB cards you need about 7 of them.
Does Qwen 3 235B-A22B need VRAM for all 235.1B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 22B parameters are read per token, so it generates text about as fast as a 22B dense model.