Qwen 3 32B VRAM Requirements
Qwen 3 32B needs about 21.4 GB of VRAM at Q4_K_M with an 8K context: 18.5 GB for weights, 2.0 GB for the KV cache and 0.9 GB of runtime overhead. At Q8_0 it needs 35.3 GB and at full FP16 precision 64.0 GB. The smallest GPU that runs it fully is the NVIDIA L4 24GB.
21.4 GB VRAM
Qwen 3 32B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 18.5 GB
- KV cache 2.0 GB
- Runtime overhead 0.9 GB
Which GPUs can run it
- RTX 3060 12GB53% on GPU~3.5 tok/s
- RTX 4060 Ti 16GB70% on GPU~4.4 tok/s
- RTX 3090 24GBFits~28 tok/s
- Radeon RX 7900 XTX 24GBFits~25 tok/s
- RTX 4090 24GBFits~31 tok/s
- RTX 5090 32GBFits~54 tok/s
- RTX 6000 Ada 48GBFits~29 tok/s
- A100 80GBFits~61 tok/s
- H100 80GB SXMFits~97 tok/s
- Ryzen AI Max+ 395 128GBFits~7.3 tok/s
- Apple M4 Max 128GBFits~17 tok/s
- Apple M5 Max 128GBFits~19 tok/s
- RTX PRO 6000 Blackwell 96GBFits~54 tok/s
- NVIDIA DGX Spark 128GBFits~7.8 tok/s
- Apple M3 Ultra 512GBFits~25 tok/s
Qwen 3 32B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 18.5 GB | 21.4 GB | 27.4 GB | 24 GB |
| Q5_K_M | 5.69 | 21.7 GB | 24.6 GB | 30.6 GB | 32 GB |
| Q6_K | 6.59 | 25.2 GB | 28.0 GB | 34.0 GB | 32 GB |
| Q8_0 | 8.50 | 32.5 GB | 35.3 GB | 41.3 GB | 48 GB |
| FP16 / BF16 | 16.00 | 61.1 GB | 64.0 GB | 70.0 GB | 80 GB |
| Q3_K_M | 3.91 | 14.9 GB | 17.8 GB | 23.8 GB | 20 GB |
| Q2_K | 3.00 | 11.5 GB | 14.3 GB | 20.3 GB | 16 GB |
VRAM by context length
At Q4_K_M. The KV cache grows linearly with context: 0.25 GB per 1,000 tokens in FP16.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 1.0 GB | 0.5 GB | 20.4 GB | 19.9 GB |
| 8K tokens | 2.0 GB | 1.1 GB | 21.4 GB | 20.4 GB |
| 32K tokens | 8.0 GB | 4.3 GB | 27.4 GB | 23.6 GB |
| 40K tokens | 10.0 GB | 5.3 GB | 29.4 GB | 24.7 GB |
GPUs that can run Qwen 3 32B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| NVIDIA L4 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~9.2 tok/s |
| Tesla P40 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~9.1 tok/s |
| Apple M4 Max 36GB | 24.0 GB | Fits | Q4_K_M | 12K | ~13 tok/s |
| Intel Arc Pro B60 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~11 tok/s |
| RTX 3090 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~28 tok/s |
| Radeon RX 7900 XTX 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~25 tok/s |
| RTX 3090 Ti 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~31 tok/s |
| RTX 4090 24GB | 24.0 GB | Fits | Q4_K_M | 12K | ~31 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | Fits | Q6_K | 40K | ~17 tok/s |
| RTX 5090 32GB | 32.0 GB | Fits | Q6_K | 40K | ~54 tok/s |
| Apple M4 Pro 48GB | 36.0 GB | Fits | Q6_K | 40K | ~8.4 tok/s |
| A100 40GB | 40.0 GB | Fits | Q8_0 | 40K | ~47 tok/s |
| Apple M5 Pro 64GB | 48.0 GB | Fits | Q8_0 | 40K | ~9.5 tok/s |
| Apple M1 Max 64GB | 48.0 GB | Fits | Q8_0 | 40K | ~12 tok/s |
| Radeon RX 7900 XT 20GB | 20.0 GB | 88% on GPU | Q3_K_M | – | ~9.7 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | 70% on GPU | Q2_K | – | ~4.4 tok/s |
| RTX 4070 Ti Super 16GB | 16.0 GB | 70% on GPU | Q2_K | – | ~5.5 tok/s |
| RTX 4080 16GB | 16.0 GB | 70% on GPU | Q2_K | – | ~5.5 tok/s |
| RTX 4080 Super 16GB | 16.0 GB | 70% on GPU | Q2_K | – | ~5.6 tok/s |
| RTX 5060 Ti 16GB | 16.0 GB | 70% on GPU | Q2_K | – | ~5.0 tok/s |
About Qwen 3 32B
The largest dense Qwen 3 model. Weights match Qwen 2.5 32B in size, and the KV cache is the same 256 KB per token in FP16. A 24 GB card runs it at Q4_K_M with about 12K of context.
| Spec | Value |
|---|---|
| Parameters | 32.8B |
| Layers | 64 |
| Hidden size | 5120 |
| Attention heads / KV heads | 64 / 8 |
| Head dimension | 128 |
| Vocabulary | 151,936 |
| Max context (used here) | 40K tokens |
| KV cache per token (FP16) | 256 KB |
| Released | 2025-04 |
| License | Apache 2.0 |
Qwen 3 32B VRAM: common questions
Is Qwen 3 32B or Qwen 3 30B-A3B better for a 24GB GPU?
30B-A3B is several times faster because only 3.3B parameters are active per token. 32B is the stronger model per token. Both fit 24 GB at Q4_K_M.
How much VRAM does Qwen 3 32B need?
About 21.4 GB at Q4_K_M with an 8K context, 35.3 GB at Q8_0 and 64.0 GB at FP16. Longer contexts add 0.25 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 3 32B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the NVIDIA L4 24GB (24.0 GB usable).