Qwen 3.5 397B-A17B VRAM Requirements
Qwen 3.5 397B-A17B needs about 226 GB of VRAM at Q4_K_M with an 8K context: 224 GB for weights, 0.4 GB for the KV cache and 1.0 GB of runtime overhead. At Q8_0 it needs 394 GB and at full FP16 precision 741 GB. The smallest GPU that runs it fully is the Apple M3 Ultra 512GB.
226 GB VRAM
Qwen 3.5 397B-A17B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 224 GB
- KV cache 0.4 GB
- Runtime overhead 1.0 GB
Mixture of experts: all 397B parameters need memory, but only 17B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GBToo small
- RTX 3090 24GBToo small
- Radeon RX 7900 XTX 24GBToo small
- RTX 4090 24GBToo small
- RTX 5090 32GBToo small
- RTX 6000 Ada 48GBToo small
- A100 80GB33% on GPU~5.6 tok/s
- H100 80GB SXM33% on GPU~5.7 tok/s
- Ryzen AI Max+ 395 128GBToo small
- Apple M4 Max 128GBToo small
- Apple M5 Max 128GBToo small
- RTX PRO 6000 Blackwell 96GB40% on GPU~6.2 tok/s
- NVIDIA DGX Spark 128GBToo small
- Apple M3 Ultra 512GBFits~39 tok/s
Qwen 3.5 397B-A17B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 224 GB | 226 GB | 226 GB | Multi-GPU / server |
| Q5_K_M | 5.69 | 263 GB | 264 GB | 265 GB | Multi-GPU / server |
| Q6_K | 6.59 | 305 GB | 306 GB | 307 GB | Multi-GPU / server |
| Q8_0 | 8.50 | 393 GB | 394 GB | 395 GB | Multi-GPU / server |
| FP16 / BF16 | 16.00 | 739 GB | 741 GB | 742 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 181 GB | 182 GB | 183 GB | Multi-GPU / server |
| Q2_K | 3.00 | 139 GB | 140 GB | 141 GB | Multi-GPU / server |
VRAM by context length
At Q4_K_M. Hybrid attention: only 15 of 60 layers use full attention and keep a KV cache (2 KV heads × 256 dimensions). The other 45 use linear attention (Gated DeltaNet) with a fixed state of about 186 MB that does not grow with context.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.3 GB | 0.2 GB | 225 GB | 225 GB |
| 8K tokens | 0.4 GB | 0.3 GB | 226 GB | 225 GB |
| 32K tokens | 1.1 GB | 0.7 GB | 226 GB | 226 GB |
| 128K tokens | 3.9 GB | 2.2 GB | 229 GB | 227 GB |
| 256K tokens | 7.7 GB | 4.2 GB | 233 GB | 229 GB |
GPUs that can run Qwen 3.5 397B-A17B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M3 Ultra 512GB | 384 GB | Fits | Q6_K | 256K | ~39 tok/s |
| AMD Instinct MI300X 192GB | 192 GB | 80% on GPU | Q2_K | – | ~17 tok/s |
| NVIDIA H200 141GB | 141 GB | 59% on GPU | – | – | ~8.9 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | 40% on GPU | – | – | ~6.2 tok/s |
| A100 80GB | 80.0 GB | 33% on GPU | – | – | ~5.6 tok/s |
| H100 80GB SXM | 80.0 GB | 33% on GPU | – | – | ~5.7 tok/s |
About Qwen 3.5 397B-A17B
Qwen's flagship open model of early 2026: 512 experts with 10 routed plus one shared per token. It is a server or Mac Studio model; at Q4_K_M it needs well over 200 GB. Its KV cache is remarkably small for its size because only 15 of 60 layers use full attention, each with 2 KV heads.
| Spec | Value |
|---|---|
| Parameters | 397B total, 17B active (MoE) |
| Layers | 60 |
| Hidden size | 4096 |
| Attention heads / KV heads | 32 / 2 |
| Head dimension | 256 |
| Vocabulary | 248,320 |
| Max context (used here) | 256K tokens |
| Linear-attention layers | 45 of 60 (Gated DeltaNet, fixed 186 MB state) |
| KV cache per token (FP16) | 30 KB |
| Released | 2026-02 |
| License | Apache 2.0 |
Qwen 3.5 397B-A17B VRAM: common questions
Can Qwen 3.5 397B run on a Mac Studio?
Yes on a 512 GB M3 Ultra, which holds Q4_K_M or even Q6_K with a long context. A 256 GB machine needs an IQ3 or smaller quant.
How much VRAM does Qwen 3.5 397B-A17B need?
About 226 GB at Q4_K_M with an 8K context, 394 GB at Q8_0 and 741 GB at FP16. Longer contexts add 0.03 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 3.5 397B-A17B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Apple M3 Ultra 512GB (384 GB usable).
Does Qwen 3.5 397B-A17B need VRAM for all 397B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 17B parameters are read per token, so it generates text about as fast as a 17B dense model.