Qwen 3.5 35B-A3B VRAM Requirements
Qwen 3.5 35B-A3B needs about 21.0 GB of VRAM at Q4_K_M with an 8K context: 19.8 GB for weights, 0.2 GB for the KV cache and 1.0 GB of runtime overhead. At Q8_0 it needs 35.9 GB and at full FP16 precision 66.4 GB. The smallest GPU that runs it fully is the Apple M5 32GB.
21.0 GB VRAM
Qwen 3.5 35B-A3B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 19.8 GB
- KV cache 0.2 GB
- Runtime overhead 1.0 GB
Mixture of experts: all 35B parameters need memory, but only 3B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GB54% on GPU~33 tok/s
- RTX 4060 Ti 16GB72% on GPU~40 tok/s
- RTX 3090 24GBFits~138 tok/s
- Radeon RX 7900 XTX 24GBFits~129 tok/s
- RTX 4090 24GBFits~143 tok/s
- RTX 5090 32GBFits~176 tok/s
- RTX 6000 Ada 48GBFits~140 tok/s
- A100 80GBFits~182 tok/s
- H100 80GB SXMFits~204 tok/s
- Ryzen AI Max+ 395 128GBFits~59 tok/s
- Apple M4 Max 128GBFits~105 tok/s
- Apple M5 Max 128GBFits~112 tok/s
- RTX PRO 6000 Blackwell 96GBFits~176 tok/s
- NVIDIA DGX Spark 128GBFits~62 tok/s
- Apple M3 Ultra 512GBFits~130 tok/s
Qwen 3.5 35B-A3B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 19.8 GB | 21.0 GB | 21.5 GB | 24 GB |
| Q5_K_M | 5.69 | 23.2 GB | 24.4 GB | 24.9 GB | 32 GB |
| Q6_K | 6.59 | 26.9 GB | 28.1 GB | 28.5 GB | 32 GB |
| Q8_0 | 8.50 | 34.6 GB | 35.9 GB | 36.3 GB | 48 GB |
| FP16 / BF16 | 16.00 | 65.2 GB | 66.4 GB | 66.9 GB | 80 GB |
| Q3_K_M | 3.91 | 15.9 GB | 17.2 GB | 17.6 GB | 20 GB |
| Q2_K | 3.00 | 12.2 GB | 13.4 GB | 13.9 GB | 16 GB |
VRAM by context length
At Q4_K_M. Hybrid attention: only 10 of 40 layers use full attention and keep a KV cache (2 KV heads × 256 dimensions). The other 30 use linear attention (Gated DeltaNet) with a fixed state of about 63 MB that does not grow with context.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.1 GB | 0.1 GB | 20.9 GB | 20.9 GB |
| 8K tokens | 0.2 GB | 0.1 GB | 21.0 GB | 20.9 GB |
| 32K tokens | 0.7 GB | 0.4 GB | 21.5 GB | 21.2 GB |
| 128K tokens | 2.6 GB | 1.4 GB | 23.3 GB | 22.2 GB |
| 256K tokens | 5.1 GB | 2.7 GB | 25.8 GB | 23.5 GB |
GPUs that can run Qwen 3.5 35B-A3B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M5 32GB | 21.3 GB | Tight fit | IQ4_XS | – | ~42 tok/s |
| NVIDIA L4 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~71 tok/s |
| Tesla P40 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~70 tok/s |
| Apple M4 Max 36GB | 24.0 GB | Fits | Q4_K_M | 88K | ~88 tok/s |
| Intel Arc Pro B60 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~79 tok/s |
| RTX 3090 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~138 tok/s |
| Radeon RX 7900 XTX 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~129 tok/s |
| RTX 3090 Ti 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~143 tok/s |
| RTX 4090 24GB | 24.0 GB | Fits | Q4_K_M | 88K | ~143 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | Fits | Q6_K | 256K | ~104 tok/s |
| RTX 5090 32GB | 32.0 GB | Fits | Q6_K | 256K | ~176 tok/s |
| Apple M4 Pro 48GB | 36.0 GB | Fits | Q6_K | 256K | ~66 tok/s |
| A100 40GB | 40.0 GB | Fits | Q8_0 | 256K | ~168 tok/s |
| Apple M5 Pro 64GB | 48.0 GB | Fits | Q8_0 | 256K | ~72 tok/s |
| Radeon RX 7900 XT 20GB | 20.0 GB | 90% on GPU | IQ4_XS | – | ~78 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | 72% on GPU | Q2_K | – | ~40 tok/s |
| RTX 4070 Ti Super 16GB | 16.0 GB | 72% on GPU | Q2_K | – | ~49 tok/s |
| RTX 4080 16GB | 16.0 GB | 72% on GPU | Q2_K | – | ~49 tok/s |
| RTX 4080 Super 16GB | 16.0 GB | 72% on GPU | Q2_K | – | ~49 tok/s |
| RTX 5060 Ti 16GB | 16.0 GB | 72% on GPU | Q2_K | – | ~45 tok/s |
About Qwen 3.5 35B-A3B
The successor to Qwen 3 30B-A3B: 256 experts with 8 routed plus one shared expert per token, about 3B active. Only 10 of 40 layers keep a KV cache, with just 2 KV heads, so the cache is tiny. It is fast on almost anything that can hold the weights, and it stays usable with partial CPU offload.
| Spec | Value |
|---|---|
| Parameters | 35B total, 3B active (MoE) |
| Layers | 40 |
| Hidden size | 2048 |
| Attention heads / KV heads | 16 / 2 |
| Head dimension | 256 |
| Vocabulary | 248,320 |
| Max context (used here) | 256K tokens |
| Linear-attention layers | 30 of 40 (Gated DeltaNet, fixed 63 MB state) |
| KV cache per token (FP16) | 20 KB |
| Released | 2026-02 |
| License | Apache 2.0 |
Qwen 3.5 35B-A3B VRAM: common questions
Can Qwen 3.5 35B-A3B run on a 16GB GPU?
Not fully at Q4_K_M, which needs about 21 GB. Because only about 3B parameters are read per token, offloading the expert layers to system RAM still gives usable speed, and IQ3-level quants fit fully.
How much VRAM does Qwen 3.5 35B-A3B need?
About 21.0 GB at Q4_K_M with an 8K context, 35.9 GB at Q8_0 and 66.4 GB at FP16. Longer contexts add 0.02 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 3.5 35B-A3B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Apple M5 32GB (21.3 GB usable).
Does Qwen 3.5 35B-A3B need VRAM for all 35B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 3B parameters are read per token, so it generates text about as fast as a 3B dense model.