Qwen 3.5 122B-A10B VRAM Requirements
Qwen 3.5 122B-A10B needs about 70.2 GB of VRAM at Q4_K_M with an 8K context: 68.9 GB for weights, 0.3 GB for the KV cache and 1.0 GB of runtime overhead. At Q8_0 it needs 122 GB and at full FP16 precision 229 GB. The smallest GPU that runs it fully is the A100 80GB.
70.2 GB VRAM
Qwen 3.5 122B-A10B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 68.9 GB
- KV cache 0.3 GB
- Runtime overhead 1.0 GB
Mixture of experts: all 122B parameters need memory, but only 10B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GBToo small
- RTX 3090 24GB32% on GPU~9.1 tok/s
- Radeon RX 7900 XTX 24GB32% on GPU~9.0 tok/s
- RTX 4090 24GB32% on GPU~9.1 tok/s
- RTX 5090 32GB43% on GPU~11 tok/s
- RTX 6000 Ada 48GB64% on GPU~15 tok/s
- A100 80GBFits~105 tok/s
- H100 80GB SXMFits~130 tok/s
- Ryzen AI Max+ 395 128GBFits~22 tok/s
- Apple M4 Max 128GBFits~44 tok/s
- Apple M5 Max 128GBFits~48 tok/s
- RTX PRO 6000 Blackwell 96GBFits~98 tok/s
- NVIDIA DGX Spark 128GBFits~23 tok/s
- Apple M3 Ultra 512GBFits~60 tok/s
Multi-GPU: about 4× 24 GB cards (RTX 3090 / 4090) with the model split across them.
Qwen 3.5 122B-A10B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 68.9 GB | 70.2 GB | 70.8 GB | 80 GB |
| Q5_K_M | 5.69 | 80.8 GB | 82.2 GB | 82.7 GB | 96 GB |
| Q6_K | 6.59 | 93.6 GB | 94.9 GB | 95.5 GB | 128 GB |
| Q8_0 | 8.50 | 121 GB | 122 GB | 123 GB | Multi-GPU / server |
| FP16 / BF16 | 16.00 | 227 GB | 229 GB | 229 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 55.5 GB | 56.9 GB | 57.4 GB | 64 GB |
| Q2_K | 3.00 | 42.6 GB | 44.0 GB | 44.5 GB | 48 GB |
VRAM by context length
At Q4_K_M. Hybrid attention: only 12 of 48 layers use full attention and keep a KV cache (2 KV heads × 256 dimensions). The other 36 use linear attention (Gated DeltaNet) with a fixed state of about 149 MB that does not grow with context.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.2 GB | 0.2 GB | 70.1 GB | 70.1 GB |
| 8K tokens | 0.3 GB | 0.2 GB | 70.2 GB | 70.1 GB |
| 32K tokens | 0.9 GB | 0.5 GB | 70.8 GB | 70.4 GB |
| 128K tokens | 3.1 GB | 1.7 GB | 73.0 GB | 71.6 GB |
| 256K tokens | 6.1 GB | 3.3 GB | 76.0 GB | 73.2 GB |
GPUs that can run Qwen 3.5 122B-A10B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| A100 80GB | 80.0 GB | Fits | Q4_K_M | 219K | ~105 tok/s |
| H100 80GB SXM | 80.0 GB | Fits | Q4_K_M | 219K | ~130 tok/s |
| Ryzen AI Max+ 395 128GB | 96.0 GB | Fits | Q5_K_M | 256K | ~22 tok/s |
| Apple M3 Max 128GB | 96.0 GB | Fits | Q5_K_M | 256K | ~34 tok/s |
| Apple M4 Max 128GB | 96.0 GB | Fits | Q5_K_M | 256K | ~44 tok/s |
| Apple M5 Max 128GB | 96.0 GB | Fits | Q5_K_M | 256K | ~48 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | Fits | Q5_K_M | 256K | ~98 tok/s |
| NVIDIA DGX Spark 128GB | 112 GB | Fits | Q6_K | 256K | ~23 tok/s |
| NVIDIA H200 141GB | 141 GB | Fits | Q8_0 | 256K | ~147 tok/s |
| Apple M2 Ultra 192GB | 144 GB | Fits | Q8_0 | 256K | ~59 tok/s |
| AMD Instinct MI300X 192GB | 192 GB | Fits | Q8_0 | 256K | ~144 tok/s |
| Apple M3 Ultra 512GB | 384 GB | Fits | FP16 / BF16 | 256K | ~60 tok/s |
| RTX 6000 Ada 48GB | 48.0 GB | 64% on GPU | Q2_K | – | ~15 tok/s |
| RTX A6000 48GB | 48.0 GB | 64% on GPU | Q2_K | – | ~15 tok/s |
| L40S 48GB | 48.0 GB | 64% on GPU | Q2_K | – | ~15 tok/s |
| A100 40GB | 40.0 GB | 54% on GPU | – | – | ~13 tok/s |
| RTX 5090 32GB | 32.0 GB | 43% on GPU | – | – | ~11 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | 43% on GPU | – | – | ~10 tok/s |
Multi-GPU: about 4× 24 GB cards (RTX 3090 or 4090) run it at Q4_K_M, using tensor or layer splitting in llama.cpp, vLLM or ExLlamaV2.
About Qwen 3.5 122B-A10B
A mid-size MoE aimed at 96–128 GB machines. With about 10B active parameters it generates text roughly as fast as a 10B dense model, while needing memory for all 122B. The hybrid attention leaves only 12 layers with a KV cache, so even a 128K context adds just a few gigabytes.
| Spec | Value |
|---|---|
| Parameters | 122B total, 10B active (MoE) |
| Layers | 48 |
| Hidden size | 3072 |
| Attention heads / KV heads | 32 / 2 |
| Head dimension | 256 |
| Vocabulary | 248,320 |
| Max context (used here) | 256K tokens |
| Linear-attention layers | 36 of 48 (Gated DeltaNet, fixed 149 MB state) |
| KV cache per token (FP16) | 24 KB |
| Released | 2026-02 |
| License | Apache 2.0 |
Qwen 3.5 122B-A10B VRAM: common questions
What hardware runs Qwen 3.5 122B-A10B locally?
At Q4_K_M it needs about 70 GB, so a 96 GB or 128 GB Mac, a Ryzen AI Max+ 395 system, DGX Spark, an RTX Pro 6000 or three to four 24 GB GPUs.
How much VRAM does Qwen 3.5 122B-A10B need?
About 70.2 GB at Q4_K_M with an 8K context, 122 GB at Q8_0 and 229 GB at FP16. Longer contexts add 0.02 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 3.5 122B-A10B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the A100 80GB (80.0 GB usable). With 24 GB cards you need about 4 of them.
Does Qwen 3.5 122B-A10B need VRAM for all 122B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 10B parameters are read per token, so it generates text about as fast as a 10B dense model.