Qwen 3.5 4B VRAM Requirements

Qwen 3.5 4B needs about 3.7 GB of VRAM at Q4_K_M with an 8K context: 2.4 GB for weights, 0.3 GB for the KV cache and 1.0 GB of runtime overhead. At Q8_0 it needs 5.5 GB and at full FP16 precision 9.1 GB. The smallest GPU that runs it fully is the RTX 3050 8GB.

By Alibaba · Updated October 14, 2026 · Values from config.json on Hugging Face

3.7 GB VRAM

Qwen 3.5 4B at Q4_K_M with 8K context and the KV cache in FP16.

  • Weights 2.4 GB
  • KV cache 0.3 GB
  • Runtime overhead 1.0 GB

Which GPUs can run it

Qwen 3.5 4B VRAM by quantization

Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.

QuantizationBits/weightWeightsTotal at 8KTotal at 32KSmallest tier
Q4_K_M4.852.4 GB3.7 GB4.4 GB4 GB
Q5_K_M5.692.8 GB4.1 GB4.8 GB6 GB
Q6_K6.593.2 GB4.5 GB5.3 GB6 GB
Q8_08.504.2 GB5.5 GB6.2 GB6 GB
FP16 / BF1616.007.8 GB9.1 GB9.9 GB10 GB
Q3_K_M3.911.9 GB3.2 GB4.0 GB4 GB
Q2_K3.001.5 GB2.8 GB3.5 GB4 GB

VRAM by context length

At Q4_K_M. Hybrid attention: only 8 of 32 layers use full attention and keep a KV cache (4 KV heads × 256 dimensions). The other 24 use linear attention (Gated DeltaNet) with a fixed state of about 50 MB that does not grow with context.

ContextKV cache (FP16)KV cache (Q8_0)Total, FP16 cacheTotal, Q8_0 cache
4K tokens0.2 GB0.1 GB3.6 GB3.5 GB
8K tokens0.3 GB0.2 GB3.7 GB3.6 GB
32K tokens1.0 GB0.6 GB4.4 GB4.0 GB
128K tokens4.0 GB2.2 GB7.4 GB5.6 GB
256K tokens8.0 GB4.3 GB11.4 GB7.7 GB

GPUs that can run Qwen 3.5 4B

At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.

GPUUsable memoryStatusBest quantMax contextSpeed
RTX 3050 8GB8.0 GBFitsQ8_0130K~52 tok/s
RTX 4060 8GB8.0 GBFitsQ8_0130K~63 tok/s
RTX 4060 Ti 8GB8.0 GBFitsQ8_0130K~67 tok/s
Radeon RX 7600 8GB8.0 GBFitsQ8_0130K~57 tok/s
RTX 3060 Ti 8GB8.0 GBFitsQ8_0130K~103 tok/s
RTX 3070 8GB8.0 GBFitsQ8_0130K~103 tok/s
RTX 5060 8GB8.0 GBFitsQ8_0130K~103 tok/s
RTX 5060 Ti 8GB8.0 GBFitsQ8_0130K~103 tok/s
RTX 3070 Ti 8GB8.0 GBFitsQ8_0130K~138 tok/s
Intel Arc B570 10GB10.0 GBFitsFP16 / BF16190K~68 tok/s
RTX 3080 10GB10.0 GBFitsFP16 / BF16190K~171 tok/s
Apple M4 16GB10.7 GBFitsFP16 / BF16211K~28 tok/s
RTX 2080 Ti 11GB11.0 GBFitsFP16 / BF16221K~140 tok/s
RTX 2060 12GB12.0 GBFitsFP16 / BF16251K~78 tok/s

About Qwen 3.5 4B

The small member of the Qwen 3.5 family, with the same hybrid design as its big siblings: three Gated DeltaNet (linear attention) layers for every full-attention layer. Only 8 of its 32 layers keep a KV cache, so a 32K or even 128K context costs a fraction of what it does on Qwen 3 4B. It is natively multimodal; the vision encoder is a separate file (mmproj) in GGUF builds.

SpecValue
Parameters4.2B
Layers32
Hidden size2560
Attention heads / KV heads16 / 4
Head dimension256
Vocabulary248,320
Max context (used here)256K tokens
Linear-attention layers24 of 32 (Gated DeltaNet, fixed 50 MB state)
KV cache per token (FP16)32 KB
Released2026-03
LicenseApache 2.0

Qwen 3.5 4B VRAM: common questions

Is Qwen 3.5 4B better than Qwen 3 4B for low-VRAM GPUs?

For long contexts, yes. The weights are about the same size, but only a quarter of the layers store a KV cache, so 32K of context costs about a quarter of what Qwen 3 4B needs.

How much VRAM does Qwen 3.5 4B need?

About 3.7 GB at Q4_K_M with an 8K context, 5.5 GB at Q8_0 and 9.1 GB at FP16. Longer contexts add 0.03 GB per 1,000 tokens with an FP16 cache.

What is the cheapest GPU that can run Qwen 3.5 4B?

The smallest listed device that fits it fully at Q4_K_M with 8K context is the RTX 3050 8GB (8.0 GB usable).

Related