IQ4_XS vs Q4_K_M: VRAM, Quality and Speed
Q4_K_M uses about 14% more memory than IQ4_XS (4.85 vs 4.25 bits per weight). On Qwen 3 32B with an 8K context that is 21.4 GB versus 19.1 GB. Use IQ4_XS when Q4_K_M is just slightly too large for your GPU with the context you want; it saves about 12% for very little quality loss.
VRAM on popular models
Total memory with an 8K context and FP16 KV cache. The last column is the smallest standard VRAM size that holds each version.
| Model | IQ4_XS | Q4_K_M | Difference | Smallest tier |
|---|---|---|---|---|
| Llama 3.1 8B | 5.8 GB | 6.3 GB | +0.6 GB | 8 GB / 8 GB |
| Qwen 3.5 9B | 5.8 GB | 6.4 GB | +0.6 GB | 8 GB / 8 GB |
| Qwen 3 14B | 9.4 GB | 10.5 GB | +1.0 GB | 12 GB / 12 GB |
| Mistral Small 3.1 24B | 14.0 GB | 15.6 GB | +1.7 GB | 16 GB / 20 GB |
| Qwen 3.6 27B | 15.0 GB | 16.9 GB | +1.9 GB | 16 GB / 20 GB |
| Gemma 4 31B | 17.7 GB | 19.8 GB | +2.1 GB | 20 GB / 24 GB |
| Qwen 3 32B | 19.1 GB | 21.4 GB | +2.3 GB | 24 GB / 24 GB |
| Qwen 3.6 35B-A3B | 18.5 GB | 21.0 GB | +2.4 GB | 20 GB / 24 GB |
| Llama 3.3 70B | 38.3 GB | 43.2 GB | +4.9 GB | 48 GB / 48 GB |
| Qwen 3.5 122B-A10B | 61.7 GB | 70.2 GB | +8.5 GB | 80 GB / 80 GB |
Largest model each VRAM size can run
The biggest model released since 2025 that fits fully at 8K context.
Context you can fit
Longer contexts need room for the KV cache. The longest context that fits fully at each format, with an FP16 cache:
| Model | GPU | IQ4_XS | Q4_K_M |
|---|---|---|---|
| Qwen 3 14B | 16 GB | 40K | 37K |
| Qwen 3.6 27B | 24 GB | 128K | 98K |
| Qwen 3 32B | 24 GB | 21K | 12K |
| Gemma 4 31B | 32 GB | 166K | 139K |
| Llama 3.3 70B | 48 GB | 29K | 14K |
Download size
File sizes in decimal GB, as Hugging Face shows them. Use the GGUF size calculator for any other model.
| Model | IQ4_XS | Q4_K_M |
|---|---|---|
| Llama 3.1 8B | 4.3 GB | 4.9 GB |
| Qwen 3 14B | 7.9 GB | 9.0 GB |
| Qwen 3.6 27B | 14.3 GB | 16.3 GB |
| Qwen 3 32B | 17.4 GB | 19.9 GB |
| Llama 3.3 70B | 37.5 GB | 42.8 GB |
Quality
IQ4_XS is an i-quant: it uses a non-linear grid and usually an importance matrix (imatrix) calibrated on sample text. At about 4.25 bits per weight it lands close to Q4_K_M in quality in community perplexity tests, while being about 12% smaller. It can be slower than Q4_K_M on some CPUs and on Apple Silicon, where k-quants have more optimized kernels.
Speed
Text generation reads every weight once per token, so smaller formats generate faster on the same GPU. Estimated on an RTX 4090 24GB:
| Model | IQ4_XS | Q4_K_M |
|---|---|---|
| Llama 3.1 8B | ~131 tok/s | ~117 tok/s |
| Qwen 3 32B | ~35 tok/s | ~31 tok/s |
Verdict
Use IQ4_XS when Q4_K_M is just slightly too large for your GPU with the context you want; it saves about 12% for very little quality loss. When both fit, Q4_K_M is the safer default and is often a little faster outside CUDA.
Frequently asked questions
What is an imatrix quant?
An importance matrix records which weights matter most on sample text. Quantization then spends more precision on those weights. Most IQ quants and many modern Q4_K_M files use one.
How much more VRAM does Q4_K_M need than IQ4_XS?
About 14% more for the weights (4.85 vs 4.25 bits per weight). For Qwen 3 32B with an 8K context that is 21.4 GB versus 19.1 GB.