Q3_K_M vs Q4_K_M: VRAM, Quality and Speed
Q4_K_M uses about 24% more memory than Q3_K_M (4.85 vs 3.91 bits per weight). On Qwen 3 32B with an 8K context that is 21.4 GB versus 17.8 GB. Treat Q3_K_M as a way to fit a large model that would not otherwise fit, not as a default.
VRAM on popular models
Total memory with an 8K context and FP16 KV cache. The last column is the smallest standard VRAM size that holds each version.
| Model | Q3_K_M | Q4_K_M | Difference | Smallest tier |
|---|---|---|---|---|
| Llama 3.1 8B | 5.5 GB | 6.3 GB | +0.9 GB | 6 GB / 8 GB |
| Qwen 3.5 9B | 5.4 GB | 6.4 GB | +1.0 GB | 6 GB / 8 GB |
| Qwen 3 14B | 8.9 GB | 10.5 GB | +1.6 GB | 10 GB / 12 GB |
| Mistral Small 3.1 24B | 13.0 GB | 15.6 GB | +2.6 GB | 16 GB / 20 GB |
| Qwen 3.6 27B | 13.9 GB | 16.9 GB | +2.9 GB | 16 GB / 20 GB |
| Gemma 4 31B | 16.5 GB | 19.8 GB | +3.4 GB | 20 GB / 24 GB |
| Qwen 3 32B | 17.8 GB | 21.4 GB | +3.6 GB | 20 GB / 24 GB |
| Qwen 3.6 35B-A3B | 17.2 GB | 21.0 GB | +3.8 GB | 20 GB / 24 GB |
| Llama 3.3 70B | 35.5 GB | 43.2 GB | +7.7 GB | 48 GB / 48 GB |
| Qwen 3.5 122B-A10B | 56.9 GB | 70.2 GB | +13.4 GB | 64 GB / 80 GB |
Largest model each VRAM size can run
The biggest model released since 2025 that fits fully at 8K context.
Context you can fit
Longer contexts need room for the KV cache. The longest context that fits fully at each format, with an FP16 cache:
| Model | GPU | Q3_K_M | Q4_K_M |
|---|---|---|---|
| Qwen 3 14B | 16 GB | 40K | 37K |
| Qwen 3.6 27B | 24 GB | 145K | 98K |
| Qwen 3 32B | 24 GB | 27K | 12K |
| Gemma 4 31B | 32 GB | 182K | 139K |
| Llama 3.3 70B | 48 GB | 38K | 14K |
Download size
File sizes in decimal GB, as Hugging Face shows them. Use the GGUF size calculator for any other model.
| Model | Q3_K_M | Q4_K_M |
|---|---|---|
| Llama 3.1 8B | 3.9 GB | 4.9 GB |
| Qwen 3 14B | 7.2 GB | 9.0 GB |
| Qwen 3.6 27B | 13.1 GB | 16.3 GB |
| Qwen 3 32B | 16.0 GB | 19.9 GB |
| Llama 3.3 70B | 34.5 GB | 42.8 GB |
Quality
Quality drops faster below 4 bits. llama.cpp's reference measurement shows Q3_K_M adding about 0.25 perplexity over FP16 on a 7B model, roughly five times the loss of Q4_K_M. Large models tolerate it far better than small ones: a 70B model at Q3_K_M is still strong, while an 8B model becomes noticeably less reliable.
Speed
Text generation reads every weight once per token, so smaller formats generate faster on the same GPU. Estimated on an RTX 4090 24GB:
| Model | Q3_K_M | Q4_K_M |
|---|---|---|
| Llama 3.1 8B | ~140 tok/s | ~117 tok/s |
| Qwen 3 32B | ~37 tok/s | ~31 tok/s |
Verdict
Treat Q3_K_M as a way to fit a large model that would not otherwise fit, not as a default. For models of 30B and up it can be the right trade; for models under 14B, use Q4_K_M or pick a smaller model instead.
Frequently asked questions
Is a 70B model at Q3 better than a 32B model at Q4?
Often yes for knowledge and reasoning, but the 70B at Q3_K_M needs about 36 GB versus about 21 GB for the 32B at Q4_K_M. Compare against what fits your GPU.
How much more VRAM does Q4_K_M need than Q3_K_M?
About 24% more for the weights (4.85 vs 3.91 bits per weight). For Qwen 3 32B with an 8K context that is 21.4 GB versus 17.8 GB.