Qwen 2.5 14B VRAM Requirements
Qwen 2.5 14B needs about 10.7 GB of VRAM at Q4_K_M with an 8K context: 8.3 GB for weights, 1.5 GB for the KV cache and 0.9 GB of runtime overhead. At Q8_0 it needs 16.9 GB and at full FP16 precision 29.7 GB. The smallest GPU that runs it fully is the Apple M4 16GB.
10.7 GB VRAM
Qwen 2.5 14B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 8.3 GB
- KV cache 1.5 GB
- Runtime overhead 0.9 GB
Which GPUs can run it
- RTX 3060 12GBFits~24 tok/s
- RTX 4060 Ti 16GBFits~19 tok/s
- RTX 3090 24GBFits~61 tok/s
- Radeon RX 7900 XTX 24GBFits~53 tok/s
- RTX 4090 24GBFits~65 tok/s
- RTX 5090 32GBFits~113 tok/s
- RTX 6000 Ada 48GBFits~62 tok/s
- A100 80GBFits~128 tok/s
- H100 80GB SXMFits~202 tok/s
- Ryzen AI Max+ 395 128GBFits~16 tok/s
- Apple M4 Max 128GBFits~36 tok/s
- Apple M5 Max 128GBFits~40 tok/s
- RTX PRO 6000 Blackwell 96GBFits~113 tok/s
- NVIDIA DGX Spark 128GBFits~17 tok/s
- Apple M3 Ultra 512GBFits~53 tok/s
Qwen 2.5 14B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 8.3 GB | 10.7 GB | 15.2 GB | 12 GB |
| Q5_K_M | 5.69 | 9.7 GB | 12.1 GB | 16.6 GB | 16 GB |
| Q6_K | 6.59 | 11.3 GB | 13.6 GB | 18.1 GB | 16 GB |
| Q8_0 | 8.50 | 14.5 GB | 16.9 GB | 21.4 GB | 20 GB |
| FP16 / BF16 | 16.00 | 27.4 GB | 29.7 GB | 34.2 GB | 32 GB |
| Q3_K_M | 3.91 | 6.7 GB | 9.1 GB | 13.6 GB | 10 GB |
| Q2_K | 3.00 | 5.1 GB | 7.5 GB | 12.0 GB | 8 GB |
VRAM by context length
At Q4_K_M. The KV cache grows linearly with context: 0.19 GB per 1,000 tokens in FP16.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.8 GB | 0.4 GB | 9.9 GB | 9.6 GB |
| 8K tokens | 1.5 GB | 0.8 GB | 10.7 GB | 10.0 GB |
| 32K tokens | 6.0 GB | 3.2 GB | 15.2 GB | 12.4 GB |
GPUs that can run Qwen 2.5 14B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M4 16GB | 10.7 GB | Tight fit | IQ4_XS | 4K | ~8.0 tok/s |
| RTX 2080 Ti 11GB | 11.0 GB | Tight fit | IQ4_XS | 6K | ~40 tok/s |
| RTX 2060 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~22 tok/s |
| RTX 3060 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~24 tok/s |
| Radeon RX 6700 XT 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~22 tok/s |
| Radeon RX 7700 XT 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~24 tok/s |
| Intel Arc B580 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~23 tok/s |
| RTX 4070 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~33 tok/s |
| RTX 4070 Super 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~33 tok/s |
| RTX 4070 Ti 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~33 tok/s |
| RTX 5070 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~44 tok/s |
| RTX 3080 12GB | 12.0 GB | Fits | Q4_K_M | 11K | ~59 tok/s |
| Apple M4 Pro 24GB | 16.0 GB | Fits | Q6_K | 31K | ~18 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | Fits | Q6_K | 31K | ~19 tok/s |
| RTX 3080 10GB | 10.0 GB | 88% on GPU | Q3_K_M | 1K | ~22 tok/s |
| Intel Arc B570 10GB | 10.0 GB | 88% on GPU | Q3_K_M | 1K | ~14 tok/s |
| RTX 3050 8GB | 8.0 GB | 70% on GPU | Q2_K | – | ~8.6 tok/s |
| RTX 3060 Ti 8GB | 8.0 GB | 70% on GPU | Q2_K | – | ~11 tok/s |
| RTX 3070 8GB | 8.0 GB | 70% on GPU | Q2_K | – | ~11 tok/s |
| RTX 3070 Ti 8GB | 8.0 GB | 70% on GPU | Q2_K | – | ~12 tok/s |
About Qwen 2.5 14B
A sweet spot for 12 GB and 16 GB cards. At Q4_K_M the weights are about 8.3 GB, leaving room for context on a 12 GB GPU. Its 48 layers give a KV cache of 192 KB per token in FP16.
| Spec | Value |
|---|---|
| Parameters | 14.7B |
| Layers | 48 |
| Hidden size | 5120 |
| Attention heads / KV heads | 40 / 8 |
| Head dimension | 128 |
| Vocabulary | 152,064 |
| Max context (used here) | 32K tokens |
| KV cache per token (FP16) | 192 KB |
| Released | 2024-09 |
| License | Apache 2.0 |
Qwen 2.5 14B VRAM: common questions
Does Qwen 2.5 14B fit on an RTX 3060 12GB?
Yes at Q4_K_M with an 8K context (about 10.7 GB). For 32K context, quantize the KV cache to Q8_0 or use a 16 GB card.
How much VRAM does Qwen 2.5 14B need?
About 10.7 GB at Q4_K_M with an 8K context, 16.9 GB at Q8_0 and 29.7 GB at FP16. Longer contexts add 0.19 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Qwen 2.5 14B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Apple M4 16GB (10.7 GB usable).