GGUF Size Calculator: File Size for Every Quantization
Find the download size and memory needed for each GGUF quantization of a model, or enter any parameter count. A 70B model is about 43 GB at Q4_K_M and 75 GB at Q8_0.
4.9 GB Q4_K_M file
Llama 3.1 8B (8.03B parameters). File sizes are in decimal GB, as shown on Hugging Face. VRAM figures are in GiB at 8K context.
Bits per weight by format
K-quants mix precisions: Q4_K_M keeps attention and output tensors at higher precision, so it averages about 4.85 bits per weight rather than 4. The values used here are typical averages for Llama-style models; very small models with large vocabularies come out slightly bigger.
| Format | Bits/weight | Typical use |
|---|---|---|
| Q8_0 | 8.5 | Near-lossless; when memory allows |
| Q6_K | 6.59 | Very close to Q8_0, about 22% smaller |
| Q5_K_M | 5.69 | Good balance for 12–16 GB cards |
| Q4_K_M | 4.85 | The default most people use |
| IQ4_XS | 4.25 | Smaller than Q4_K_M at similar quality; slower on some CPUs |
| Q3_K_M | 3.91 | Noticeable loss; for squeezing large models in |
| Q2_K | ≈3.0 | Large quality loss; last resort for 70B+ models |
Read more in Q4 vs Q5 vs Q8: quantization explained.
Frequently asked questions
Why is my GGUF file a different size from this table?
Quantizers choose per-tensor types differently, and Hugging Face shows decimal GB while your OS may show GiB (7% smaller numbers). Expect ±5%.
How much bigger is VRAM use than the file size?
The weights take roughly the file size in memory. On top of that come the KV cache (set by context length) and 0.6–1.2 GB of runtime buffers.