Llama 3.2 1B VRAM Requirements
Llama 3.2 1B needs about 1.7 GB of VRAM at Q4_K_M with an 8K context: 0.7 GB for weights, 0.3 GB for the KV cache and 0.8 GB of runtime overhead. At Q8_0 it needs 2.3 GB and at full FP16 precision 3.3 GB. The smallest GPU that runs it fully is the RTX 3050 8GB.
1.7 GB VRAM
Llama 3.2 1B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 0.7 GB
- KV cache 0.3 GB
- Runtime overhead 0.8 GB
Which GPUs can run it
- RTX 3060 12GBFits~253 tok/s
- RTX 4060 Ti 16GBFits~204 tok/s
- RTX 3090 24GBFits~619 tok/s
- Radeon RX 7900 XTX 24GBFits~544 tok/s
- RTX 4090 24GBFits~661 tok/s
- RTX 5090 32GBFits~1086 tok/s
- RTX 6000 Ada 48GBFits~633 tok/s
- A100 80GBFits~1207 tok/s
- H100 80GB SXMFits~1764 tok/s
- Ryzen AI Max+ 395 128GBFits~169 tok/s
- Apple M4 Max 128GBFits~376 tok/s
- Apple M5 Max 128GBFits~420 tok/s
- RTX PRO 6000 Blackwell 96GBFits~1086 tok/s
- NVIDIA DGX Spark 128GBFits~180 tok/s
- Apple M3 Ultra 512GBFits~548 tok/s
Llama 3.2 1B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 0.7 GB | 1.7 GB | 2.5 GB | 4 GB |
| Q5_K_M | 5.69 | 0.8 GB | 1.8 GB | 2.6 GB | 4 GB |
| Q6_K | 6.59 | 1.0 GB | 2.0 GB | 2.7 GB | 4 GB |
| Q8_0 | 8.50 | 1.2 GB | 2.3 GB | 3.0 GB | 4 GB |
| FP16 / BF16 | 16.00 | 2.3 GB | 3.3 GB | 4.1 GB | 4 GB |
| Q3_K_M | 3.91 | 0.6 GB | 1.6 GB | 2.3 GB | 4 GB |
| Q2_K | 3.00 | 0.4 GB | 1.5 GB | 2.2 GB | 4 GB |
VRAM by context length
At Q4_K_M. The KV cache grows linearly with context: 0.03 GB per 1,000 tokens in FP16.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.1 GB | 68 MB | 1.6 GB | 1.5 GB |
| 8K tokens | 0.3 GB | 0.1 GB | 1.7 GB | 1.6 GB |
| 32K tokens | 1.0 GB | 0.5 GB | 2.5 GB | 2.0 GB |
| 128K tokens | 4.0 GB | 2.1 GB | 5.5 GB | 3.6 GB |
GPUs that can run Llama 3.2 1B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| RTX 3050 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~160 tok/s |
| RTX 4060 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~193 tok/s |
| RTX 4060 Ti 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~204 tok/s |
| Radeon RX 7600 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~174 tok/s |
| RTX 3060 Ti 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~312 tok/s |
| RTX 3070 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~312 tok/s |
| RTX 5060 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~312 tok/s |
| RTX 5060 Ti 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~312 tok/s |
| RTX 3070 Ti 8GB | 8.0 GB | Fits | FP16 / BF16 | 128K | ~416 tok/s |
| Intel Arc B570 10GB | 10.0 GB | Fits | FP16 / BF16 | 128K | ~207 tok/s |
| RTX 3080 10GB | 10.0 GB | Fits | FP16 / BF16 | 128K | ~512 tok/s |
| Apple M4 16GB | 10.7 GB | Fits | FP16 / BF16 | 128K | ~87 tok/s |
| RTX 2080 Ti 11GB | 11.0 GB | Fits | FP16 / BF16 | 128K | ~421 tok/s |
| RTX 2060 12GB | 12.0 GB | Fits | FP16 / BF16 | 128K | ~237 tok/s |
About Llama 3.2 1B
Meta's smallest Llama 3 model, built for phones and edge devices. The 128K-token vocabulary is a large share of its 1.24B parameters, so it quantizes less efficiently than its size suggests. Useful as a draft model for speculative decoding with larger Llama models.
| Spec | Value |
|---|---|
| Parameters | 1.24B |
| Layers | 16 |
| Hidden size | 2048 |
| Attention heads / KV heads | 32 / 8 |
| Head dimension | 64 |
| Vocabulary | 128,256 |
| Max context (used here) | 128K tokens |
| KV cache per token (FP16) | 32 KB |
| Released | 2024-09 |
| License | Llama 3.2 Community License |
Llama 3.2 1B VRAM: common questions
Can Llama 3.2 1B run without a GPU?
Yes. At Q4_K_M the weights are under 1 GB, so any modern CPU runs it at usable speed. A GPU only matters if you need very high throughput.
How much VRAM does Llama 3.2 1B need?
About 1.7 GB at Q4_K_M with an 8K context, 2.3 GB at Q8_0 and 3.3 GB at FP16. Longer contexts add 0.03 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Llama 3.2 1B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the RTX 3050 8GB (8.0 GB usable).