Llama 3.3 70B VRAM Requirements
Llama 3.3 70B needs about 43.2 GB of VRAM at Q4_K_M with an 8K context: 39.9 GB for weights, 2.5 GB for the KV cache and 0.9 GB of runtime overhead. At Q8_0 it needs 73.2 GB and at full FP16 precision 135 GB. The smallest GPU that runs it fully is the Apple M5 Pro 64GB.
43.2 GB VRAM
Llama 3.3 70B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 39.9 GB
- KV cache 2.5 GB
- Runtime overhead 0.9 GB
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GB35% on GPU~1.3 tok/s
- RTX 3090 24GB52% on GPU~1.8 tok/s
- Radeon RX 7900 XTX 24GB52% on GPU~1.8 tok/s
- RTX 4090 24GB52% on GPU~1.9 tok/s
- RTX 5090 32GB70% on GPU~2.9 tok/s
- RTX 6000 Ada 48GBFits~14 tok/s
- A100 80GBFits~29 tok/s
- H100 80GB SXMFits~47 tok/s
- Ryzen AI Max+ 395 128GBFits~3.5 tok/s
- Apple M4 Max 128GBFits~8.0 tok/s
- Apple M5 Max 128GBFits~9.0 tok/s
- RTX PRO 6000 Blackwell 96GBFits~26 tok/s
- NVIDIA DGX Spark 128GBFits~3.7 tok/s
- Apple M3 Ultra 512GBFits~12 tok/s
Multi-GPU: about 2× 24 GB cards (RTX 3090 / 4090) with the model split across them.
Llama 3.3 70B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 39.9 GB | 43.2 GB | 50.7 GB | 48 GB |
| Q5_K_M | 5.69 | 46.8 GB | 50.1 GB | 57.6 GB | 64 GB |
| Q6_K | 6.59 | 54.2 GB | 57.5 GB | 65.0 GB | 64 GB |
| Q8_0 | 8.50 | 69.9 GB | 73.2 GB | 80.7 GB | 80 GB |
| FP16 / BF16 | 16.00 | 132 GB | 135 GB | 142 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 32.1 GB | 35.5 GB | 43.0 GB | 48 GB |
| Q2_K | 3.00 | 24.7 GB | 28.0 GB | 35.5 GB | 32 GB |
VRAM by context length
At Q4_K_M. The KV cache grows linearly with context: 0.31 GB per 1,000 tokens in FP16.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 1.3 GB | 0.7 GB | 42.0 GB | 41.4 GB |
| 8K tokens | 2.5 GB | 1.3 GB | 43.2 GB | 42.1 GB |
| 32K tokens | 10.0 GB | 5.3 GB | 50.7 GB | 46.0 GB |
| 128K tokens | 40.0 GB | 21.3 GB | 80.7 GB | 62.0 GB |
GPUs that can run Llama 3.3 70B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M5 Pro 64GB | 48.0 GB | Fits | Q4_K_M | 14K | ~4.5 tok/s |
| Apple M1 Max 64GB | 48.0 GB | Fits | Q4_K_M | 14K | ~5.9 tok/s |
| Apple M4 Max 64GB | 48.0 GB | Fits | Q4_K_M | 14K | ~8.0 tok/s |
| RTX A6000 48GB | 48.0 GB | Fits | Q4_K_M | 14K | ~11 tok/s |
| L40S 48GB | 48.0 GB | Fits | Q4_K_M | 14K | ~13 tok/s |
| RTX 6000 Ada 48GB | 48.0 GB | Fits | Q4_K_M | 14K | ~14 tok/s |
| A100 80GB | 80.0 GB | Fits | Q8_0 | 110K | ~29 tok/s |
| H100 80GB SXM | 80.0 GB | Fits | Q8_0 | 110K | ~47 tok/s |
| Ryzen AI Max+ 395 128GB | 96.0 GB | Fits | Q8_0 | 128K | ~3.5 tok/s |
| Apple M3 Max 128GB | 96.0 GB | Fits | Q8_0 | 128K | ~5.9 tok/s |
| Apple M4 Max 128GB | 96.0 GB | Fits | Q8_0 | 128K | ~8.0 tok/s |
| Apple M5 Max 128GB | 96.0 GB | Fits | Q8_0 | 128K | ~9.0 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | Fits | Q8_0 | 128K | ~26 tok/s |
| NVIDIA DGX Spark 128GB | 112 GB | Fits | Q8_0 | 128K | ~3.7 tok/s |
| A100 40GB | 40.0 GB | 87% on GPU | Q3_K_M | – | ~5.7 tok/s |
| RTX 5090 32GB | 32.0 GB | 70% on GPU | Q2_K | – | ~2.9 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | 70% on GPU | Q2_K | – | ~2.5 tok/s |
| RTX 3090 24GB | 24.0 GB | 52% on GPU | – | – | ~1.8 tok/s |
| RTX 3090 Ti 24GB | 24.0 GB | 52% on GPU | – | – | ~1.9 tok/s |
| RTX 4090 24GB | 24.0 GB | 52% on GPU | – | – | ~1.9 tok/s |
Multi-GPU: about 2× 24 GB cards (RTX 3090 or 4090) run it at Q4_K_M, using tensor or layer splitting in llama.cpp, vLLM or ExLlamaV2.
About Llama 3.3 70B
Same architecture as Llama 3.1 70B with better post-training; Meta positioned it as close to Llama 3.1 405B on many benchmarks. The 80 layers make the KV cache 320 KB per token in FP16, so context length matters more here than on 8B models. The classic setup is two 24 GB cards at Q4_K_M.
| Spec | Value |
|---|---|
| Parameters | 70.6B |
| Layers | 80 |
| Hidden size | 8192 |
| Attention heads / KV heads | 64 / 8 |
| Head dimension | 128 |
| Vocabulary | 128,256 |
| Max context (used here) | 128K tokens |
| KV cache per token (FP16) | 320 KB |
| Released | 2024-12 |
| License | Llama 3.3 Community License |
Llama 3.3 70B VRAM: common questions
Can Llama 3.3 70B run on a single RTX 4090?
Not fully. Q4_K_M needs about 43 GB with an 8K context. On 24 GB you need Q2_K (still over 24 GB) or heavy CPU offload, which drops speed to a few tokens per second.
What is the cheapest way to run Llama 3.3 70B?
Two used RTX 3090s (48 GB total) at Q4_K_M, or a Mac or Ryzen AI Max system with 64 GB or more of unified memory.
How much VRAM does Llama 3.3 70B need?
About 43.2 GB at Q4_K_M with an 8K context, 73.2 GB at Q8_0 and 135 GB at FP16. Longer contexts add 0.31 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Llama 3.3 70B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Apple M5 Pro 64GB (48.0 GB usable). With 24 GB cards you need about 2 of them.