Gemma 4 31B VRAM Requirements
Gemma 4 31B needs about 19.8 GB of VRAM at Q4_K_M with an 8K context: 17.3 GB for weights, 1.4 GB for the KV cache and 1.1 GB of runtime overhead. At Q8_0 it needs 32.9 GB and at full FP16 precision 59.7 GB. The smallest GPU that runs it fully is the Radeon RX 7900 XT 20GB.
19.8 GB VRAM
Gemma 4 31B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 17.3 GB
- KV cache 1.4 GB
- Runtime overhead 1.1 GB
Which GPUs can run it
- RTX 3060 12GB57% on GPU~4.0 tok/s
- RTX 4060 Ti 16GB76% on GPU~5.1 tok/s
- RTX 3090 24GBFits~30 tok/s
- Radeon RX 7900 XTX 24GBFits~26 tok/s
- RTX 4090 24GBFits~32 tok/s
- RTX 5090 32GBFits~57 tok/s
- RTX 6000 Ada 48GBFits~31 tok/s
- A100 80GBFits~64 tok/s
- H100 80GB SXMFits~103 tok/s
- Ryzen AI Max+ 395 128GBFits~7.7 tok/s
- Apple M4 Max 128GBFits~18 tok/s
- Apple M5 Max 128GBFits~20 tok/s
- RTX PRO 6000 Blackwell 96GBFits~57 tok/s
- NVIDIA DGX Spark 128GBFits~8.2 tok/s
- Apple M3 Ultra 512GBFits~26 tok/s
Gemma 4 31B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 17.3 GB | 19.8 GB | 21.7 GB | 24 GB |
| Q5_K_M | 5.69 | 20.3 GB | 22.8 GB | 24.7 GB | 32 GB |
| Q6_K | 6.59 | 23.6 GB | 26.0 GB | 27.9 GB | 32 GB |
| Q8_0 | 8.50 | 30.4 GB | 32.9 GB | 34.7 GB | 48 GB |
| FP16 / BF16 | 16.00 | 57.2 GB | 59.7 GB | 61.5 GB | 64 GB |
| Q3_K_M | 3.91 | 14.0 GB | 16.5 GB | 18.3 GB | 20 GB |
| Q2_K | 3.00 | 10.7 GB | 13.2 GB | 15.1 GB | 16 GB |
VRAM by context length
At Q4_K_M. 50 of 60 layers use a 1,024-token sliding window. The 10 global layers keep the full context with 4 KV heads of 512 dimensions. Figures assume separate keys and values; runtimes that store Gemma 4's unified key-value once need about half as much on the global layers.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 1.1 GB | 0.6 GB | 19.5 GB | 19.0 GB |
| 8K tokens | 1.4 GB | 0.7 GB | 19.8 GB | 19.2 GB |
| 32K tokens | 3.3 GB | 1.7 GB | 21.7 GB | 20.2 GB |
| 128K tokens | 10.8 GB | 5.7 GB | 29.2 GB | 24.1 GB |
| 256K tokens | 20.8 GB | 11.0 GB | 39.2 GB | 29.5 GB |
GPUs that can run Gemma 4 31B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Radeon RX 7900 XT 20GB | 20.0 GB | Tight fit | IQ4_XS | – | ~22 tok/s |
| Apple M5 32GB | 21.3 GB | Fits | Q4_K_M | 10K | ~5.0 tok/s |
| NVIDIA L4 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~9.8 tok/s |
| Tesla P40 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~9.6 tok/s |
| Apple M4 Max 36GB | 24.0 GB | Fits | Q4_K_M | 43K | ~13 tok/s |
| Intel Arc Pro B60 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~11 tok/s |
| RTX 3090 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~30 tok/s |
| Radeon RX 7900 XTX 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~26 tok/s |
| RTX 3090 Ti 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~32 tok/s |
| RTX 4090 24GB | 24.0 GB | Fits | Q4_K_M | 43K | ~32 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | Fits | Q6_K | 139K | ~18 tok/s |
| RTX 5090 32GB | 32.0 GB | Fits | Q6_K | 139K | ~57 tok/s |
| Apple M4 Pro 48GB | 36.0 GB | Fits | Q8_0 | 187K | ~8.9 tok/s |
| A100 40GB | 40.0 GB | Fits | Q8_0 | 235K | ~50 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~5.1 tok/s |
| RTX 4070 Ti Super 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~6.7 tok/s |
| RTX 4080 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~6.8 tok/s |
| RTX 4080 Super 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~6.9 tok/s |
| RTX 5060 Ti 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~6.0 tok/s |
| RTX 5070 Ti 16GB | 16.0 GB | 76% on GPU | Q2_K | – | ~7.1 tok/s |
About Gemma 4 31B
The largest dense Gemma 4. Its 256K context is affordable because five of every six layers only look at the last 1,024 tokens. At Q4_K_M it is a natural fit for 24 GB cards, and its vision encoder adds about 0.5 GB when loaded.
| Spec | Value |
|---|---|
| Parameters | 30.7B |
| Layers | 60 |
| Hidden size | 5376 |
| Attention heads / KV heads | 32 / 16 (varies by layer) |
| Head dimension | 256 |
| Vocabulary | 262,144 |
| Max context (used here) | 256K tokens |
| Layers with a growing cache | 10 of 60 |
| KV cache per token (FP16) | 80 KB |
| Released | 2026-04 |
| License | Apache 2.0 |
Gemma 4 31B VRAM: common questions
Does Gemma 4 31B fit on an RTX 3090 or 4090?
Yes at Q4_K_M with up to about 44K of context (about 90K with a Q8_0 cache). The weights are about 17 GB, and the cache grows slowly because only 10 of 60 layers keep the full context.
How much VRAM does Gemma 4 31B need?
About 19.8 GB at Q4_K_M with an 8K context, 32.9 GB at Q8_0 and 59.7 GB at FP16. Longer contexts add 0.08 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Gemma 4 31B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Radeon RX 7900 XT 20GB (20.0 GB usable).