Gemma 4 26B-A4B VRAM Requirements
Gemma 4 26B-A4B needs about 15.6 GB of VRAM at Q4_K_M with an 8K context: 14.2 GB for weights, 0.4 GB for the KV cache and 1.0 GB of runtime overhead. At Q8_0 it needs 26.3 GB and at full FP16 precision 48.3 GB. The smallest GPU that runs it fully is the Apple M4 Pro 24GB.
15.6 GB VRAM
Gemma 4 26B-A4B at Q4_K_M with 8K context and the KV cache in FP16.
- Weights 14.2 GB
- KV cache 0.4 GB
- Runtime overhead 1.0 GB
Mixture of experts: all 25.2B parameters need memory, but only 3.8B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GB72% on GPU~35 tok/s
- RTX 4060 Ti 16GBTight fit~59 tok/s
- RTX 3090 24GBFits~138 tok/s
- Radeon RX 7900 XTX 24GBFits~126 tok/s
- RTX 4090 24GBFits~144 tok/s
- RTX 5090 32GBFits~191 tok/s
- RTX 6000 Ada 48GBFits~140 tok/s
- A100 80GBFits~202 tok/s
- H100 80GB SXMFits~239 tok/s
- Ryzen AI Max+ 395 128GBFits~50 tok/s
- Apple M4 Max 128GBFits~97 tok/s
- Apple M5 Max 128GBFits~105 tok/s
- RTX PRO 6000 Blackwell 96GBFits~191 tok/s
- NVIDIA DGX Spark 128GBFits~53 tok/s
- Apple M3 Ultra 512GBFits~127 tok/s
Gemma 4 26B-A4B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| Q4_K_M | 4.85 | 14.2 GB | 15.6 GB | 16.1 GB | 20 GB |
| Q5_K_M | 5.69 | 16.7 GB | 18.1 GB | 18.6 GB | 20 GB |
| Q6_K | 6.59 | 19.3 GB | 20.7 GB | 21.2 GB | 24 GB |
| Q8_0 | 8.50 | 24.9 GB | 26.3 GB | 26.8 GB | 32 GB |
| FP16 / BF16 | 16.00 | 46.9 GB | 48.3 GB | 48.8 GB | 64 GB |
| Q3_K_M | 3.91 | 11.5 GB | 12.9 GB | 13.3 GB | 16 GB |
| Q2_K | 3.00 | 8.8 GB | 10.2 GB | 10.7 GB | 12 GB |
VRAM by context length
At Q4_K_M. 25 of 30 layers use a 1,024-token sliding window. The 5 global layers keep the full context with only 2 KV heads of 512 dimensions, so each extra 1,000 tokens costs only about 20 MB in FP16. Figures assume separate keys and values; runtimes that store Gemma 4's unified key-value once need even less.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.3 GB | 0.1 GB | 15.5 GB | 15.4 GB |
| 8K tokens | 0.4 GB | 0.2 GB | 15.6 GB | 15.5 GB |
| 32K tokens | 0.8 GB | 0.4 GB | 16.1 GB | 15.7 GB |
| 128K tokens | 2.7 GB | 1.4 GB | 18.0 GB | 16.7 GB |
| 256K tokens | 5.2 GB | 2.8 GB | 20.5 GB | 18.0 GB |
GPUs that can run Gemma 4 26B-A4B
At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M4 Pro 24GB | 16.0 GB | Tight fit | IQ4_XS | – | ~57 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~59 tok/s |
| Radeon RX 9060 XT 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~56 tok/s |
| Tesla T4 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~65 tok/s |
| RTX 5060 Ti 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~84 tok/s |
| Radeon RX 6800 XT 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~82 tok/s |
| Intel Arc A770 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~81 tok/s |
| Radeon RX 7800 XT 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~95 tok/s |
| Radeon RX 9070 XT 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~96 tok/s |
| RTX 4070 Ti Super 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~112 tok/s |
| RTX 4080 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~117 tok/s |
| RTX 4080 Super 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~119 tok/s |
| RTX 5070 Ti 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~134 tok/s |
| RTX 5080 16GB | 16.0 GB | Tight fit | IQ4_XS | – | ~140 tok/s |
| RTX 2060 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~35 tok/s |
| RTX 3060 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~35 tok/s |
| RTX 3080 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~43 tok/s |
| RTX 4070 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~39 tok/s |
| RTX 4070 Super 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~39 tok/s |
| RTX 4070 Ti 12GB | 12.0 GB | 72% on GPU | Q2_K | – | ~39 tok/s |
About Gemma 4 26B-A4B
Google's first Gemma mixture-of-experts: 128 experts plus a shared expert, 8 routed per token, about 3.8B active. It runs at small-model speed and just squeezes onto a 16 GB card at Q4_K_M; IQ4_XS leaves more room. Long contexts are cheap thanks to sliding-window layers and very lean global layers.
| Spec | Value |
|---|---|
| Parameters | 25.2B total, 3.8B active (MoE) |
| Layers | 30 |
| Hidden size | 2816 |
| Attention heads / KV heads | 16 / 8 (varies by layer) |
| Head dimension | 256 |
| Vocabulary | 262,144 |
| Max context (used here) | 256K tokens |
| Layers with a growing cache | 5 of 30 |
| KV cache per token (FP16) | 20 KB |
| Released | 2026-04 |
| License | Apache 2.0 |
Gemma 4 26B-A4B VRAM: common questions
Can Gemma 4 26B-A4B run on a 16GB GPU?
Just. At Q4_K_M it needs about 15.6 GB with an 8K context, which is tight on 16 GB; IQ4_XS is more comfortable. The small cache means a 32K context adds only about 0.5 GB.
How much VRAM does Gemma 4 26B-A4B need?
About 15.6 GB at Q4_K_M with an 8K context, 26.3 GB at Q8_0 and 48.3 GB at FP16. Longer contexts add 0.02 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Gemma 4 26B-A4B?
The smallest listed device that fits it fully at Q4_K_M with 8K context is the Apple M4 Pro 24GB (16.0 GB usable).
Does Gemma 4 26B-A4B need VRAM for all 25.2B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 3.8B parameters are read per token, so it generates text about as fast as a 4B dense model.