Gemma 3 12B VRAM Requirements

Gemma 3 12B needs about 8.8 GB of VRAM at Q4_K_M with an 8K context: 6.9 GB for weights, 0.8 GB for the KV cache and 1.1 GB of runtime overhead. At Q8_0 it needs 13.9 GB and at full FP16 precision 24.6 GB. The smallest GPU that runs it fully is the Intel Arc B570 10GB.

By Google · Updated October 14, 2026 · Values from config.json on Hugging Face

8.8 GB VRAM

Gemma 3 12B at Q4_K_M with 8K context and the KV cache in FP16.

  • Weights 6.9 GB
  • KV cache 0.8 GB
  • Runtime overhead 1.1 GB

Which GPUs can run it

Gemma 3 12B VRAM by quantization

Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.

QuantizationBits/weightWeightsTotal at 8KTotal at 32KSmallest tier
Q4_K_M4.856.9 GB8.8 GB10.3 GB10 GB
Q5_K_M5.698.1 GB10.0 GB11.5 GB12 GB
Q6_K6.599.4 GB11.2 GB12.7 GB12 GB
Q8_08.5012.1 GB13.9 GB15.4 GB16 GB
FP16 / BF1616.0022.7 GB24.6 GB26.1 GB32 GB
Q3_K_M3.915.6 GB7.4 GB8.9 GB8 GB
Q2_K3.004.3 GB6.1 GB7.6 GB8 GB

VRAM by context length

At Q4_K_M. 8 of 48 layers keep the full context. The other 40 use a 1024-token sliding window, so their cache stops growing after 1K tokens.

ContextKV cache (FP16)KV cache (Q8_0)Total, FP16 cacheTotal, Q8_0 cache
4K tokens0.6 GB0.3 GB8.5 GB8.2 GB
8K tokens0.8 GB0.4 GB8.8 GB8.4 GB
32K tokens2.3 GB1.2 GB10.3 GB9.2 GB
128K tokens8.3 GB4.4 GB16.3 GB12.4 GB

GPUs that can run Gemma 3 12B

At Q4_K_M with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at Q4_K_M.

GPUUsable memoryStatusBest quantMax contextSpeed
Intel Arc B570 10GB10.0 GBFitsQ4_K_M18K~23 tok/s
RTX 3080 10GB10.0 GBFitsQ4_K_M18K~60 tok/s
Apple M4 16GB10.7 GBFitsQ5_K_M28K~9.7 tok/s
RTX 2080 Ti 11GB11.0 GBFitsQ5_K_M33K~49 tok/s
RTX 2060 12GB12.0 GBFitsQ6_K48K~27 tok/s
RTX 3060 12GB12.0 GBFitsQ6_K48K~29 tok/s
Radeon RX 6700 XT 12GB12.0 GBFitsQ6_K48K~26 tok/s
Radeon RX 7700 XT 12GB12.0 GBFitsQ6_K48K~29 tok/s
Intel Arc B580 12GB12.0 GBFitsQ6_K48K~28 tok/s
RTX 4070 12GB12.0 GBFitsQ6_K48K~40 tok/s
RTX 4070 Super 12GB12.0 GBFitsQ6_K48K~40 tok/s
RTX 4070 Ti 12GB12.0 GBFitsQ6_K48K~40 tok/s
RTX 5070 12GB12.0 GBFitsQ6_K48K~53 tok/s
RTX 3080 12GB12.0 GBFitsQ6_K48K~72 tok/s
RTX 3050 8GB8.0 GB86% on GPUQ3_K_M–~13 tok/s
RTX 3060 Ti 8GB8.0 GB86% on GPUQ3_K_M–~20 tok/s
RTX 3070 8GB8.0 GB86% on GPUQ3_K_M–~20 tok/s
RTX 3070 Ti 8GB8.0 GB86% on GPUQ3_K_M–~22 tok/s
RTX 4060 8GB8.0 GB86% on GPUQ3_K_M–~15 tok/s
RTX 4060 Ti 8GB8.0 GB86% on GPUQ3_K_M–~16 tok/s

About Gemma 3 12B

A strong multimodal mid-size model. Its 256-dimension heads make each cached token heavy on the 8 full-attention layers, but sliding windows on the other 40 layers keep long contexts affordable. Google also published official QAT (quantization-aware trained) Q4_0 weights.

SpecValue
Parameters12.2B
Layers48
Hidden size3840
Attention heads / KV heads16 / 8
Head dimension256
Vocabulary262,208
Max context (used here)128K tokens
Sliding window1024 tokens on 40 of 48 layers
KV cache per token (FP16)64 KB
Released2025-03
LicenseGemma Terms of Use

Gemma 3 12B VRAM: common questions

Can Gemma 3 12B run on 8GB VRAM?

At Q4_K_M the weights are about 6.9 GB, so 8 GB cards fit it only with a short context. 12 GB is the comfortable minimum.

How much VRAM does Gemma 3 12B need?

About 8.8 GB at Q4_K_M with an 8K context, 13.9 GB at Q8_0 and 24.6 GB at FP16. Longer contexts add 0.06 GB per 1,000 tokens with an FP16 cache.

What is the cheapest GPU that can run Gemma 3 12B?

The smallest listed device that fits it fully at Q4_K_M with 8K context is the Intel Arc B570 10GB (10.0 GB usable).

Related