gpt-oss-20b VRAM Requirements
gpt-oss-20b needs about 13.9 GB of VRAM at MXFP4 (official) with an 8K context: 12.8 GB for weights, 0.2 GB for the KV cache and 0.9 GB of runtime overhead. At Q4_K_M it needs 12.9 GB. At Q8_0 it needs 21.8 GB and at full FP16 precision 40.0 GB. The smallest GPU that runs it fully is the Apple M4 Pro 24GB.
13.9 GB VRAM
gpt-oss-20b at MXFP4 (official) with 8K context and the KV cache in FP16.
- Weights 12.8 GB
- KV cache 0.2 GB
- Runtime overhead 0.9 GB
Mixture of experts: all 20.9B parameters need memory, but only 3.6B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GB81% on GPU~45 tok/s
- RTX 4060 Ti 16GBFits~64 tok/s
- RTX 3090 24GBFits~155 tok/s
- Radeon RX 7900 XTX 24GBFits~141 tok/s
- RTX 4090 24GBFits~162 tok/s
- RTX 5090 32GBFits~221 tok/s
- RTX 6000 Ada 48GBFits~157 tok/s
- A100 80GBFits~234 tok/s
- H100 80GB SXMFits~283 tok/s
- Ryzen AI Max+ 395 128GBFits~54 tok/s
- Apple M4 Max 128GBFits~107 tok/s
- Apple M5 Max 128GBFits~116 tok/s
- RTX PRO 6000 Blackwell 96GBFits~221 tok/s
- NVIDIA DGX Spark 128GBFits~57 tok/s
- Apple M3 Ultra 512GBFits~142 tok/s
gpt-oss-20b VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| MXFP4 (official) | 5.26 | 12.8 GB | 13.9 GB | 14.5 GB | 16 GB |
| Q4_K_M | 4.85 | 11.8 GB | 12.9 GB | 13.5 GB | 16 GB |
| Q5_K_M | 5.69 | 13.8 GB | 15.0 GB | 15.5 GB | 16 GB |
| Q6_K | 6.59 | 16.0 GB | 17.2 GB | 17.7 GB | 20 GB |
| Q8_0 | 8.50 | 20.7 GB | 21.8 GB | 22.4 GB | 24 GB |
| FP16 / BF16 | 16.00 | 38.9 GB | 40.0 GB | 40.6 GB | 48 GB |
| Q3_K_M | 3.91 | 9.5 GB | 10.6 GB | 11.2 GB | 12 GB |
| Q2_K | 3.00 | 7.3 GB | 8.4 GB | 9.0 GB | 10 GB |
VRAM by context length
At MXFP4 (official). 12 of 24 layers keep the full context. The other 12 use a 128-token sliding window, so their cache stops growing after 128 tokens.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.1 GB | 53 MB | 13.8 GB | 13.8 GB |
| 8K tokens | 0.2 GB | 0.1 GB | 13.9 GB | 13.8 GB |
| 32K tokens | 0.8 GB | 0.4 GB | 14.5 GB | 14.1 GB |
| 128K tokens | 3.0 GB | 1.6 GB | 16.7 GB | 15.3 GB |
GPUs that can run gpt-oss-20b
At MXFP4 (official) with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at MXFP4 (official).
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| Apple M4 Pro 24GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~61 tok/s |
| RTX 4060 Ti 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~64 tok/s |
| Radeon RX 9060 XT 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~61 tok/s |
| Tesla T4 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~70 tok/s |
| RTX 5060 Ti 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~92 tok/s |
| Radeon RX 6800 XT 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~89 tok/s |
| Intel Arc A770 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~89 tok/s |
| Radeon RX 7800 XT 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~104 tok/s |
| Radeon RX 9070 XT 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~106 tok/s |
| RTX 4070 Ti Super 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~124 tok/s |
| RTX 4080 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~130 tok/s |
| RTX 4080 Super 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~132 tok/s |
| RTX 5070 Ti 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~151 tok/s |
| RTX 5080 16GB | 16.0 GB | Fits | MXFP4 (official) | 55K | ~157 tok/s |
| RTX 2060 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~44 tok/s |
| RTX 3060 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~45 tok/s |
| RTX 3080 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~59 tok/s |
| RTX 4070 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~51 tok/s |
| RTX 4070 Super 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~51 tok/s |
| RTX 4070 Ti 12GB | 12.0 GB | 81% on GPU | Q3_K_M | – | ~51 tok/s |
About gpt-oss-20b
OpenAI's smaller open-weight model, a mixture-of-experts with 3.6B active parameters. It ships natively in MXFP4, so the official checkpoint is already about 13 GB and re-quantizing to GGUF saves little. Half the layers use a 128-token sliding window, which keeps long contexts cheap.
| Spec | Value |
|---|---|
| Parameters | 20.9B total, 3.6B active (MoE) |
| Layers | 24 |
| Hidden size | 2880 |
| Attention heads / KV heads | 64 / 8 |
| Head dimension | 64 |
| Vocabulary | 201,088 |
| Max context (used here) | 128K tokens |
| Sliding window | 128 tokens on 12 of 24 layers |
| KV cache per token (FP16) | 24 KB |
| Released | 2025-08 |
| License | Apache 2.0 |
gpt-oss-20b VRAM: common questions
Can gpt-oss-20b run on a 16GB GPU?
Yes. The native MXFP4 weights are about 12.8 GiB, leaving room for a long context on 16 GB cards. OpenAI designed it for 16 GB devices.
Should I use a Q4_K_M GGUF of gpt-oss-20b?
There is little benefit. The experts are already 4-bit, so most GGUF versions keep MXFP4 for them and end up close to the native size.
How much VRAM does gpt-oss-20b need?
About 13.9 GB at MXFP4 (official) with an 8K context, 21.8 GB at Q8_0 and 40.0 GB at FP16. Longer contexts add 0.02 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run gpt-oss-20b?
The smallest listed device that fits it fully at MXFP4 (official) with 8K context is the Apple M4 Pro 24GB (16.0 GB usable).
Does gpt-oss-20b need VRAM for all 20.9B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 3.6B parameters are read per token, so it generates text about as fast as a 4B dense model.