gpt-oss-120b VRAM Requirements
gpt-oss-120b needs about 62.0 GB of VRAM at MXFP4 (official) with an 8K context: 60.8 GB for weights, 0.3 GB for the KV cache and 0.9 GB of runtime overhead. At Q4_K_M it needs 67.2 GB. At Q8_0 it needs 117 GB and at full FP16 precision 219 GB. The smallest GPU that runs it fully is the A100 80GB.
62.0 GB VRAM
gpt-oss-120b at MXFP4 (official) with 8K context and the KV cache in FP16.
- Weights 60.8 GB
- KV cache 0.3 GB
- Runtime overhead 0.9 GB
Mixture of experts: all 116.8B parameters need memory, but only 5.1B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GBToo small
- RTX 3090 24GB36% on GPU~20 tok/s
- Radeon RX 7900 XTX 24GB36% on GPU~20 tok/s
- RTX 4090 24GB36% on GPU~20 tok/s
- RTX 5090 32GB49% on GPU~24 tok/s
- RTX 6000 Ada 48GB73% on GPU~38 tok/s
- A100 80GBFits~170 tok/s
- H100 80GB SXMFits~201 tok/s
- Ryzen AI Max+ 395 128GBFits~43 tok/s
- Apple M4 Max 128GBFits~83 tok/s
- Apple M5 Max 128GBFits~90 tok/s
- RTX PRO 6000 Blackwell 96GBFits~162 tok/s
- NVIDIA DGX Spark 128GBFits~46 tok/s
- Apple M3 Ultra 512GBFits~108 tok/s
Multi-GPU: about 3× 24 GB cards (RTX 3090 / 4090) with the model split across them.
gpt-oss-120b VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| MXFP4 (official) | 4.47 | 60.8 GB | 62.0 GB | 62.9 GB | 80 GB |
| Q4_K_M | 4.85 | 65.9 GB | 67.2 GB | 68.0 GB | 80 GB |
| Q5_K_M | 5.69 | 77.4 GB | 78.6 GB | 79.4 GB | 96 GB |
| Q6_K | 6.59 | 89.6 GB | 90.8 GB | 91.7 GB | 128 GB |
| Q8_0 | 8.50 | 116 GB | 117 GB | 118 GB | 128 GB |
| FP16 / BF16 | 16.00 | 218 GB | 219 GB | 220 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 53.2 GB | 54.4 GB | 55.2 GB | 64 GB |
| Q2_K | 3.00 | 40.8 GB | 42.0 GB | 42.8 GB | 48 GB |
VRAM by context length
At MXFP4 (official). 18 of 36 layers keep the full context. The other 18 use a 128-token sliding window, so their cache stops growing after 128 tokens.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 0.1 GB | 79 MB | 61.9 GB | 61.8 GB |
| 8K tokens | 0.3 GB | 0.2 GB | 62.0 GB | 61.9 GB |
| 32K tokens | 1.1 GB | 0.6 GB | 62.9 GB | 62.3 GB |
| 128K tokens | 4.5 GB | 2.4 GB | 66.2 GB | 64.1 GB |
GPUs that can run gpt-oss-120b
At MXFP4 (official) with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at MXFP4 (official).
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| A100 80GB | 80.0 GB | Fits | MXFP4 (official) | 128K | ~170 tok/s |
| H100 80GB SXM | 80.0 GB | Fits | MXFP4 (official) | 128K | ~201 tok/s |
| Ryzen AI Max+ 395 128GB | 96.0 GB | Fits | MXFP4 (official) | 128K | ~43 tok/s |
| Apple M3 Max 128GB | 96.0 GB | Fits | MXFP4 (official) | 128K | ~66 tok/s |
| Apple M4 Max 128GB | 96.0 GB | Fits | MXFP4 (official) | 128K | ~83 tok/s |
| Apple M5 Max 128GB | 96.0 GB | Fits | MXFP4 (official) | 128K | ~90 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | Fits | MXFP4 (official) | 128K | ~162 tok/s |
| NVIDIA DGX Spark 128GB | 112 GB | Fits | MXFP4 (official) | 128K | ~46 tok/s |
| NVIDIA H200 141GB | 141 GB | Fits | MXFP4 (official) | 128K | ~219 tok/s |
| Apple M2 Ultra 192GB | 144 GB | Fits | MXFP4 (official) | 128K | ~107 tok/s |
| AMD Instinct MI300X 192GB | 192 GB | Fits | MXFP4 (official) | 128K | ~216 tok/s |
| Apple M3 Ultra 512GB | 384 GB | Fits | MXFP4 (official) | 128K | ~108 tok/s |
| RTX 6000 Ada 48GB | 48.0 GB | 73% on GPU | Q2_K | – | ~38 tok/s |
| RTX A6000 48GB | 48.0 GB | 73% on GPU | Q2_K | – | ~36 tok/s |
| L40S 48GB | 48.0 GB | 73% on GPU | Q2_K | – | ~37 tok/s |
| A100 40GB | 40.0 GB | 61% on GPU | – | – | ~30 tok/s |
| RTX 5090 32GB | 32.0 GB | 49% on GPU | – | – | ~24 tok/s |
| Radeon AI PRO R9700 32GB | 32.0 GB | 49% on GPU | – | – | ~22 tok/s |
Multi-GPU: about 3× 24 GB cards (RTX 3090 or 4090) run it at MXFP4 (official), using tensor or layer splitting in llama.cpp, vLLM or ExLlamaV2.
About gpt-oss-120b
OpenAI's larger open-weight model: 117B total and 5.1B active parameters, shipped natively in MXFP4 at about 61 GiB. It fits on a single 80 GB H100 or a 96 GB RTX PRO 6000, and it is fast on 128 GB unified-memory systems because so little is read per token.
| Spec | Value |
|---|---|
| Parameters | 116.8B total, 5.1B active (MoE) |
| Layers | 36 |
| Hidden size | 2880 |
| Attention heads / KV heads | 64 / 8 |
| Head dimension | 64 |
| Vocabulary | 201,088 |
| Max context (used here) | 128K tokens |
| Sliding window | 128 tokens on 18 of 36 layers |
| KV cache per token (FP16) | 36 KB |
| Released | 2025-08 |
| License | Apache 2.0 |
gpt-oss-120b VRAM: common questions
Can gpt-oss-120b run on a 128GB Mac or Ryzen AI Max?
Yes. About 61 GiB of weights plus cache fits in the GPU-usable portion of 128 GB, and only 5.1B active parameters keep it fast.
Can gpt-oss-120b run on a 24GB GPU?
Only with expert offload to system RAM (64 GB+). Attention layers stay on the GPU and the experts run on the CPU, which still gives usable speed.
How much VRAM does gpt-oss-120b need?
About 62.0 GB at MXFP4 (official) with an 8K context, 117 GB at Q8_0 and 219 GB at FP16. Longer contexts add 0.04 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run gpt-oss-120b?
The smallest listed device that fits it fully at MXFP4 (official) with 8K context is the A100 80GB (80.0 GB usable). With 24 GB cards you need about 3 of them.
Does gpt-oss-120b need VRAM for all 116.8B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 5.1B parameters are read per token, so it generates text about as fast as a 5B dense model.