RTX 5080 16GB vs RTX 5090 32GB for Local LLMs
The RTX 5090 32GB holds bigger models (32.0 GB vs 16.0 GB), and the RTX 5090 32GB generates tokens about 81% faster. Below: speeds on seven popular models, the models only one card can run, and our verdict.
Specs that matter for AI
| RTX 5080 16GB | RTX 5090 32GB | |
|---|---|---|
| Memory | 16 GB | 32 GB |
| GPU-usable memory | 16.0 GB | 32.0 GB |
| Bandwidth | 960 GB/s | 1792 GB/s |
| Architecture | Blackwell | Blackwell |
| Released | 2025-01 | 2025-01 |
| Llama 3.1 8B Q4_K_M | ~111 tok/s | ~202 tok/s |
Speed and fit by model
Q4_K_M (or native format) with 8K context.
| Model | VRAM | RTX 5080 16GB | RTX 5090 32GB |
|---|---|---|---|
| Llama 3.1 8B | 6.3 GB | Fits ~111 tok/s | Fits ~202 tok/s |
| Qwen 3 14B | 10.5 GB | Fits ~63 tok/s | Fits ~115 tok/s |
| gpt-oss-20b | 13.9 GB | Fits ~157 tok/s | Fits ~221 tok/s |
| Qwen 3 30B-A3B | 18.8 GB | 80% on GPU ~52 tok/s | Fits ~146 tok/s |
| Qwen 3 32B | 21.4 GB | 70% on GPU ~5.8 tok/s | Fits ~54 tok/s |
| Llama 3.3 70B | 43.2 GB | 35% on GPU ~1.4 tok/s | 70% on GPU ~2.9 tok/s |
| gpt-oss-120b | 62.0 GB | Too small | 49% on GPU ~24 tok/s |
Verdict
The 5090 doubles the memory and nearly doubles the bandwidth (1,792 vs 960 GB/s). The 32 GB is the important part: it runs every 27–32B dense model and the 30–35B MoE models at Q4_K_M or better, while the 5080 tops out around 20–24B models. If local AI is the goal, the 5090 is worth far more than its price difference suggests.
Which should you buy?
Choose the RTX 5080 16GB if…
- Your models are 20B or smaller.
- Budget or power supply rules out a 5090.
- You mainly game and run AI occasionally.
Choose the RTX 5090 32GB if…
- You want to run 27–35B models at good quality.
- You want long contexts for coding agents.
- You want the fastest generation on one consumer card.
Best quality and longest context on each card
The highest-quality quantization that fits fully at 8K context, and the longest context that fits at Q4_K_M with an FP16 cache.
| Model | RTX 5080 16GB: best quant | RTX 5090 32GB: best quant | RTX 5080 16GB: max context | RTX 5090 32GB: max context |
|---|---|---|---|---|
| Qwen 3.5 9B | Q8_0 | FP16 / BF16 | 256K | 256K |
| Qwen 3 14B | Q6_K | FP16 / BF16 | 37K | 40K |
| gpt-oss-20b | MXFP4 (official) | MXFP4 (official) | 55K | 128K |
| Qwen 3.6 27B | IQ4_XS | Q8_0 | – | 219K |
| Qwen 3 32B | Q2_K | Q6_K | – | 40K |
| Llama 3.3 70B | Does not fit | Q2_K | – | – |
Image generation
Best weight precision for image models with model swapping, at each model's default resolution. Image generation is also limited by compute, so the card with more tensor-core throughput finishes each image faster even when both fit.
| Model | RTX 5080 16GB | RTX 5090 32GB |
|---|---|---|
| SDXL 1.0 | FP16 / BF16 | FP16 / BF16 |
| FLUX.1 [dev] | FP8 | FP16 / BF16 |
| Qwen-Image | GGUF Q5_K_S | FP8 |
| Wan 2.2 A14B (T2V / I2V) | GGUF Q5_K_S | FP16 / BF16 |
Models only one card can run
Only RTX 5080 16GB
None
Frequently asked questions
Is the RTX 5080 16GB or RTX 5090 32GB better for local LLMs?
The 5090 doubles the memory and nearly doubles the bandwidth (1,792 vs 960 GB/s). The 32 GB is the important part: it runs every 27–32B dense model and the 30–35B MoE models at Q4_K_M or better, while the 5080 tops out around 20–24B models. If local AI is the goal, the 5090 is worth far more than its price difference suggests.
Which is faster for LLM inference?
The RTX 5090 32GB, by about 81% for token generation on models that fit both (about 202 vs 111 tokens/s on Llama 3.1 8B Q4_K_M).
Which can run bigger models?
The RTX 5090 32GB, with 32.0 GB usable versus 16.0 GB.