RTX 4090 24GB vs RTX 5090 32GB for Local LLMs
The RTX 5090 32GB holds bigger models (32.0 GB vs 24.0 GB), and the RTX 5090 32GB generates tokens about 73% faster. Below: speeds on seven popular models, the models only one card can run, and our verdict.
Specs that matter for AI
| RTX 4090 24GB | RTX 5090 32GB | |
|---|---|---|
| Memory | 24 GB | 32 GB |
| GPU-usable memory | 24.0 GB | 32.0 GB |
| Bandwidth | 1008 GB/s | 1792 GB/s |
| Architecture | Ada Lovelace | Blackwell |
| Released | 2022-10 | 2025-01 |
| Llama 3.1 8B Q4_K_M | ~117 tok/s | ~202 tok/s |
Speed and fit by model
Q4_K_M (or native format) with 8K context.
| Model | VRAM | RTX 4090 24GB | RTX 5090 32GB |
|---|---|---|---|
| Llama 3.1 8B | 6.3 GB | Fits ~117 tok/s | Fits ~202 tok/s |
| Qwen 3 14B | 10.5 GB | Fits ~66 tok/s | Fits ~115 tok/s |
| gpt-oss-20b | 13.9 GB | Fits ~162 tok/s | Fits ~221 tok/s |
| Qwen 3 30B-A3B | 18.8 GB | Fits ~118 tok/s | Fits ~146 tok/s |
| Qwen 3 32B | 21.4 GB | Fits ~31 tok/s | Fits ~54 tok/s |
| Llama 3.3 70B | 43.2 GB | 52% on GPU ~1.9 tok/s | 70% on GPU ~2.9 tok/s |
| gpt-oss-120b | 62.0 GB | 36% on GPU ~20 tok/s | 49% on GPU ~24 tok/s |
Verdict
The 5090 adds 8 GB and nearly doubles bandwidth (1792 vs 1008 GB/s), so it generates tokens about 75% faster on the same model. The extra memory matters most for 32B models: on the 5090 they fit at Q6_K or with 32K+ context. Neither card runs 70B models at Q4_K_M on its own.
Which should you buy?
Choose the RTX 4090 24GB if…
- Your models fit comfortably in 24 GB.
- You can find one at a good used price.
- Your case or power supply cannot handle a 575 W card.
Choose the RTX 5090 32GB if…
- You run 27–32B models and want Q6_K or long contexts.
- You want the fastest single consumer card for generation.
- You use FP4 or NVFP4 models in vLLM or TensorRT-LLM.
Best quality and longest context on each card
The highest-quality quantization that fits fully at 8K context, and the longest context that fits at Q4_K_M with an FP16 cache.
| Model | RTX 4090 24GB: best quant | RTX 5090 32GB: best quant | RTX 4090 24GB: max context | RTX 5090 32GB: max context |
|---|---|---|---|---|
| Qwen 3.5 9B | FP16 / BF16 | FP16 / BF16 | 256K | 256K |
| Qwen 3 14B | Q8_0 | FP16 / BF16 | 40K | 40K |
| gpt-oss-20b | MXFP4 (official) | MXFP4 (official) | 128K | 128K |
| Qwen 3.6 27B | Q6_K | Q8_0 | 98K | 219K |
| Qwen 3 32B | Q4_K_M | Q6_K | 12K | 40K |
| Llama 3.3 70B | Does not fit | Q2_K | – | – |
Image generation
Best weight precision for image models with model swapping, at each model's default resolution. Image generation is also limited by compute, so the card with more tensor-core throughput finishes each image faster even when both fit.
| Model | RTX 4090 24GB | RTX 5090 32GB |
|---|---|---|
| SDXL 1.0 | FP16 / BF16 | FP16 / BF16 |
| FLUX.1 [dev] | FP16 / BF16 | FP16 / BF16 |
| Qwen-Image | FP8 | FP8 |
| Wan 2.2 A14B (T2V / I2V) | FP8 | FP16 / BF16 |
Models only one card can run
Only RTX 4090 24GB
None
Only RTX 5090 32GB
Frequently asked questions
Is the RTX 4090 24GB or RTX 5090 32GB better for local LLMs?
The 5090 adds 8 GB and nearly doubles bandwidth (1792 vs 1008 GB/s), so it generates tokens about 75% faster on the same model. The extra memory matters most for 32B models: on the 5090 they fit at Q6_K or with 32K+ context. Neither card runs 70B models at Q4_K_M on its own.
Which is faster for LLM inference?
The RTX 5090 32GB, by about 73% for token generation on models that fit both (about 202 vs 117 tokens/s on Llama 3.1 8B Q4_K_M).
Which can run bigger models?
The RTX 5090 32GB, with 32.0 GB usable versus 24.0 GB.