RTX 3090 24GB vs RTX 5070 Ti 16GB for Local LLMs
The RTX 3090 24GB holds bigger models (24.0 GB vs 16.0 GB), and the RTX 3090 24GB generates tokens about 4% faster. Below: speeds on seven popular models, the models only one card can run, and our verdict.
Specs that matter for AI
| RTX 3090 24GB | RTX 5070 Ti 16GB | |
|---|---|---|
| Memory | 24 GB | 16 GB |
| GPU-usable memory | 24.0 GB | 16.0 GB |
| Bandwidth | 936 GB/s | 896 GB/s |
| Architecture | Ampere | Blackwell |
| Released | 2020-09 | 2025-02 |
| Llama 3.1 8B Q4_K_M | ~109 tok/s | ~104 tok/s |
Speed and fit by model
Q4_K_M (or native format) with 8K context.
| Model | VRAM | RTX 3090 24GB | RTX 5070 Ti 16GB |
|---|---|---|---|
| Llama 3.1 8B | 6.3 GB | Fits ~109 tok/s | Fits ~104 tok/s |
| Qwen 3 14B | 10.5 GB | Fits ~62 tok/s | Fits ~59 tok/s |
| gpt-oss-20b | 13.9 GB | Fits ~155 tok/s | Fits ~151 tok/s |
| Qwen 3 30B-A3B | 18.8 GB | Fits ~114 tok/s | 80% on GPU ~51 tok/s |
| Qwen 3 32B | 21.4 GB | Fits ~28 tok/s | 70% on GPU ~5.7 tok/s |
| Llama 3.3 70B | 43.2 GB | 52% on GPU ~1.8 tok/s | 35% on GPU ~1.4 tok/s |
| gpt-oss-120b | 62.0 GB | 36% on GPU ~20 tok/s | Too small |
Verdict
The used 3090 has 8 GB more memory, which decides what you can run: 32B models at Q4_K_M fit on the 3090 but not the 5070 Ti. On models that fit both, speed is similar (936 vs 896 GB/s). The 5070 Ti is newer, uses less power and supports FP4, but for local LLMs capacity usually matters more.
Which should you buy?
Choose the RTX 3090 24GB if…
- You want to run 27–35B models fully on the GPU.
- You plan a two-card setup for 70B models.
- Price per gigabyte matters most.
Choose the RTX 5070 Ti 16GB if…
- You want a new card with warranty and lower power draw.
- Your models are 20B or smaller.
- You also game and want current features.
Best quality and longest context on each card
The highest-quality quantization that fits fully at 8K context, and the longest context that fits at Q4_K_M with an FP16 cache.
| Model | RTX 3090 24GB: best quant | RTX 5070 Ti 16GB: best quant | RTX 3090 24GB: max context | RTX 5070 Ti 16GB: max context |
|---|---|---|---|---|
| Qwen 3.5 9B | FP16 / BF16 | Q8_0 | 256K | 256K |
| Qwen 3 14B | Q8_0 | Q6_K | 40K | 37K |
| gpt-oss-20b | MXFP4 (official) | MXFP4 (official) | 128K | 55K |
| Qwen 3.6 27B | Q6_K | IQ4_XS | 98K | – |
| Qwen 3 32B | Q4_K_M | Q2_K | 12K | – |
| Llama 3.3 70B | Does not fit | Does not fit | – | – |
Image generation
Best weight precision for image models with model swapping, at each model's default resolution. Image generation is also limited by compute, so the card with more tensor-core throughput finishes each image faster even when both fit.
| Model | RTX 3090 24GB | RTX 5070 Ti 16GB |
|---|---|---|
| SDXL 1.0 | FP16 / BF16 | FP16 / BF16 |
| FLUX.1 [dev] | FP16 / BF16 | FP8 |
| Qwen-Image | FP8 | GGUF Q5_K_S |
| Wan 2.2 A14B (T2V / I2V) | FP8 | GGUF Q5_K_S |
Models only one card can run
Only RTX 3090 24GB
Only RTX 5070 Ti 16GB
None
Frequently asked questions
Is the RTX 3090 24GB or RTX 5070 Ti 16GB better for local LLMs?
The used 3090 has 8 GB more memory, which decides what you can run: 32B models at Q4_K_M fit on the 3090 but not the 5070 Ti. On models that fit both, speed is similar (936 vs 896 GB/s). The 5070 Ti is newer, uses less power and supports FP4, but for local LLMs capacity usually matters more.
Which is faster for LLM inference?
The RTX 3090 24GB, by about 4% for token generation on models that fit both (about 109 vs 104 tokens/s on Llama 3.1 8B Q4_K_M).
Which can run bigger models?
The RTX 3090 24GB, with 24.0 GB usable versus 16.0 GB.