RTX 3090 24GB vs Tesla P40 24GB for Local LLMs
Both have 24.0 GB, so they run the same models. The RTX 3090 24GB generates tokens about 211% faster. Below: speeds on seven popular models, the models only one card can run, and our verdict.
Specs that matter for AI
| RTX 3090 24GB | Tesla P40 24GB | |
|---|---|---|
| Memory | 24 GB | 24 GB |
| GPU-usable memory | 24.0 GB | 24.0 GB |
| Bandwidth | 936 GB/s | 347 GB/s |
| Architecture | Ampere | Pascal |
| Released | 2020-09 | 2016-09 |
| Llama 3.1 8B Q4_K_M | ~109 tok/s | ~35 tok/s |
Speed and fit by model
Q4_K_M (or native format) with 8K context.
| Model | VRAM | RTX 3090 24GB | Tesla P40 24GB |
|---|---|---|---|
| Llama 3.1 8B | 6.3 GB | Fits ~109 tok/s | Fits ~35 tok/s |
| Qwen 3 14B | 10.5 GB | Fits ~62 tok/s | Fits ~20 tok/s |
| gpt-oss-20b | 13.9 GB | Fits ~155 tok/s | Fits ~65 tok/s |
| Qwen 3 30B-A3B | 18.8 GB | Fits ~114 tok/s | Fits ~57 tok/s |
| Qwen 3 32B | 21.4 GB | Fits ~28 tok/s | Fits ~9.1 tok/s |
| Llama 3.3 70B | 43.2 GB | 52% on GPU ~1.8 tok/s | 52% on GPU ~1.6 tok/s |
| gpt-oss-120b | 62.0 GB | 36% on GPU ~20 tok/s | 36% on GPU ~18 tok/s |
Verdict
Same 24 GB, very different cards. The 3090 has almost three times the bandwidth (936 vs 347 GB/s) and modern tensor cores, so it is about three times faster at generating text and far faster at prompt processing and image generation. The P40 is only worth it as a very cheap way to add 24 GB to a multi-GPU llama.cpp rig, and it needs its own cooling.
Which should you buy?
Choose the RTX 3090 24GB if…
- You want usable speed and modern tool support.
- You generate images or fine-tune.
- You want a card that cools itself.
Choose the Tesla P40 24GB if…
- You only need cheap extra memory for llama.cpp.
- You can add server-style cooling.
- Speed is not a priority.
Best quality and longest context on each card
The highest-quality quantization that fits fully at 8K context, and the longest context that fits at Q4_K_M with an FP16 cache.
| Model | RTX 3090 24GB: best quant | Tesla P40 24GB: best quant | RTX 3090 24GB: max context | Tesla P40 24GB: max context |
|---|---|---|---|---|
| Qwen 3.5 9B | FP16 / BF16 | FP16 / BF16 | 256K | 256K |
| Qwen 3 14B | Q8_0 | Q8_0 | 40K | 40K |
| gpt-oss-20b | MXFP4 (official) | MXFP4 (official) | 128K | 128K |
| Qwen 3.6 27B | Q6_K | Q6_K | 98K | 98K |
| Qwen 3 32B | Q4_K_M | Q4_K_M | 12K | 12K |
| Llama 3.3 70B | Does not fit | Does not fit | – | – |
Image generation
Best weight precision for image models with model swapping, at each model's default resolution. Image generation is also limited by compute, so the card with more tensor-core throughput finishes each image faster even when both fit.
| Model | RTX 3090 24GB | Tesla P40 24GB |
|---|---|---|
| SDXL 1.0 | FP16 / BF16 | FP16 / BF16 |
| FLUX.1 [dev] | FP16 / BF16 | FP16 / BF16 |
| Qwen-Image | FP8 | FP8 |
| Wan 2.2 A14B (T2V / I2V) | FP8 | FP8 |
Frequently asked questions
Is the RTX 3090 24GB or Tesla P40 24GB better for local LLMs?
Same 24 GB, very different cards. The 3090 has almost three times the bandwidth (936 vs 347 GB/s) and modern tensor cores, so it is about three times faster at generating text and far faster at prompt processing and image generation. The P40 is only worth it as a very cheap way to add 24 GB to a multi-GPU llama.cpp rig, and it needs its own cooling.
Which is faster for LLM inference?
The RTX 3090 24GB, by about 211% for token generation on models that fit both (about 109 vs 35 tokens/s on Llama 3.1 8B Q4_K_M).
Which can run bigger models?
Neither: both have 24.0 GB usable, so they run the same models.