What LLMs Can Your GPU Run?
Pick your card to see every model it runs. Memory decides which models fit; bandwidth decides how fast they run. Not sure? Use Can I Run It.
Consumer GPUs
Workstation and datacenter
| GPU | Memory | Bandwidth | 8B speed | Largest model (Q4_K_M) |
|---|---|---|---|---|
| Tesla T4 16GB | 16.0 GB | 320 GB/s | ~38 tok/s | Gemma 4 26B-A4B |
| NVIDIA L4 24GB | 24.0 GB | 300 GB/s | ~36 tok/s | Qwen 3.5 35B-A3B |
| Tesla P40 24GB | 24.0 GB | 347 GB/s | ~35 tok/s | Qwen 3.5 35B-A3B |
| Intel Arc Pro B60 24GB | 24.0 GB | 456 GB/s | ~42 tok/s | Qwen 3.5 35B-A3B |
| Radeon AI PRO R9700 32GB | 32.0 GB | 640 GB/s | ~64 tok/s | Mixtral 8x7B |
| A100 40GB | 40.0 GB | 1555 GB/s | ~176 tok/s | Mixtral 8x7B |
| RTX A6000 48GB | 48.0 GB | 768 GB/s | ~90 tok/s | Qwen3-Next 80B-A3B |
| L40S 48GB | 48.0 GB | 864 GB/s | ~101 tok/s | Qwen3-Next 80B-A3B |
| RTX 6000 Ada 48GB | 48.0 GB | 960 GB/s | ~111 tok/s | Qwen3-Next 80B-A3B |
| A100 80GB | 80.0 GB | 2039 GB/s | ~227 tok/s | Qwen 3.5 122B-A10B |
| H100 80GB SXM | 80.0 GB | 3350 GB/s | ~357 tok/s | Qwen 3.5 122B-A10B |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | 1792 GB/s | ~202 tok/s | Qwen 3.5 122B-A10B |
| NVIDIA H200 141GB | 141 GB | 4800 GB/s | ~487 tok/s | Qwen 3 235B-A22B |
| AMD Instinct MI300X 192GB | 192 GB | 5300 GB/s | ~460 tok/s | DeepSeek V4 Flash |