Comparisons for Local LLMs
Side-by-side memory, bandwidth, speed and model fit for GPUs, plus quantization formats compared on real models. Everything is calculated from the same data as the rest of the site.
GPU vs GPU
RTX 3060 12GB vs RTX 4060 Ti 16GB12 GB vs 16 GB · 360 vs 288 GB/sRTX 3090 24GB vs RTX 4090 24GB24 GB vs 24 GB · 936 vs 1008 GB/sRTX 4090 24GB vs RTX 5090 32GB24 GB vs 32 GB · 1008 vs 1792 GB/sRTX 4060 Ti 16GB vs RTX 5060 Ti 16GB16 GB vs 16 GB · 288 vs 448 GB/sRTX 3090 24GB vs RTX 5070 Ti 16GB24 GB vs 16 GB · 936 vs 896 GB/sRTX 4090 24GB vs Radeon RX 7900 XTX 24GB24 GB vs 24 GB · 1008 vs 960 GB/sRTX 5070 Ti 16GB vs Radeon RX 9070 XT 16GB16 GB vs 16 GB · 896 vs 640 GB/sApple M4 Max 128GB vs RTX 5090 32GB128 GB vs 32 GB · 546 vs 1792 GB/sA100 80GB vs H100 80GB SXM80 GB vs 80 GB · 2039 vs 3350 GB/sNVIDIA DGX Spark 128GB vs Ryzen AI Max+ 395 128GB128 GB vs 128 GB · 273 vs 256 GB/sRTX 3060 12GB vs RTX 3070 8GB12 GB vs 8 GB · 360 vs 448 GB/sRTX 3090 24GB vs RTX 4070 Ti Super 16GB24 GB vs 16 GB · 936 vs 672 GB/sApple M4 Max 128GB vs Apple M4 Pro 48GB128 GB vs 48 GB · 546 vs 273 GB/sRTX 5060 Ti 16GB vs Radeon RX 9060 XT 16GB16 GB vs 16 GB · 448 vs 320 GB/sRTX 3090 24GB vs Radeon RX 7900 XTX 24GB24 GB vs 24 GB · 936 vs 960 GB/sRTX 5080 16GB vs RTX 5090 32GB16 GB vs 32 GB · 960 vs 1792 GB/sRTX 5090 32GB vs RTX PRO 6000 Blackwell 96GB32 GB vs 96 GB · 1792 vs 1792 GB/sApple M3 Ultra 512GB vs Apple M4 Max 128GB512 GB vs 128 GB · 819 vs 546 GB/sApple M4 Max 128GB vs Apple M5 Max 128GB128 GB vs 128 GB · 546 vs 614 GB/sIntel Arc B580 12GB vs RTX 3060 12GB12 GB vs 12 GB · 456 vs 360 GB/sApple M4 Max 128GB vs NVIDIA DGX Spark 128GB128 GB vs 128 GB · 546 vs 273 GB/sRTX 3090 24GB vs Tesla P40 24GB24 GB vs 24 GB · 936 vs 347 GB/sH100 80GB SXM vs NVIDIA H200 141GB80 GB vs 141 GB · 3350 vs 4800 GB/sRTX 4060 Ti 16GB vs RTX 4070 12GB16 GB vs 12 GB · 288 vs 504 GB/s
Quantization and formats
Q4_K_M vs Q5_K_M4.85 vs 5.69 bits per weightQ4_K_M vs Q8_04.85 vs 8.5 bits per weightFP16 / BF16 vs Q8_016 vs 8.5 bits per weightIQ4_XS vs Q4_K_M4.25 vs 4.85 bits per weightQ3_K_M vs Q4_K_M3.91 vs 4.85 bits per weightQ5_K_M vs Q6_K5.69 vs 6.59 bits per weightFP8 vs Q8_08 vs 8.5 bits per weightAWQ vs GGUF (Q4_K_M)4.25 vs 4.85 bits per weight