FP8 vs Q8_0: VRAM, Quality and Speed
Q8_0 uses about 6% more memory than FP8 (8.5 vs 8 bits per weight). On Qwen 3 32B with an 8K context that is 35.3 GB versus 33.4 GB. Memory needs are nearly the same (8 vs 8.5 bits per weight).
VRAM on popular models
Total memory with an 8K context and FP16 KV cache. The last column is the smallest standard VRAM size that holds each version.
| Model | FP8 | Q8_0 | Difference | Smallest tier |
|---|---|---|---|---|
| Llama 3.1 8B | 9.3 GB | 9.8 GB | +0.5 GB | 10 GB / 12 GB |
| Qwen 3.5 9B | 9.7 GB | 10.2 GB | +0.5 GB | 12 GB / 12 GB |
| Qwen 3 14B | 15.9 GB | 16.8 GB | +0.9 GB | 20 GB / 20 GB |
| Mistral Small 3.1 24B | 24.4 GB | 25.8 GB | +1.4 GB | 32 GB / 32 GB |
| Qwen 3.6 27B | 26.8 GB | 28.3 GB | +1.6 GB | 32 GB / 32 GB |
| Gemma 4 31B | 31.1 GB | 32.9 GB | +1.8 GB | 48 GB / 48 GB |
| Qwen 3 32B | 33.4 GB | 35.3 GB | +1.9 GB | 48 GB / 48 GB |
| Qwen 3.6 35B-A3B | 33.8 GB | 35.9 GB | +2.0 GB | 48 GB / 48 GB |
| Llama 3.3 70B | 69.1 GB | 73.2 GB | +4.1 GB | 80 GB / 80 GB |
| Qwen 3.5 122B-A10B | 115 GB | 122 GB | +7.1 GB | 128 GB / Multi-GPU |
Largest model each VRAM size can run
The biggest model released since 2025 that fits fully at 8K context.
Context you can fit
Longer contexts need room for the KV cache. The longest context that fits fully at each format, with an FP16 cache:
| Model | GPU | FP8 | Q8_0 |
|---|---|---|---|
| Qwen 3 14B | 16 GB | 2K | Does not fit |
| Qwen 3.6 27B | 24 GB | Does not fit | Does not fit |
| Qwen 3 32B | 24 GB | Does not fit | Does not fit |
| Gemma 4 31B | 32 GB | Does not fit | Does not fit |
| Llama 3.3 70B | 48 GB | Does not fit | Does not fit |
Download size
File sizes in decimal GB, as Hugging Face shows them. Use the GGUF size calculator for any other model.
| Model | FP8 | Q8_0 |
|---|---|---|
| Llama 3.1 8B | 8.0 GB | 8.5 GB |
| Qwen 3 14B | 14.8 GB | 15.7 GB |
| Qwen 3.6 27B | 26.9 GB | 28.6 GB |
| Qwen 3 32B | 32.8 GB | 34.8 GB |
| Llama 3.3 70B | 70.6 GB | 75.0 GB |
Quality
Both are 8-bit and close to the original model. FP8 (E4M3) is a floating-point format used by vLLM, SGLang and TensorRT-LLM, with hardware support on Ada, Hopper and Blackwell GPUs. Q8_0 is llama.cpp's 8-bit integer format with a scale per 32 weights, which works on any GPU, CPU or Mac.
Speed
Text generation reads every weight once per token, so smaller formats generate faster on the same GPU. Estimated on an RTX 4090 24GB:
| Model | FP8 | Q8_0 |
|---|---|---|
| Llama 3.1 8B | ~75 tok/s | ~71 tok/s |
| Qwen 3 32B | ~3.4 tok/s (partial CPU offload) | ~2.9 tok/s (partial CPU offload) |
Verdict
Memory needs are nearly the same (8 vs 8.5 bits per weight). Pick by runtime: FP8 for vLLM and other serving stacks on RTX 40-series, 50-series, L40S, H100 and newer; Q8_0 for llama.cpp, Ollama, LM Studio, Macs and older GPUs.
Frequently asked questions
Can an RTX 3090 run FP8 models?
Not natively. Ampere GPUs lack FP8 tensor cores; some runtimes emulate it with weight-only FP8, which saves memory but not compute. Q8_0 GGUF is the simpler choice on a 3090.
How much more VRAM does Q8_0 need than FP8?
About 6% more for the weights (8.5 vs 8 bits per weight). For Qwen 3 32B with an 8K context that is 35.3 GB versus 33.4 GB.