Mistral Small 4 119B VRAM Requirements
Mistral Small 4 119B needs about 112 GB of VRAM at FP8 (official) with an 8K context: 111 GB for weights, 0.2 GB for the KV cache and 0.8 GB of runtime overhead. At Q4_K_M it needs 68.2 GB. At Q8_0 it needs 119 GB and at full FP16 precision 223 GB. The smallest GPU that runs it fully is the NVIDIA DGX Spark 128GB.
112 GB VRAM
Mistral Small 4 119B at FP8 (official) with 8K context and the KV cache in FP16.
- Weights 111 GB
- KV cache 0.2 GB
- Runtime overhead 0.8 GB
Mixture of experts: all 119B parameters need memory, but only 6.5B are read per token, so it runs at the speed of a much smaller model.
Which GPUs can run it
- RTX 3060 12GBToo small
- RTX 4060 Ti 16GBToo small
- RTX 3090 24GBToo small
- Radeon RX 7900 XTX 24GBToo small
- RTX 4090 24GBToo small
- RTX 5090 32GBToo small
- RTX 6000 Ada 48GB40% on GPU~9.8 tok/s
- A100 80GB67% on GPU~17 tok/s
- H100 80GB SXM67% on GPU~17 tok/s
- Ryzen AI Max+ 395 128GBToo small
- Apple M4 Max 128GBToo small
- Apple M5 Max 128GBToo small
- RTX PRO 6000 Blackwell 96GB81% on GPU~26 tok/s
- NVIDIA DGX Spark 128GBTight fit~23 tok/s
- Apple M3 Ultra 512GBFits~62 tok/s
Multi-GPU: about 6× 24 GB cards (RTX 3090 / 4090) with the model split across them.
Mistral Small 4 119B VRAM by quantization
Total memory with an FP16 KV cache. "Smallest tier" is the smallest standard VRAM size with a 6% safety margin at 8K.
| Quantization | Bits/weight | Weights | Total at 8K | Total at 32K | Smallest tier |
|---|---|---|---|---|---|
| FP8 (official) | 8.01 | 111 GB | 112 GB | 113 GB | 128 GB |
| Q4_K_M | 4.85 | 67.2 GB | 68.2 GB | 68.7 GB | 80 GB |
| Q5_K_M | 5.69 | 78.8 GB | 79.8 GB | 80.3 GB | 96 GB |
| Q6_K | 6.59 | 91.3 GB | 92.3 GB | 92.8 GB | 128 GB |
| Q8_0 | 8.50 | 118 GB | 119 GB | 119 GB | 128 GB |
| FP16 / BF16 | 16.00 | 222 GB | 223 GB | 223 GB | Multi-GPU / server |
| Q3_K_M | 3.91 | 54.2 GB | 55.2 GB | 55.7 GB | 64 GB |
| Q2_K | 3.00 | 41.6 GB | 42.5 GB | 43.1 GB | 48 GB |
VRAM by context length
At FP8 (official). This model uses Multi-head Latent Attention: each layer caches one compressed 320-value vector per token instead of full keys and values. Runtimes without MLA support use many times more memory.
| Context | KV cache (FP16) | KV cache (Q8_0) | Total, FP16 cache | Total, Q8_0 cache |
|---|---|---|---|---|
| 4K tokens | 90 MB | 48 MB | 112 GB | 112 GB |
| 8K tokens | 0.2 GB | 96 MB | 112 GB | 112 GB |
| 32K tokens | 0.7 GB | 0.4 GB | 113 GB | 112 GB |
| 128K tokens | 2.8 GB | 1.5 GB | 115 GB | 113 GB |
| 256K tokens | 5.6 GB | 3.0 GB | 117 GB | 115 GB |
GPUs that can run Mistral Small 4 119B
At FP8 (official) with 8K context. "Best quant" is the highest-quality format that fits fully; "max context" is at FP8 (official).
| GPU | Usable memory | Status | Best quant | Max context | Speed |
|---|---|---|---|---|---|
| NVIDIA DGX Spark 128GB | 112 GB | Tight fit | Q6_K | – | ~23 tok/s |
| NVIDIA H200 141GB | 141 GB | Fits | FP8 (official) | 256K | ~175 tok/s |
| Apple M2 Ultra 192GB | 144 GB | Fits | FP8 (official) | 256K | ~61 tok/s |
| AMD Instinct MI300X 192GB | 192 GB | Fits | FP8 (official) | 256K | ~170 tok/s |
| Apple M3 Ultra 512GB | 384 GB | Fits | FP8 (official) | 256K | ~62 tok/s |
| RTX PRO 6000 Blackwell 96GB | 96.0 GB | 81% on GPU | Q5_K_M | – | ~26 tok/s |
| A100 80GB | 80.0 GB | 67% on GPU | Q4_K_M | – | ~17 tok/s |
| H100 80GB SXM | 80.0 GB | 67% on GPU | Q4_K_M | – | ~17 tok/s |
| RTX 6000 Ada 48GB | 48.0 GB | 40% on GPU | Q2_K | – | ~9.8 tok/s |
| RTX A6000 48GB | 48.0 GB | 40% on GPU | Q2_K | – | ~9.7 tok/s |
| L40S 48GB | 48.0 GB | 40% on GPU | Q2_K | – | ~9.8 tok/s |
Multi-GPU: about 6× 24 GB cards (RTX 3090 or 4090) run it at FP8 (official), using tensor or layer splitting in llama.cpp, vLLM or ExLlamaV2.
About Mistral Small 4 119B
Despite the name, Mistral Small 4 is a 119B mixture-of-experts (128 experts, 4 active plus a shared expert, about 6.5B active). It uses DeepSeek-style latent attention, caching a 320-value latent per layer, so its long-context cache is very small. The official release is FP8; an NVFP4 checkpoint is also available for Blackwell GPUs.
| Spec | Value |
|---|---|
| Parameters | 119B total, 6.5B active (MoE) |
| Layers | 36 |
| Hidden size | 4096 |
| Attention heads / KV heads | 32 / MLA latent (320) |
| Head dimension | 128 |
| Vocabulary | 131,072 |
| Max context (used here) | 256K tokens |
| KV cache per token (FP16) | 23 KB |
| Released | 2026-03 |
| License | Apache 2.0 |
Mistral Small 4 119B VRAM: common questions
Is Mistral Small 4 actually small?
No. It has 119B total parameters, so even at Q4_K_M it needs about 68 GB. It is fast for its size because only about 6.5B parameters are active per token.
How much VRAM does Mistral Small 4 119B need?
About 112 GB at FP8 (official) with an 8K context, 119 GB at Q8_0 and 223 GB at FP16. Longer contexts add 0.02 GB per 1,000 tokens with an FP16 cache.
What is the cheapest GPU that can run Mistral Small 4 119B?
The smallest listed device that fits it fully at FP8 (official) with 8K context is the NVIDIA DGX Spark 128GB (112 GB usable). With 24 GB cards you need about 6 of them.
Does Mistral Small 4 119B need VRAM for all 119B parameters?
Yes. All experts must be loaded because any token can use any expert. Only 6.5B parameters are read per token, so it generates text about as fast as a 7B dense model.