What LLMs Can the RTX 4070 12GB Run?
The RTX 4070 12GB has 12 GB of VRAM and 504 GB/s of bandwidth. It runs 25 of 58 models fully at Q4_K_M with 8K context, up to Qwen 3 14B, and generates about 59 tokens/s on an 8B model.
25 of 58 models fit
RTX 4070 12GB (12.0 GB) at Q4_K_M with 8K context. Largest that fits fully: Qwen 3 14B.
| Model | Needs | Status | Speed | Best quant that fits |
|---|---|---|---|---|
| Qwen 2.5 14B | 10.7 GB | Fits | ~33 tok/s | Q4_K_M |
| DeepSeek R1 Distill Qwen 14B | 10.7 GB | Fits | ~33 tok/s | Q4_K_M |
| Phi-4 14B | 10.6 GB | Fits | ~33 tok/s | Q4_K_M |
| Qwen 3 14B | 10.5 GB | Fits | ~34 tok/s | Q4_K_M |
| Ministral 3 14B | 9.7 GB | Fits | ~36 tok/s | Q5_K_M |
| Mistral Nemo 12B | 9.0 GB | Fits | ~40 tok/s | Q5_K_M |
| Gemma 3 12B | 8.8 GB | Fits | ~40 tok/s | Q6_K |
| Ministral 3 8B | 6.7 GB | Fits | ~56 tok/s | Q8_0 |
| Qwen 3 8B | 6.6 GB | Fits | ~58 tok/s | Q8_0 |
| Qwen 3.5 9B | 6.4 GB | Fits | ~57 tok/s | Q8_0 |
| Llama 3.1 8B | 6.3 GB | Fits | ~59 tok/s | Q8_0 |
| DeepSeek R1 Distill Llama 8B | 6.3 GB | Fits | ~59 tok/s | Q8_0 |
| Mistral 7B v0.3 | 5.7 GB | Fits | ~65 tok/s | Q8_0 |
| Gemma 4 E4B | 5.7 GB | Fits | ~65 tok/s | Q8_0 |
| Qwen 2.5 7B | 5.6 GB | Fits | ~66 tok/s | Q8_0 |
| DeepSeek R1 Distill Qwen 7B | 5.6 GB | Fits | ~66 tok/s | Q8_0 |
| Qwen 3 4B | 4.2 GB | Fits | ~104 tok/s | FP16 / BF16 |
| Phi-4 Mini | 4.1 GB | Fits | ~110 tok/s | FP16 / BF16 |
| Gemma 4 E2B | 4.0 GB | Fits | ~101 tok/s | FP16 / BF16 |
| Gemma 3 4B | 3.7 GB | Fits | ~112 tok/s | FP16 / BF16 |
| Qwen 3.5 4B | 3.7 GB | Fits | ~115 tok/s | FP16 / BF16 |
| Ministral 3 3B | 3.5 GB | Fits | ~126 tok/s | FP16 / BF16 |
| Llama 3.2 3B | 3.5 GB | Fits | ~131 tok/s | FP16 / BF16 |
| Llama 3.2 1B | 1.7 GB | Fits | ~349 tok/s | FP16 / BF16 |
| Gemma 3 1B | 1.6 GB | Fits | ~455 tok/s | FP16 / BF16 |
| Mixtral 8x7B | 28.0 GB | 40% on GPU | ~7.6 tok/s | – |
| Qwen 2.5 32B | 21.4 GB | 53% on GPU | ~3.7 tok/s | – |
| Qwen 2.5 Coder 32B | 21.4 GB | 53% on GPU | ~3.7 tok/s | – |
| DeepSeek R1 Distill Qwen 32B | 21.4 GB | 53% on GPU | ~3.7 tok/s | – |
| Qwen 3 32B | 21.4 GB | 53% on GPU | ~3.7 tok/s | – |
| Qwen 3.5 35B-A3B | 21.0 GB | 54% on GPU | ~35 tok/s | – |
| Qwen 3.6 35B-A3B | 21.0 GB | 54% on GPU | ~35 tok/s | – |
| Gemma 4 31B | 19.8 GB | 57% on GPU | ~4.2 tok/s | – |
| Qwen 3 30B-A3B | 18.8 GB | 60% on GPU | ~31 tok/s | – |
| Qwen3-Coder 30B-A3B | 18.8 GB | 60% on GPU | ~31 tok/s | – |
| Gemma 3 27B | 17.6 GB | 64% on GPU | ~5.5 tok/s | – |
| Qwen 3.5 27B | 16.9 GB | 67% on GPU | ~6.0 tok/s | Q2_K |
| Qwen 3.6 27B | 16.9 GB | 67% on GPU | ~6.0 tok/s | Q2_K |
| Qwen 3.8 27B | 16.9 GB | 67% on GPU | ~6.0 tok/s | Q2_K |
| Mistral Small 3.1 24B | 15.6 GB | 72% on GPU | ~7.4 tok/s | Q2_K |
| Gemma 4 26B-A4B | 15.6 GB | 72% on GPU | ~39 tok/s | Q2_K |
| gpt-oss-20b | 12.9 GB | 87% on GPU | ~65 tok/s | Q3_K_M |
| Kimi K2.6 | 566 GB | Too small | – | – |
| DeepSeek R1 671B | 380 GB | Too small | – | – |
| Llama 3.1 405B | 234 GB | Too small | – | – |
| Qwen 3.5 397B-A17B | 226 GB | Too small | – | – |
| DeepSeek V4 Flash | 161 GB | Too small | – | – |
| Qwen 3 235B-A22B | 135 GB | Too small | – | – |
| MiniMax-M2 | 133 GB | Too small | – | – |
| Qwen 3.5 122B-A10B | 70.2 GB | Too small | – | – |
| Mistral Small 4 119B | 68.2 GB | Too small | – | – |
| gpt-oss-120b | 67.2 GB | Too small | – | – |
| Llama 4 Scout | 64.0 GB | Too small | – | – |
| GLM-4.5-Air | 62.1 GB | Too small | – | – |
| Qwen3-Next 80B-A3B | 46.3 GB | Too small | – | – |
| Qwen 2.5 72B | 44.5 GB | Too small | – | – |
| Llama 3.3 70B | 43.2 GB | Too small | – | – |
| DeepSeek R1 Distill Llama 70B | 43.2 GB | Too small | – | – |
RTX 4070 12GB for local AI
Same 12 GB as the RTX 3060 with more bandwidth, so it runs the same models about 40% faster.
Setup tips: excellent for 8–14B models, where its bandwidth gives quick replies. Keep contexts to 8–16K on 14B models, or use a Q8_0 KV cache. For image generation, FLUX in GGUF Q5–Q6 with model swapping fits well.
Among the 10 devices we track with about 12 GB of usable memory, the RTX 4070 12GB ranks 3 of 10 for text-generation speed, because generation speed follows memory bandwidth. Others in the same class: RTX 3080 12GB, RTX 5070 12GB, RTX 4070 Super 12GB, RTX 4070 Ti 12GB. They all hold the same models; the faster ones simply answer sooner.
Our picks for the RTX 4070 12GB
The model we would run first for each job, chosen from our recommendations by what fits fully with an 8K context. "Best quant" is the highest quality that fits; "max context" is at Q4_K_M or the native format.
| Task | Model | Best quant | Max context | Speed |
|---|---|---|---|---|
| Best all-rounder | Qwen 3 14B | Q4_K_M | 13K | ~34 tok/s |
| Coding | Qwen 3 14B | Q4_K_M | 13K | ~34 tok/s |
| Fastest replies | Qwen 3.5 4B | FP16 / BF16 | 251K | ~115 tok/s |
| Long documents (64K+) | Qwen 3.5 9B | Q8_0 | 164K | ~57 tok/s |
| Images in, text out | Ministral 3 14B | Q5_K_M | 18K | ~36 tok/s |
| Image generation | FLUX.1 [dev] | GGUF Q6_K | – | – |
| Spec | Value |
|---|---|
| Memory | 12 GB |
| Memory bandwidth | 504 GB/s |
| Architecture | Ada Lovelace |
| Released | 2023-04 |
| Llama 3.1 8B Q4_K_M speed | ~59 tok/s |
| Largest model that fits | Qwen 3 14B |
Image and video models on the RTX 4070 12GB
Peak VRAM with model swapping (ComfyUI default on smaller cards).
| Model | FP8 | FP8 fit | GGUF Q4 | Q4 fit |
|---|---|---|---|---|
| Stable Diffusion 1.5 | 1.8 GB | Fits | 1.7 GB | Fits |
| SDXL 1.0 | 4.1 GB | Fits | 4.1 GB | Fits |
| Stable Diffusion 3.5 Medium | 6.4 GB | Fits | 6.4 GB | Fits |
| Stable Diffusion 3.5 Large | 8.6 GB | Fits | 6.4 GB | Fits |
| FLUX.1 [dev] | 12.1 GB | 93% on GPU | 7.3 GB | Fits |
| FLUX.1 [schnell] | 12.1 GB | 93% on GPU | 7.3 GB | Fits |
| Qwen-Image | 20.4 GB | 55% on GPU | 12.3 GB | 92% on GPU |
| Wan 2.2 A14B (T2V / I2V) | 18.8 GB | 60% on GPU | 13.2 GB | 85% on GPU |
| Wan 2.2 TI2V 5B | 8.0 GB | Fits | 6.5 GB | Fits |
| HunyuanVideo | 18.3 GB | 61% on GPU | 13.1 GB | 86% on GPU |
| FLUX.1 Kontext [dev] | 12.3 GB | 92% on GPU | 7.5 GB | Fits |
| FLUX.2 [dev] | 31.0 GB | 36% on GPU | 23.6 GB | 48% on GPU |
| Z-Image Turbo | 6.8 GB | Fits | 4.9 GB | Fits |
| Qwen-Image-Edit 2509 | 20.5 GB | 55% on GPU | 12.4 GB | 91% on GPU |
| LTX-2 | 19.2 GB | 59% on GPU | 12.6 GB | 90% on GPU |
Upgrade path
The next step up is the RTX 4060 Ti 16GB with 16 GB.
Comparisons
RTX 4070 12GB for LLMs: common questions
Can the RTX 4070 12GB run Llama 3.3 70B?
No. It needs about 43.2 GB at Q4_K_M, far more than the 12.0 GB available.
Can the RTX 4070 12GB run Qwen 3 32B?
Only partly. It needs about 21.4 GB at Q4_K_M, so about 53% fits in VRAM and the rest runs from system RAM, which is slow.
Can the RTX 4070 12GB run Qwen 3 14B?
Yes. Qwen 3 14B needs about 10.5 GB at Q4_K_M with 8K context, and the best quality that fits is Q4_K_M.
How fast is the RTX 4070 12GB for LLMs?
About 59 tokens per second for an 8B model at Q4_K_M, based on its 504 GB/s memory bandwidth. Larger models scale down roughly in proportion to their size.