Is 8 GB of VRAM enough for LLMs?

Yes for 7B and 8B models at Q4_K_M with up to about 16K of context. 14B models need 12 GB or more.

How much VRAM does a 70B model need?

About 43.2 GB at Q4_K_M with 8K context, so two 24 GB GPUs, one 48 GB card or a Mac with 64 GB or more.

Does system RAM count as VRAM?

Not on a discrete GPU. Runtimes can spill layers into system RAM, but it is 10–30× slower. On Apple Silicon and AMD Strix Halo, memory is shared, so most of it is usable by the GPU.

Is 12GB of VRAM enough for LLMs?

Yes for 12–14B models at Q4_K_M with a moderate context. Qwen 3 14B needs about 10.5 GB at 8K. 27B and larger models need 16–24 GB.

How much VRAM do I need for a 32B model?

About 21.4 GB at Q4_K_M with 8K context, so a 24 GB GPU. Newer hybrid-attention 27B models such as Qwen 3.6 27B need only 16.9 GB.

How Much VRAM Do You Need to Run an LLM?

Updated October 14, 2026 · Figures calculated live from our model data

As a rule of thumb, an LLM needs about 0.6 GB of VRAM per billion parameters at Q4_K_M, plus 1–2 GB for context and overhead. An 8B model needs about 6.3 GB, a 32B model about 21.4 GB and a 70B model about 43.2 GB with an 8K context.

The three things that use memory

Every running model holds three things in GPU memory:

The weights. Parameters × bits per weight ÷ 8. At FP16 that is 2 bytes per parameter, so an 8B model is about 15 GB. Quantized to Q4_K_M (about 4.85 bits) it drops to about 4.5 GB.
The KV cache. The model stores keys and values for every token in the context, on every layer. For Llama 3.1 8B that is 128 KB per token, so 32K tokens add 4.0 GB. Use the KV cache calculator to see any model.
Overhead. The CUDA or Metal context and working buffers: usually 0.6–1.2 GB.

VRAM by model size

At Q4_K_M with 8K context and an FP16 cache:

Model size	Example	VRAM needed	Typical GPU
1–4B	Llama 3.2 3B	3.5 GB	4–6 GB, or CPU only
7–8B	Llama 3.1 8B	6.3 GB	8 GB
12–14B	Qwen 3 14B	10.5 GB	12 GB
24–27B	Gemma 3 27B	17.6 GB	24 GB (or 20 GB)
32B	Qwen 3 32B	21.4 GB	24 GB
70B	Llama 3.3 70B	43.2 GB	48 GB (2× 24 GB)
120B MoE	gpt-oss-120b	62.0 GB	80 GB or 128 GB unified

Context length can matter more than model size

Long contexts are expensive on deep models without memory-saving attention. Qwen 2.5 14B needs 10.7 GB at 8K but 15.2 GB at 32K. Models with sliding-window attention are the exception: Gemma 3 27B grows from 17.6 GB to only 19.5 GB over the same range.

Two easy wins: set your app's context to what you actually use (some default to the model maximum), and use a Q8_0 KV cache, which halves cache memory with little quality loss.

Mixture-of-experts models

MoE models such as Qwen 3 30B-A3B and gpt-oss-120b need memory for all their parameters, but only read the active ones per token. That makes them fast for their size and a good match for big-memory, lower-bandwidth systems like Macs and the Ryzen AI Max+ 395 128GB.

What if the model doesn't fit?

Drop one quantization step (Q5_K_M → Q4_K_M → IQ4_XS). See quantization explained.
Shorten the context or quantize the KV cache.
Offload some layers to system RAM. It works, but speed drops sharply.
Add a second GPU. llama.cpp, Ollama and vLLM split models across cards.

For any specific model, the LLM VRAM calculator gives the exact figure and the GPUs that fit.

Worked example: sizing a model step by step

Take Qwen 3 14B at Q4_K_M with a 16K context, a common setup on a 12 GB card.

Weights: 14.8B parameters × 4.85 bits ÷ 8 = 8.4 GB.
KV cache: 160 KB per token × 16,384 tokens = 2.5 GB.
Overhead: about 0.9 GB for the CUDA context and compute buffers.
Total: 11.7 GB. A 12 GB card has about 11.3 GB of safe working room, so this is a tight fit. Dropping to 12K context or using a Q8_0 cache (10.6 GB) gives comfortable headroom.

Common mistakes when estimating VRAM

Using the download size as the answer. The file is only the weights. The KV cache and buffers come on top, and for long contexts they can double the total.
Counting active parameters for MoE models. A model labelled A3B still needs memory for all its experts. See MoE models and VRAM.
Forgetting the display. If your monitor is plugged into the same GPU, the desktop and browser can take 0.5–1.5 GB.
Trusting a 2 GB per billion rule. That is the FP16 figure. Almost nobody runs local models in FP16; at Q4_K_M it is closer to 0.6 GB per billion.
Ignoring the runtime's default context. Some apps reserve the model's full context window, which can be 128K tokens or more.

Unified memory: Macs and AMD Strix Halo

On Apple Silicon and on AMD's Ryzen AI Max systems, the GPU shares system memory. The whole pool is not available to the GPU by default: macOS gives it about two thirds on smaller machines and about three quarters on larger ones. A 64 GB Mac therefore behaves like a GPU with about 48 GB, enough for Llama 3.3 70B at Q4_K_M with a short context. Bandwidth is lower than on high-end discrete GPUs, so mixture-of-experts models are the best match. Our Mac guide covers this in detail.

How much VRAM should you buy?

Buy for the models you want to run in a year, not today. Model sizes people actually use have settled into clear bands: 8–9B for laptops and 8 GB cards, 14B for 12 GB cards, 27–35B for 24 GB cards, and 70B or 100B-class mixture-of-experts for 48 GB and up. Moving up one band makes a bigger difference to quality than any quantization choice. If you can afford 24 GB, it is the most useful single step for local AI in 2026.