Ollama VRAM Settings That Actually Matter

Updated October 14, 2026 · Figures calculated live from our model data

Three settings decide how much VRAM Ollama uses: the context length (num_ctx), how many requests it serves in parallel, and the KV cache type. Get those right and a model that "doesn't fit" often does. Check the result with ollama ps: anything below 100% GPU means part of the model runs on the CPU.

1. Check where the model is running

ollama ps
# NAME         SIZE     PROCESSOR          CONTEXT
# qwen3:14b    12 GB    18%/82% CPU/GPU    16384

The PROCESSOR column is the key number. "100% GPU" is what you want. Any CPU share means layers were offloaded to system RAM, and generation slows sharply (see CPU offloading).

2. Set the context length deliberately

The context is reserved in full when the model loads. For Qwen 3 14B at Q4_K_M:

Ways to set it, from most to least permanent:

# server-wide default (Linux service, macOS app or Windows env var)
OLLAMA_CONTEXT_LENGTH=16384

# per model, in a Modelfile
PARAMETER num_ctx 16384

# per session, inside ollama run
/set parameter num_ctx 16384

# per request, through the API
{"model": "qwen3:14b", "options": {"num_ctx": 16384}}

Recent Ollama versions choose a default context from your available VRAM, and older ones defaulted to 2K–4K tokens, which silently truncates long prompts. Setting it yourself avoids both surprises.

3. Turn on flash attention and quantize the KV cache

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0     # or q4_0 for maximum savings

A Q8_0 cache halves cache memory with very little quality loss; it requires flash attention. On Qwen 3 14B at 32K that is 5.0 GB → 2.7 GB. Q4_0 saves more but can hurt recall over long documents.

4. Limit parallel requests and loaded models

OLLAMA_NUM_PARALLEL=1         # each parallel slot needs its own cache
OLLAMA_MAX_LOADED_MODELS=1    # keep one model in VRAM at a time
OLLAMA_KEEP_ALIVE=10m         # unload after 10 minutes idle

With four parallel slots, an 8K context costs as much cache as one 32K context. For a single user, one slot is enough.

5. Force or limit GPU layers

Ollama estimates how many layers fit and offloads the rest. If it is too cautious or too aggressive, set num_gpu (the number of layers on the GPU) in a Modelfile or API options. A setting that is too high leads to out-of-memory errors at load time or during long prompts.

6. MoE models: keep experts on the CPU, attention on the GPU

For big mixture-of-experts models that don't fit, offloading whole layers wastes the GPU. llama.cpp can keep only the expert weights in system RAM (--n-cpu-moe), which is much faster; with Ollama, a model like Qwen 3.6 35B-A3B that needs 21.0 GB still runs at a usable ~40 tokens/s on a 16 GB card with automatic offloading.

Use the LLM VRAM calculator to find the exact context that fits your GPU before changing settings.

Where to set environment variables

• Linux (systemd service): run sudo systemctl edit ollama, add the variables under [Service] as Environment="OLLAMA_FLASH_ATTENTION=1", then sudo systemctl restart ollama.
• macOS app: launchctl setenv OLLAMA_FLASH_ATTENTION 1, then quit and reopen Ollama.
• Windows: add the variables in System Properties → Environment Variables, then quit Ollama from the tray and start it again.

A Modelfile that pins the settings

FROM qwen3:14b
PARAMETER num_ctx 16384
PARAMETER num_gpu 99
# then: ollama create qwen3-14b-16k -f Modelfile

Creating a named variant means every app that uses it (Open WebUI, Continue, scripts) gets the same context without its own settings.

Recommended settings by GPU size

Troubleshooting checklist

1. Run ollama ps. Anything other than 100% GPU means the model or its context does not fit.
2. Check the server log (journalctl -u ollama on Linux, or the logs folder on macOS and Windows) for the number of offloaded layers and the memory it estimated.
3. Make sure the GPU is detected: the log lists it at start-up. If not, update the driver; for AMD, check the card is on the supported list.
4. Close other GPU-heavy apps, then reload the model.

Prefer a graphical app? LM Studio exposes the same settings as sliders.