LM Studio GPU Offload Explained

Updated October 14, 2026 · Figures calculated live from our model data

The GPU offload slider sets how many of the model's layers live in VRAM. Put it at the maximum whenever the model and its context fit; every layer left on the CPU slows generation. If it does not fit, lower the context or the quantization first, and only then reduce offloaded layers.

What the slider means

A model is a stack of identical layers. Qwen 3 32B has 64, each about 0.3 GB at Q4_K_M. Offloading 40 of 64 layers puts about 11.6 GB of weights on the GPU and leaves the rest in system RAM. The KV cache for offloaded layers also moves to the GPU unless you turn off "Offload KV cache to GPU memory".

How many layers fit?

Approximate layers that fit with an 8K context at Q4_K_M:

Estimates. LM Studio shows its own memory estimate before loading; trust whichever is more conservative.

The settings that change memory

• Context Length: reserved in full at load. The single biggest lever after model size.
• Flash Attention: reduces working memory and is required for a quantized KV cache.
• K and V Cache Quantization: Q8_0 halves the cache with little loss.
• Keep Model in Memory / Try mmap: affect system RAM, not VRAM.
• Number of experts (MoE models): leave at the default; changing it changes quality, not memory.

Speed with partial offload

Generation speed falls quickly as layers move to the CPU, because system RAM is 5–15× slower than VRAM. For Qwen 3 32B at Q4_K_M (21.4 GB):

Mac users: GGUF or MLX?

On Apple Silicon, LM Studio offers MLX models alongside GGUF. MLX is often faster for generation and uses the same memory for the same bit width. There is no offload slider to worry about, because the GPU and CPU share memory; the limit is the GPU-usable share of RAM (see Macs for local LLMs).

Guardrails

LM Studio's model-loading guardrails refuse to load models that would exhaust memory. If a model you know fits is blocked, lower the context first; relaxing the guardrail can freeze the machine if the estimate was right.

A good starting configuration

1. Download the Q4_K_M build of the model (or the MLX 4-bit build on a Mac).
2. In the load settings, move GPU Offload to the maximum.
3. Set Context Length to what you need: 8K for chat, 16–32K for documents, 32–64K for coding agents.
4. Turn on Flash Attention, and set K and V cache quantization to Q8_0 if you need more context.
5. Load the model and watch the memory estimate. If it turns red or loading fails, reduce context first.

Using LM Studio as a local server

The Developer tab runs an OpenAI-compatible server on http://localhost:1234/v1. Coding tools such as Continue and Cline can point at it. The server uses the same load settings, so the context you set is the context the tool gets; set it high enough for the tool, or prompts will be cut off.

Running two models at once

LM Studio can keep several models loaded, for example a chat model and an embedding model. Their memory adds up. Qwen 3 14B at 8K needs 10.5 GB; a small embedding model adds about 0.5–1 GB. On a 12 GB card that leaves almost nothing, so unload models you are not using.

Windows and AMD notes

On Windows with an AMD GPU, LM Studio offers ROCm and Vulkan runtimes. Vulkan works on more cards; ROCm can be faster on supported ones. On laptops with both integrated and discrete graphics, make sure LM Studio uses the discrete GPU in the runtime settings. See AMD GPUs for local LLMs.

When to stop offloading and change models

If you can only offload about half the layers, generation speed falls to a few tokens per second. At that point a smaller model or a mixture-of-experts model usually gives a better experience than squeezing a large dense model. Qwen 3.6 35B-A3B with partial offload stays much faster than a dense 32B because each token reads only about 3B parameters.