How Much VRAM Do You Need to Fine-Tune an LLM?
With QLoRA, an 8B model fine-tunes in about 8.3 GB and a 70B model in about 45.9 GB. Full fine-tuning is a different scale: about 133 GB for the same 8B model, because every parameter needs gradients and optimizer states.
VRAM by method
Sequence length 2,048, batch size 1, rank 16 on all linear layers, gradient checkpointing on, AdamW.
| Model | QLoRA | LoRA (16-bit) | Full fine-tune |
|---|---|---|---|
| Llama 3.2 3B | 5.2 GB | 10.0 GB | 54.8 GB |
| Llama 3.1 8B | 8.3 GB | 20.3 GB | 133 GB |
| Qwen 3 14B | 12.8 GB | 34.9 GB | 242 GB |
| Mistral Small 3.1 24B | 18.0 GB | 53.9 GB | 390 GB |
| Qwen 3 32B | 23.9 GB | 72.9 GB | 533 GB |
| Llama 3.3 70B | 45.9 GB | 151 GB | 1142 GB |
Where the memory goes
- Base weights: 2 bytes per parameter in BF16, or about 0.5 in QLoRA's 4-bit NF4.
- Trainable parameters: LoRA adds small adapter matrices, typically under 1% of the model. Each trainable parameter needs about 16 bytes for weights, gradients and Adam states.
- Activations: grow with sequence length × batch size. Gradient checkpointing stores far fewer and recomputes the rest.
- Logits: vocabulary × sequence length, large for models with 150K–260K-token vocabularies.
Sequence length matters
For Llama 3.1 8B with QLoRA:
| Sequence length | VRAM |
|---|---|
| 1,024 tokens | 7.1 GB |
| 2,048 tokens | 8.3 GB |
| 4,096 tokens | 10.7 GB |
| 8,192 tokens | 15.6 GB |
Memory-saving settings
- Gradient checkpointing (on by default in Unsloth, Axolotl and TRL configs).
- 8-bit or paged AdamW.
- Shorter sequences, or packing several short samples into one.
- Gradient accumulation instead of a larger batch.
- Lower LoRA rank or fewer target modules.
GPU guide for fine-tuning
- 12 GB: QLoRA on 7–8B models.
- 16 GB: QLoRA up to 14B.
- 24 GB: QLoRA up to 32B, LoRA on 8B.
- 48 GB: QLoRA on 70B.
- 80 GB+ or multiple GPUs: full fine-tuning of 8B and up.
NVIDIA GPUs are strongly preferred for training; ROCm works for many PyTorch workflows, and Apple Silicon can fine-tune smaller models with MLX. Calculate your own setup with the fine-tuning VRAM calculator.
Choosing a method
- QLoRA: the default for single GPUs. The base model is stored in 4-bit, and small adapters are trained in higher precision. Quality is close to LoRA for most tasks.
- LoRA: the base model stays in 16-bit. Slightly better quality and faster training, at about 2.4× the memory of QLoRA.
- Full fine-tuning: every weight is trained. Only needed for large changes in behaviour or continued pre-training, and it needs multiple GPUs for anything above a few billion parameters.
Example: QLoRA on a 24 GB GPU
# Unsloth / TRL style settings
model: Qwen3-14B, load_in_4bit: true
lora_r: 16, lora_alpha: 16, target_modules: all linear
max_seq_length: 4096, per_device_train_batch_size: 2
gradient_accumulation_steps: 8, gradient_checkpointing: true
optim: adamw_8bitWith these settings Qwen 3 14B needs about 21.8 GB, which leaves headroom on a 24 GB card.
Dataset size and training time
Memory depends on sequence length and batch size, not on how many examples you have. A dataset of a few thousand good examples is enough for most style and format changes. Time scales with total tokens: a few million tokens of QLoRA training on a 14B model takes hours, not days, on one modern GPU.
After training
Merge the adapter into the base model, then convert and quantize to GGUF for llama.cpp, Ollama or LM Studio, or load the adapter alongside the base model in vLLM. The fine-tuned model needs the same inference memory as the original; check it with the LLM VRAM calculator.