Fine-Tuning VRAM Calculator (LoRA, QLoRA, Full)
Estimate memory for LoRA, QLoRA and full fine-tuning, broken into base weights, optimizer states and activations. QLoRA on Llama 3.1 8B needs about 8.3 GB at 2,048 tokens; Llama 3.3 70B needs about 45.9 GB.
8.3 GB VRAM
QLoRA (4-bit base) of Llama 3.1 8B at 2048 tokens × batch 1. Trainable parameters: 41.9M (0.52% of the model).
- Base weights (4-bit) 3.9 GB
- Adapter + optimizer 0.6 GB
- Activations 2.3 GB
- Overhead 1.5 GB
GPUs that can train it
- RTX 3060 12GBFits
- RTX 4060 Ti 16GBFits
- RTX 3090 24GBFits
- Radeon RX 7900 XTX 24GBFits
- RTX 4090 24GBFits
- RTX 5090 32GBFits
- RTX 6000 Ada 48GBFits
- A100 80GBFits
- H100 80GB SXMFits
- Ryzen AI Max+ 395 128GBFits
- Apple M4 Max 128GBFits
- Apple M5 Max 128GBFits
- RTX PRO 6000 Blackwell 96GBFits
- NVIDIA DGX Spark 128GBFits
- Apple M3 Ultra 512GBFits
What uses memory during fine-tuning
- Base weights: 2 bytes per parameter in BF16 for LoRA and full fine-tuning, about 0.52 bytes for QLoRA's 4-bit NF4 with double quantization.
- Optimizer and gradients: full fine-tuning with AdamW needs about 14 extra bytes per parameter (BF16 gradients, FP32 master weights, two FP32 Adam moments). That is why full fine-tuning an 8B model needs about 133 GB. LoRA only trains small adapter matrices, usually under 1% of parameters.
- Activations: grow with sequence length × batch size. Gradient checkpointing stores only each layer's input and recomputes the rest, cutting this several-fold.
- Logits: models with large vocabularies (128K to 262K tokens) produce big output tensors at long sequence lengths.
Mixture-of-experts models are not listed because their memory depends heavily on the training framework. Inference needs are on the LLM VRAM calculator.
Frequently asked questions
How much VRAM do I need to fine-tune a 7B or 8B model?
About 8.3 GB with QLoRA at 2,048 tokens, so a 12 GB card works. LoRA in 16-bit needs about 20.3 GB, and full fine-tuning needs about 133 GB without memory-saving tricks.
Can I fine-tune a 70B model on one GPU?
With QLoRA, a 48 GB card such as an RTX 6000 Ada or A6000 can just about do it at short sequence lengths (about 45.9 GB at 2,048 tokens). Two 24 GB cards work with frameworks that split the model.
Does LoRA rank change VRAM much?
Barely. Rank 16 on all linear layers of an 8B model is about 42M trainable parameters, which adds under 1 GB. Sequence length and batch size matter far more.