GPU Memory Estimator
Estimate VRAM for weights, optimizer, and activations.
About this tool
Enter the parameter count, batch size, sequence length and architecture sizes, choose precision, attention type and optimizer, and get an estimated VRAM total with a breakdown. Pick the optimizer None for an inference-style estimate with a KV cache.
How to use
How it works
Weights, gradients and optimizer states are charged per parameter (bf16/fp16 mixed precision uses 2 bytes for weights, 2 for gradients and 12 for Adam states including fp32 master weights). Activations are about 34·s·b·h bytes per layer, and FlashAttention avoids storing the s² score matrices. Loss logits are counted in fp32, then 1 GB for the CUDA context and 10% allocator headroom are added. 1 GB here means 1024³ bytes.
Worked example
A 125M model, Adam, bf16
- Input
- 125M params; batch 4; seq 2,048; hidden 768; 12 layers; 12 heads; vocab 50,257; mixed, FlashAttention, Adam
- Output
- Total 9.15 GB: weights 238.4 MB, gradients 238.4 MB, optimizer 1.40 GB, activations 2.39 GB, logits 3.07 GB, overhead 1.00 GB, headroom 852 MB
- Walkthrough
- The fp32 loss logits (3.07 GB) are larger than the weights because vocab × seq × batch is big. For a 7B model with optimizer None, batch 1 and seq 2,048 the tool gives 16.61 GB, including a 1.00 GB KV cache.
Limitations
- A planning estimate, usually within a modest margin but not exact; real usage depends on framework, kernels and fragmentation.
- Does not model tensor/pipeline parallelism, ZeRO sharding or quantised weights below 16 bits.
- Treats the model as a standard decoder transformer.
Questions
How much GPU memory do I need to train a model?
How much VRAM for a 7B model at inference?
Why is Adam so memory-hungry?
What does FlashAttention change?
Features
- Full memory breakdown
- Precision, optimizer, attention options
- Inference mode with KV cache
Related resources
Guides and videos from around the web—opens on YouTube or the publisher’s site.
Big-O Notation in 100 Seconds
Fireship
Complexity intuition when estimating training/inference cost.
Watch on YouTubeFinding mean, median, and mode
Khan Academy
Baseline stats for accuracy and evaluation helpers.
Watch on YouTube
- Machine Learning Crash Course
Google Developers
Official ML crash course pairing with metric calculators.
- OpenAI Research
OpenAI
Research notes for people exploring AI tooling trends.