GPU Memory Estimator

Estimate VRAM for weights, optimizer, and activations.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter the parameter count, batch size, sequence length and architecture sizes, choose precision, attention type and optimizer, and get an estimated VRAM total with a breakdown. Pick the optimizer None for an inference-style estimate with a KV cache.

How to use

1
Step 1
Enter parameters, batch size, sequence length, hidden size, layers, heads and vocabulary.
2
Step 2
Choose precision, attention memory and optimizer.
3
Step 3
Press Calculate and read the breakdown.

How it works

Weights, gradients and optimizer states are charged per parameter (bf16/fp16 mixed precision uses 2 bytes for weights, 2 for gradients and 12 for Adam states including fp32 master weights). Activations are about 34·s·b·h bytes per layer, and FlashAttention avoids storing the s² score matrices. Loss logits are counted in fp32, then 1 GB for the CUDA context and 10% allocator headroom are added. 1 GB here means 1024³ bytes.

Worked example

A 125M model, Adam, bf16

Input
125M params; batch 4; seq 2,048; hidden 768; 12 layers; 12 heads; vocab 50,257; mixed, FlashAttention, Adam
Output
Total 9.15 GB: weights 238.4 MB, gradients 238.4 MB, optimizer 1.40 GB, activations 2.39 GB, logits 3.07 GB, overhead 1.00 GB, headroom 852 MB
Walkthrough
The fp32 loss logits (3.07 GB) are larger than the weights because vocab × seq × batch is big. For a 7B model with optimizer None, batch 1 and seq 2,048 the tool gives 16.61 GB, including a 1.00 GB KV cache.

Limitations

  • A planning estimate, usually within a modest margin but not exact; real usage depends on framework, kernels and fragmentation.
  • Does not model tensor/pipeline parallelism, ZeRO sharding or quantised weights below 16 bits.
  • Treats the model as a standard decoder transformer.

Questions

How much GPU memory do I need to train a model?
Enter your model size, batch and sequence length. The tool shows weights, gradients, optimizer, activations and overhead.
How much VRAM for a 7B model at inference?
At bf16 about 13 GB for weights plus KV cache and overhead, around 16.6 GB at batch 1 and a 2,048-token context.
Why is Adam so memory-hungry?
It stores two moment estimates per parameter and often fp32 master weights, so optimizer state is several times the weight size.
What does FlashAttention change?
It does not store the full attention score matrices, which lowers activation memory for long sequences.

Features

  • Full memory breakdown
  • Precision, optimizer, attention options
  • Inference mode with KV cache

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • Big-O Notation in 100 Seconds

    Fireship

    Complexity intuition when estimating training/inference cost.

    Watch on YouTube
  • Finding mean, median, and mode

    Khan Academy

    Baseline stats for accuracy and evaluation helpers.

    Watch on YouTube