Batch Size Calculator
Estimate max batch size from available VRAM.
About this tool
Enter your GPU's VRAM and your model's size, sequence length and architecture to estimate the largest batch size that fits. For a 125M-parameter model at 2,048 tokens on a 24 GB card the estimate is a batch of 13. Treat it as a starting point and back off if you hit out-of-memory errors.
How to use
How it works
The tool uses 90% of the VRAM, subtracts the fixed cost (weights, gradients, optimizer states and overhead), and divides the remainder by the memory one sample needs (activations and logits). The result is rounded down. It uses the same memory model as the GPU memory estimator.
batch ≈ (0.9 × VRAM − fixed cost) ÷ memory per sample
Worked example
A 125M model on a 24 GB GPU
- Input
- VRAM 24 GB; 125M params; seq 2,048; hidden 768; 12 layers; 12 heads; bf16, FlashAttention, Adam
- Output
- Max batch ~13; fixed cost 2.86 GB; 1.36 GB per sample; headroom 18.74 GB
- Walkthrough
- 0.9 × 24 = 21.6 GB, minus 2.86 GB fixed = 18.74 GB. 18.74 / 1.36 = 13.8, rounded down to 13.
Limitations
- An estimate; fragmentation, data loader buffers and other processes on the GPU are not modelled.
- Assumes a single GPU without sharding.
- Using a powers-of-two batch is often more efficient than the exact maximum.
Questions
How do I find the maximum batch size for my GPU?
Why not use the exact maximum?
How can I fit a bigger batch?
Does the tool know my GPU?
Features
- Max batch estimate
- Fixed vs per-sample memory
- Runs in your browser
Related resources
Guides and videos from around the web—opens on YouTube or the publisher’s site.
TensorFlow 2.0 Complete Course
freeCodeCamp.org
Practical ML stack context next to AI utilities.
Watch on YouTubeBig-O Notation in 100 Seconds
Fireship
Complexity intuition when estimating training/inference cost.
Watch on YouTube
- Machine Learning Crash Course
Google Developers
Official ML crash course pairing with metric calculators.
- Hugging Face docs
Hugging Face
Model hub docs next to tokenizer and embedding helpers.