Batch Size Calculator

Estimate max batch size from available VRAM.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter your GPU's VRAM and your model's size, sequence length and architecture to estimate the largest batch size that fits. For a 125M-parameter model at 2,048 tokens on a 24 GB card the estimate is a batch of 13. Treat it as a starting point and back off if you hit out-of-memory errors.

How to use

1
Step 1
Enter the VRAM in GB and the model details.
2
Step 2
Choose precision, attention memory and optimizer.
3
Step 3
Press Calculate.

How it works

The tool uses 90% of the VRAM, subtracts the fixed cost (weights, gradients, optimizer states and overhead), and divides the remainder by the memory one sample needs (activations and logits). The result is rounded down. It uses the same memory model as the GPU memory estimator.

batch ≈ (0.9 × VRAM − fixed cost) ÷ memory per sample

Worked example

A 125M model on a 24 GB GPU

Input
VRAM 24 GB; 125M params; seq 2,048; hidden 768; 12 layers; 12 heads; bf16, FlashAttention, Adam
Output
Max batch ~13; fixed cost 2.86 GB; 1.36 GB per sample; headroom 18.74 GB
Walkthrough
0.9 × 24 = 21.6 GB, minus 2.86 GB fixed = 18.74 GB. 18.74 / 1.36 = 13.8, rounded down to 13.

Limitations

  • An estimate; fragmentation, data loader buffers and other processes on the GPU are not modelled.
  • Assumes a single GPU without sharding.
  • Using a powers-of-two batch is often more efficient than the exact maximum.

Questions

How do I find the maximum batch size for my GPU?
Enter the VRAM and model settings. The estimate is shown as max batch.
Why not use the exact maximum?
Leave headroom for fragmentation, and consider gradient accumulation if you need a larger effective batch.
How can I fit a bigger batch?
Shorten the sequence, use activation checkpointing, or switch to an 8-bit optimizer.
Does the tool know my GPU?
No, you type in the VRAM.

Features

  • Max batch estimate
  • Fixed vs per-sample memory
  • Runs in your browser

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • TensorFlow 2.0 Complete Course

    freeCodeCamp.org

    Practical ML stack context next to AI utilities.

    Watch on YouTube
  • Big-O Notation in 100 Seconds

    Fireship

    Complexity intuition when estimating training/inference cost.

    Watch on YouTube