FLOPs Calculator
Estimate forward or training FLOPs from params × tokens.
About this tool
Enter the number of parameters and the number of tokens, choose training or forward, and get the total floating-point operations. The rule of thumb is 6 × N × T for training and 2 × N × T for a forward pass, and the tool also converts the total to petaFLOP/s-days.
How to use
How it works
Each token costs about 2 FLOPs per parameter in the forward pass and about twice that again for the backward pass, giving 6NT for training. One petaFLOP/s-day is 10¹⁵ × 86,400 FLOPs. The headline figure uses a K, M or B suffix below 10¹² FLOPs and scientific notation (for example 2.250e+20) above that. The table shows both forms.
training ≈ 6·N·T forward ≈ 2·N·T
Worked example
A 125M model on 300B tokens
- Input
- N = 125,000,000; T = 300,000,000,000; training
- Output
- FLOPs 2.250e+20; 2.604 PFLOP/s-days
- Walkthrough
- 6 × 1.25e8 × 3e11 = 2.25e20. A 7B model reading 1,000 tokens forward costs 2 × 7e9 × 1000 = 1.4e13 FLOPs.
Limitations
- Ignores attention FLOPs that grow with sequence length, so it understates cost for long contexts.
- Counts operations, not time. Real throughput depends on hardware utilisation.
- Assumes dense models; mixture-of-experts uses active parameters.
Questions
How do I estimate training FLOPs?
Why 6 and not 2?
What is a PFLOP/s-day?
Does it include activation recomputation?
Features
- 6NT and 2NT
- PFLOP/s-days
- Runs in your browser
Related resources
Guides and videos from around the web—opens on YouTube or the publisher’s site.
TensorFlow 2.0 Complete Course
freeCodeCamp.org
Practical ML stack context next to AI utilities.
Watch on YouTubeMachine Learning for Everybody
freeCodeCamp.org
Accessible ML survey for AI calculator and metric tools.
Watch on YouTube
- Machine Learning Crash Course
Google Developers
Official ML crash course pairing with metric calculators.
- DeepLearning.AI
DeepLearning.AI
Courses and explainers for modern AI tooling.