FLOPs Calculator

Estimate forward or training FLOPs from params × tokens.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter the number of parameters and the number of tokens, choose training or forward, and get the total floating-point operations. The rule of thumb is 6 × N × T for training and 2 × N × T for a forward pass, and the tool also converts the total to petaFLOP/s-days.

How to use

1
Step 1
Enter parameters and tokens (plain numbers, such as 125000000).
2
Step 2
Choose training or forward.
3
Step 3
Press Calculate.

How it works

Each token costs about 2 FLOPs per parameter in the forward pass and about twice that again for the backward pass, giving 6NT for training. One petaFLOP/s-day is 10¹⁵ × 86,400 FLOPs. The headline figure uses a K, M or B suffix below 10¹² FLOPs and scientific notation (for example 2.250e+20) above that. The table shows both forms.

training ≈ 6·N·T      forward ≈ 2·N·T

Worked example

A 125M model on 300B tokens

Input
N = 125,000,000; T = 300,000,000,000; training
Output
FLOPs 2.250e+20; 2.604 PFLOP/s-days
Walkthrough
6 × 1.25e8 × 3e11 = 2.25e20. A 7B model reading 1,000 tokens forward costs 2 × 7e9 × 1000 = 1.4e13 FLOPs.

Limitations

  • Ignores attention FLOPs that grow with sequence length, so it understates cost for long contexts.
  • Counts operations, not time. Real throughput depends on hardware utilisation.
  • Assumes dense models; mixture-of-experts uses active parameters.

Questions

How do I estimate training FLOPs?
Multiply 6 by the parameter count and the number of training tokens.
Why 6 and not 2?
The forward pass costs about 2 FLOPs per parameter per token and the backward pass about 4.
What is a PFLOP/s-day?
One petaFLOP per second sustained for a day, or 8.64e19 FLOPs.
Does it include activation recomputation?
No. Gradient checkpointing adds roughly another forward pass.

Features

  • 6NT and 2NT
  • PFLOP/s-days
  • Runs in your browser

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • TensorFlow 2.0 Complete Course

    freeCodeCamp.org

    Practical ML stack context next to AI utilities.

    Watch on YouTube
  • Machine Learning for Everybody

    freeCodeCamp.org

    Accessible ML survey for AI calculator and metric tools.

    Watch on YouTube