Cross Entropy Calculator

Discrete cross-entropy between two probability distributions.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter a true distribution p and a predicted distribution q over the same classes and get their cross-entropy. With a one-hot p it equals the negative log of the probability given to the right class, the standard classification loss.

How to use

1
Step 1
Enter the true distribution, for example 0, 1, 0.
2
Step 2
Enter the predicted distribution in the same order.
3
Step 3
Press Calculate.

How it works

H(p, q) = −Σ pᵢ ln qᵢ, using natural logarithms, so the unit is nats. It is smallest when q matches p. For a one-hot p only the true class contributes, which is why the example reduces to −ln 0.8.

H(p, q) = − Σ pᵢ · ln qᵢ

Worked example

A three-class prediction

Input
True: 0, 1, 0 Predicted: 0.1, 0.8, 0.1
Output
Cross-entropy 0.223144 (3 classes)
Walkthrough
Only the middle class has p = 1, so the loss is −ln 0.8 = 0.2231.

Limitations

  • Both lists need the same length.
  • Natural log only (nats), not bits.
  • A predicted probability at or near 0 for the true class gives a very large loss.

Questions

How do I calculate cross-entropy?
Enter the true and predicted distributions. The tool computes −Σ p ln q.
Cross-entropy vs log loss?
Log loss is cross-entropy for binary labels; this tool handles any number of classes.
Are the probabilities required to add up to 1?
They should, but the tool does not enforce it.
Does it use bits or nats?
Nats (natural log).

Features

  • Any number of classes
  • Natural log
  • Runs in your browser

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • Mean, Variance and Standard Deviation

    StatQuest with Josh Starmer

    Stats foundations behind many ML metrics tools.

    Watch on YouTube
  • Finding mean, median, and mode

    Khan Academy

    Baseline stats for accuracy and evaluation helpers.

    Watch on YouTube