Cross Entropy Calculator
Discrete cross-entropy between two probability distributions.
About this tool
Enter a true distribution p and a predicted distribution q over the same classes and get their cross-entropy. With a one-hot p it equals the negative log of the probability given to the right class, the standard classification loss.
How to use
How it works
H(p, q) = −Σ pᵢ ln qᵢ, using natural logarithms, so the unit is nats. It is smallest when q matches p. For a one-hot p only the true class contributes, which is why the example reduces to −ln 0.8.
H(p, q) = − Σ pᵢ · ln qᵢ
Worked example
A three-class prediction
- Input
- True: 0, 1, 0 Predicted: 0.1, 0.8, 0.1
- Output
- Cross-entropy 0.223144 (3 classes)
- Walkthrough
- Only the middle class has p = 1, so the loss is −ln 0.8 = 0.2231.
Limitations
- Both lists need the same length.
- Natural log only (nats), not bits.
- A predicted probability at or near 0 for the true class gives a very large loss.
Questions
How do I calculate cross-entropy?
Cross-entropy vs log loss?
Are the probabilities required to add up to 1?
Does it use bits or nats?
Features
- Any number of classes
- Natural log
- Runs in your browser
Related resources
Guides and videos from around the web—opens on YouTube or the publisher’s site.
Mean, Variance and Standard Deviation
StatQuest with Josh Starmer
Stats foundations behind many ML metrics tools.
Watch on YouTubeFinding mean, median, and mode
Khan Academy
Baseline stats for accuracy and evaluation helpers.
Watch on YouTube
- Machine Learning Crash Course
Google Developers
Official ML crash course pairing with metric calculators.
- Metrics and scoring
scikit-learn
Precision, recall, F1 definitions used by AI metric tools.