ROUGE Score

Compute ROUGE-1, ROUGE-2, and ROUGE-L F1.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter a hypothesis (such as a model summary) and a reference and get ROUGE-1 and ROUGE-2 F1 (unigram and bigram overlap) and ROUGE-L F1 (longest common subsequence). ROUGE-1 precision and recall are shown too.

How to use

1
Step 1
Enter the hypothesis.
2
Step 2
Enter the reference.
3
Step 3
Press Calculate.

How it works

ROUGE-N counts overlapping n-grams between the two texts. Precision is the share of hypothesis n-grams found in the reference, and recall is the share of reference n-grams found in the hypothesis. F1 is their harmonic mean. ROUGE-L uses the longest common subsequence, so it rewards correct word order without needing adjacent words. Text is lowercased and punctuation removed.

F1 = 2PR / (P + R)

Worked example

Cat on the mat

Input
Hypothesis: the cat sat on the mat Reference: the cat is on the mat
Output
ROUGE-1 F1 0.833333; ROUGE-2 F1 0.6; ROUGE-L F1 0.833333; ROUGE-1 P/R 0.833 / 0.833
Walkthrough
Five of six words overlap (P = R = 5/6), and 3 of 5 bigrams match (the cat, on the, the mat), giving ROUGE-2 of 0.6.

Limitations

  • Single reference only.
  • No stemming or stopword options, unlike the official ROUGE package, so scores differ from published numbers.
  • ROUGE rewards word overlap, not meaning; a good paraphrase can score low.

Questions

How do I calculate ROUGE?
Enter the hypothesis and reference and press Calculate.
ROUGE-1 vs ROUGE-2 vs ROUGE-L?
ROUGE-1 counts single words, ROUGE-2 adjacent word pairs, and ROUGE-L the longest in-order common sequence.
Is ROUGE or BLEU better for summaries?
ROUGE is recall-oriented and usual for summarization; BLEU is precision-oriented and common for translation.
Does it match the official ROUGE package?
Not exactly. There is no stemming, so scores can differ.

Features

  • ROUGE-1, 2, L F1
  • Precision and recall
  • Runs in your browser

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • Finding mean, median, and mode

    Khan Academy

    Baseline stats for accuracy and evaluation helpers.

    Watch on YouTube
  • Machine Learning for Everybody

    freeCodeCamp.org

    Accessible ML survey for AI calculator and metric tools.

    Watch on YouTube