ROUGE (metric)

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

ROUGE, or Recall-Oriented Understudy for Gisting Evaluation,[1] is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing. The metrics compare an automatically produced summary or translation against a reference or a set of references (human-produced) summary or translation.

Metrics

The following five evaluation metrics[2] are available.

  • ROUGE-N: Overlap of N-grams[3] between the system and reference summaries.
    • ROUGE-1 refers to the overlap of 1-gram (each word) between the system and reference summaries.
    • ROUGE-2 refers to the overlap of bigrams between the system and reference summaries.
  • ROUGE-L: Longest Common Subsequence (LCS)[4] based statistics. Longest common subsequence problem takes into account sentence level structure similarity naturally and identifies longest co-occurring in sequence n-grams automatically.
  • ROUGE-W: Weighted LCS-based statistics that favors consecutive LCSes .
  • ROUGE-S: Skip-bigram[5] based co-occurrence statistics. Skip-bigram is any pair of words in their sentence order.
  • ROUGE-SU: Skip-bigram plus unigram-based co-occurrence statistics.

See also

References

  1. ^ Slides of talk by Chin-Yew Lin
  2. ^ Lin, Chin-Yew. 2004. ROUGE: a Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, July 25 - 26, 2004.
  3. ^ Lin, Chin-Yew and E.H. Hovy 2003. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proceedings of 2003 Language Technology Conference (HLT-NAACL 2003), Edmonton, Canada, May 27 - June 1, 2003.
  4. ^ Lin, Chin-Yew and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004), Barcelona, Spain, July 21 - 26, 2004.
  5. ^ Lin, Chin-Yew and Franz Josef Och. 2004a. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004), Barcelona, Spain, July 21 - 26, 2004.

External links

  • ROUGE Usage Tutorial
  • Java Implementation of ROUGE
Retrieved from "https://en.wikipedia.org/w/index.php?title=ROUGE_(metric)&oldid=845513445"
This content was retrieved from Wikipedia : http://en.wikipedia.org/wiki/ROUGE_(metric)
This page is based on the copyrighted Wikipedia article "ROUGE (metric)"; it is used under the Creative Commons Attribution-ShareAlike 3.0 Unported License (CC-BY-SA). You may redistribute it, verbatim or modified, providing that you comply with the terms of the CC-BY-SA