Elamur Research⌕
☰
home / papers / 2019 / Better Rewards Yield Better Summaries: Learning to Summarise Without References

Better Rewards Yield Better Summaries: Learning to Summarise Without References

Paper · 2019 · Florian Böhm, Yang Gao, Christian M. Meyer et al.

Summary

RL-based summarisers usually optimise ROUGE. ROUGE ranks whole systems reasonably well, but it is weak at telling a strong individual summary from a mediocre one, and it needs reference summaries. The authors replace it with a reward learned directly from human judgements. They use the ratings collected by Chaganty et al. (2018) for 2,500 summaries of 500 CNN/DailyMail news articles. They frame reward learning as either regression or preference learning and test several text encoders and neural architectures. The resulting reward model sees only the source document and the candidate summary, so no references are needed once it is trained. The learned reward agrees with human ratings significantly better than ROUGE, BLEU, METEOR, embedding-similarity metrics and a feature-based baseline. It was used to train an extractive system (NeuralTD) and an abstractive one (ExtAbsRL). In both cases human evaluators preferred the outputs over ROUGE-trained and state-of-the-art systems, even though ROUGE scores dropped. The work shows that a modest amount of human feedback can guide summarisers more effectively than large reference corpora.

Key points

  • Data: each article has five summaries: outputs from See et al.'s seq2seq and pointer models, Paulus et al.'s ml and ml+rl models, plus the reference. Five crowd workers rated each one on a −1 to 1 scale.
  • Correlation study: all metrics tested correlate poorly with humans at summary level, with G-Pre and G-Rec below .50. These metrics are ROUGE variants, BLEU-1–5, METEOR, InferSent-Cosine and BERT-Cosine.
  • Extractive result: NeuralTD trained with the learned reward received significantly higher human ratings (p = 0.0057 in the reported comparison).
  • Abstractive result:
    • In a 20-document, three-rater evaluation, ExtAbsRL with the learned reward was rated higher than with ROUGE-L.
    • Summaries with redundant sentences fell from six to two.
  • Data efficiency and availability:
    • The reward comes from about 2,500 ratings, compared with the ~287k document-reference pairs used by competing systems.
    • Code and the learned reward are available at github.com/yg211/summary-reward-no-reference.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technische Universität DarmstadtDarmstadt · Germany76 works · 35,008 citations🎓Bar-Ilan UniversityRamat Gan · Israel118 works · 21,077 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Florian Böhm, Yang Gao, Christian M. Meyer, Ori Shapira, Ido Dagan and Iryna Gurevych. 2019. Better Rewards Yield Better Summaries: Learning to Summarise Without References. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1307

@inproceedings{bohm2019better,
  title = {{Better Rewards Yield Better Summaries: Learning to Summarise Without References}},
  author = {Florian Böhm, Yang Gao, Christian M. Meyer, Ori Shapira, Ido Dagan and Iryna Gurevych},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1307},
  url = {https://doi.org/10.18653/v1/d19-1307}
}

References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yang Gao, Christian M. Meyer, and Iryna Gurevych. 2018. APRIL: interactively learning to summarise by combining active preference learning and reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4120–4130, Brussels, Belgium. find
  2. Chris Alberti, Kenton Lee, and Michael Collins. 2019. A BERT Baseline for the Natural Questions. arXiv e-prints. find
  3. Yang Gao, Christian M. Meyer, and Iryna Gurevych. 2019. Preference-based Interactive Multi-Document Summarisation. arXiv e-prints. find
  4. Yang Gao, Christian M. Meyer, Mohsen Mesgar, and Iryna Gurevych. 2019. Reward learning for efficient reinforcement learning in extractive document summarisation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 2350–2356, Macao, China. find
  5. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pa…
  6. Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, pages 169…
  7. Chris Kedzie, Kathleen R. McKeown, and Hal Daumé III. 2018. Content selection in deep learning models of summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1818–1828, Brussels, Belgium.
  8. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1746–1751, Doha, Qatar.
  9. Julia Kreutzer, Artem Sokolov, and Stefan Riezler. 2017. Bandit structured prediction for neural sequence-to-sequence learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 1503–1513, Vancouver, Canada. find
  10. Julia Kreutzer, Joshua Uyheng, and Stefan Riezler. 2018. Reliability and learnability of human bandit feedback for sequence-to-sequence reinforcement learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 1777–1788, Melbourn… find
  11. Wojciech Kryscinski, Romain Paulus, Caiming Xiong, and Richard Socher. 2018. Improving abstraction in text summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1808–1817, Brussels, Belgium. find
  12. Alon Lavie and Michael J. Denkowski. 2009. The meteor metric for automatic evaluation of machine translation. Machine Translation, 23(2-3):105–115. find
  13. Chin-Yew Lin. 2004a. Looking for a few good metrics: Rouge and its evaluation. In NTCIR Workshop. find
  14. Chin-Yew Lin. 2004b. ROUGE: A package for automatic evaluation of summaries. In ACL Workshop “Text Summarization Branches Out”.
  15. Annie Louis and Ani Nenkova. 2013. Automatically assessing machine summary content without a gold standard. Computational Linguistics, 39(2):267– 300.
  16. Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, and Yi Chang. 2018a. Document modeling with external attention for sentence extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long P… find
  17. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018b. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pape…
  18. Andrew Y. Ng and Stuart J. Russell. 2000. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 663–670, Stanford University, Stanford, CA, USA. find
  19. Jekaterina Novikova, Ondrej Dusek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2241–2252, Copenhagen, Denmark.
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, PA, USA.
  21. Ramakanth Pasunuru and Mohit Bansal. 2018. Multi-reward reinforced summarization with saliency and entailment. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 646– 653, N…
  22. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In 6th International Conference on Learning Representations, Conference Track Proceedings, Vancouver, BC, Canada.
  23. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar.
  24. Maxime Peyrard, Teresa Botschen, and Iryna Gurevych. 2017. Learning to score system summaries for better content selection evaluation. In Proceedings of the Workshop on New Frontiers in Summarization, pages 74–84, Copenhagen, Denmark. find
  25. Maxime Peyrard and Iryna Gurevych. 2018. Objective function learning to match human judgements for optimization-based summarization. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Pape… find
  26. Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana, USA. find
  27. Xingxing Zhang, Mirella Lapata, Furu Wei, and Ming Zhou. 2018. Neural latent extractive document summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 779–784, Brussels, Belgium.
  28. Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 654–663, Melbour…
  29. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100, 000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas, USA.
  30. Cody Rioux, Sadid A. Hasan, and Yllias Chali. 2014. Fear the REAPER: A system for automatic multi-document summarization with reinforcement learning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, pages 681–690, Doha, Qatar. find
  31. Andreas Rücklé, Steffen Eger, Maxime Peyrard, and Iryna Gurevych. 2018. Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations. arXiv e-prints. find
  32. Seonggi Ryang and Takeshi Abekawa. 2012. Framework of automatic text summarization using reinforcement learning. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 256–265, Jeju Island, Korea. find
  33. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 1073– 1083, Vancouver, Canada. find
  34. Simeng Sun, Ori Shapira, Ido Dagan, and Ani Nenkova. 2019. How to Compare Summarizers without Target Length? Pitfalls, Solutions and Re-Examination of the Neural Summarization Literature. In Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, pages 21–29,… find
  35. Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bohm-2019-better-rewards-yield-better-summaries/