Elamur Research⌕
☰
home / papers / 2021 / MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization

MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization

Summary

Public corpora for dialogue summarization are scarce. AMI and ICSI contain just 137 and 59 meetings, and larger collections are either tied to a single niche domain (CRD3) or not released (DiDi). The authors fill this gap with MediaSum, built from broadcast interviews. NPR transcripts from the INTERVIEW dataset are matched with the overview text on each program's web page. CNN transcripts come with lists of discussed topics. Multi-topic CNN shows are split at commercial breaks, and each topic is linked to the segment sharing the most non-stop words with it. This yields about 463.6K transcript–summary pairs of open-domain, multi-party conversation, an order of magnitude more than earlier public resources. Analysis shows that summary-relevant words concentrate at the start of interviews, and radio (NPR) shows also revisit them near the end. BART-Large is the strongest of the tested baselines. Fine-tuning on MediaSum improves results on AMI, ICSI and SAMSum, which makes the corpus a valuable pre-training source for meeting and chat summarization.

Key points

  • Size and sources: MediaSum has 463,596 pairs.
    • 49,420 come from NPR, where web-page overviews serve as summaries. Descriptions over 200 words were dropped.
    • 414,176 are CNN segments derived from 269.4K transcripts covering 2000–2020. Summaries under 5 words were dropped.
  • Dialogue profile: a dialogue averages 30.0 turns, 6.5 speakers and 1,553.7 words, and a summary averages 14.4 words. NPR summaries are longer and more abstractive than CNN ones (33.6% vs. 24.9% novel words).
  • Topic segmentation: 58.6% of CNN transcripts (157.9K) had several topics. The last 50 words of each segment are ignored during matching, because hosts often preview the next topic there.
  • Baselines: on the 10K-instance test set, models scored the following ROUGE-1/2/L:
    • LEAD-3: 14.96/5.10/13.29
    • PTGen: 28.77/12.24/24.18
    • UniLM: 32.70/17.27/29.82
    • BART-Large: 35.09/18.05/31.44
  • Topics: LDA analysis finds politics is the dominant topic (26.3%), followed by international news, crime, economy and US news. Data and code are available at github.com/zcgzcgzcg1/MediaSum.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

86 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 82 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.naacl-main.474

@inproceedings{zhu2021mediasum,
  title = {{MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization}},
  author = {Zhu, Chenguang and Liu, Yang and Mei, Jie and Zeng, Michael},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.474},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.474}
}

References

21 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. We have used only the publicly available transcripts data from the media sources and adhere to their only-for-research-purpose guideline. References As media and guests may have biased views, David M Blei, Andrew Y Ng, and Michael I Jordan. the transcripts and summaries will likely contain 2003. Lat… find our card
  2. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278. find
  3. Jiaao Chen and Diyi Yang. 2020. Multi-view sequence-to-sequence models with conversational structure for abstractive dialogue summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4106–4118. find our card
  4. Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. arXiv preprint arXiv:1905.03197. find our card
  5. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. arXiv preprint arXiv:1911.12237. find
  6. Adam Janin, Don Baron, Jane Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, et al. 2003. The icsi meeting corpus. 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03), 1:I–I. find
  7. Chris Kedzie, Kathleen McKeown, and Hal Daume III. 2018. Content selection in deep learning models of summarization. arXiv preprint arXiv:1810.12343. find
  8. Kundan Krishna, Sopan Khosla, Jeffrey P Bigham, and Zachary C Lipton. 2020. Generating soap notes from doctor-patient conversations. arXiv preprint arXiv:2005.01795. — [arXiv:2005.01795](https://arxiv.org/abs/2005.01795) find
  9. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. find our card
  10. Alison Macadam. The journey from print to radio storytelling: A guide for navigating a new landscape. NPR.org. find
  11. Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. 2020. Interview: A large-scale open-source corpus of media dialog. arXiv:2004.03090. — [arXiv:2004.03090](https://arxiv.org/abs/2004.03090) find
  12. Iain McCowan, Jean Carletta, Wessel Kraaij, Simone Ashby, S Bourban, M Flynn, M Guillemot, Thomas Hain, J Kadlec, Vasilis Karaiskos, et al. 2005. The ami meeting corpus. Proceedings of the 5th International Conference on Methods and Techniques in Behavioral Research, 88:100. find
  13. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:282… find
  14. Revanth Rameshkumar and Peter Bailey. Storytelling with dialogue: A Critical Role Dungeons and Dragons Dataset. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. find
  15. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. find
  16. Gaurav Sood. 2017. CNN Transcripts 2000–2014. find
  17. Lin Yuan and Zhou Yu. 2019. Abstractive dialog summarization with semantic scaffolds. arXiv preprint arXiv:1910.00825. — [arXiv:1910.00825](https://arxiv.org/abs/1910.00825) find
  18. Chenguang Zhu, Ruochen Xu, Michael Zeng, and Xuedong Huang. 2020. A hierarchical network for abstractive meeting summarization with cross-domain pretraining. arXiv:2004.02016. find
  19. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out. find
  20. Chunyi Liu, Peng Wang, Jiang Xu, Zang Li, and Jieping Ye. 2019. Automatic dialogue summary generation for customer service. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1957– 1965. find
  21. Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. arXiv preprint arXiv:1506.08909. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2021-mediasum-large-scale-media-interview/