Elamur Research⌕
☰
home / papers / 2019 / Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization

Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization

Paper · 2019 · Manling Li, Lingyu Zhang, Heng Ji et al.

Summary

Meeting transcripts consist of fragmented, loosely structured speech from several people. Because of this, neural generators trained mostly on news-style text tend to drift away from the main topics. The authors build an abstractive summarizer for recorded meetings that uses both transcripts and participant videos.

The model extends the Pointer-Generator Network (PGN) with hierarchical attention at three levels: topic segments, utterances and words. Topic segmentation, adapted from SegBot, is learned jointly with summarization, and word attention is confined to the current segment. A visual signal adds evidence of importance: Visual Focus Of Attention (VFOA), estimated from head pose and gaze. The intuition is that a speaker whom others watch is probably saying something important.

On the AMI Meeting Corpus, the system beats the extractive CoreRank and the abstractive PGN on both ROUGE and BLEU. VFOA also improves segmentation. The work shows that cheap, automatically detectable visual cues can steer abstractive summaries toward the core content of a meeting.

Key points

  • Data: AMI Meeting Corpus (remote-control design meetings, four roles, about 30 minutes each), split into 97 training, 20 development and 20 test meetings. Reference summaries run to roughly 300 words.
  • VFOA estimation: OpenFace extracts head pose and gaze as a 5-dimensional feature vector. A seven-layer network maps this to a gaze target. It was trained on annotations from 14 meetings and reaches 64.5% accuracy.
  • Summarization results: the full model (TopicSeg+VFOA) scores ROUGE-1 53.29, compared with 36.75 for PGN and 37.86 for CoreRank. It also reaches BLEU-1 40.98 versus 37.89 for PGN. Without VFOA, ROUGE-1 drops to 51.53.
  • Segmentation results: adding VFOA raises topic segmentation accuracy from 57.74% to 60.11%. Ranking utterances by received VFOA alone selects salient ones with 45.8% accuracy.
  • Significance: by t-test, the gains are significant at p ≤ 0.09, except BLEU-4 (p = 0.27). Compared with CoreRank, BLEU scores roughly double, indicating more fluent output.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rensselaer Polytechnic InstituteTroy · United States11 works · 4,917 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

🧭 Topics 2 Vision & language · 269 works Summarization · 159 works

Cite

Cite as Manling Li, Lingyu Zhang, Heng Ji and Richard J. Radke. 2019. Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1210

@inproceedings{li2019keep,
  title = {{Keep Meeting Summaries on Topic: Abstractive Multi-Modal Meeting Summarization}},
  author = {Manling Li, Lingyu Zhang, Heng Ji and Richard J. Radke},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1210},
  url = {https://doi.org/10.18653/v1/p19-1210}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tadas Baltrusaitis, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency. 2018. Openface 2.0: Facial behavior analysis toolkit. 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), pages 59–66. find
  2. Indrani Bhattacharya, Michael Foley, Christine Ku, Ni Zhang, Tongtao Zhang, Cameron Mine, Manling Li, Heng Ji, Christoph Riedl, Brooke Foucault Welles, and Richard J. Radke. 2019. The unobtrusive group interaction (UGI) corpus. In 10th ACM Multimedia Systems Conference (MMSys 2019). find
  3. Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al. 2005. The AMI meeting corpus: A pre-announcement. In International Workshop on Machine Learning for Multimodal Interaction, pages 28–… find
  4. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua… find
  5. Günes Erkan and Dragomir R Radev. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 22:457–479. find
  6. Philip N Garner, John Dines, Thomas Hain, Asmaa El Hannani, Martin Karafiat, Danil Korchagin, Mike Lincoln, Vincent Wan, and Le Zhang. 2009. Real-time ASR from meetings. In Tenth Annual Conference of the International Speech Communication Association. find
  7. Chih-Wen Goo and Yun-Nung Chen. 2018. Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts. arXiv preprint arXiv:1809.05715. — [arXiv:1809.05715](https://arxiv.org/abs/1809.05715) find
  8. Jing Li, Aixin Sun, and Shafiq Joty. 2018. Segbot: A generic neural text segmentation model with pointer network. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI). find
  9. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out. our card
  10. Hui Lin and Jeff Bilmes. 2010. Multi-document summarization via budgeted maximization of submodular functions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 912–920. find
  11. Fei Liu, Feifan Liu, and Yang Liu. 2011. A supervised framework for keyword extraction from meeting transcripts. IEEE Transactions on Audio, Speech, and Language Processing, 19(3):538–548. find
  12. Feifan Liu, Deana Pennell, Fei Liu, and Yang Liu. 2009. Unsupervised approaches for automatic keyword extraction using meeting transcripts. In Proceedings of human language technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages… find
  13. Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015. Multi-task sequence to sequence learning. arXiv preprint arXiv:1511.06114. our card
  14. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. find
  15. George A Miller. 1995. Wordnet: a lexical database for English. Communications of the ACM, 38(11):39– 41. our card
  16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics. our card
  17. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. our card
  18. Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379–389. find
  19. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1073–1083. find
  20. Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Jean-Pierre Tixier, Polykarpos Meladianos, Michalis Vazirgiannis, and Jean-Pierre Lorré. 2018. Unsupervised abstractive meeting summarization with multi-sentence compression and budgeted submodular maximization. In Proceedings of the 56th Annual Meetin… find
  21. Antoine Tixier, Fragkiskos Malliaros, and Michalis Vazirgiannis. 2016a. A graph degeneracy-based approach to keyword extraction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1860–1870. find
  22. Antoine Tixier, Konstantinos Skianis, and Michalis Vazirgiannis. 2016b. Gowvis: a web application for graph-of-words-based text visualization and summarization. Proceedings of ACL-2016 System Demonstrations, pages 151–156. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2019-keep-meeting-summaries-topic-abstractive/