Elamur Research⌕
☰
home / papers / 2017 / Concept-Map-Based Multi-Document Summarization using Concept Coreference Resolution and Global Importance Optimization

Concept-Map-Based Multi-Document Summarization using Concept Coreference Resolution and Global Importance Optimization

Summary

Falke, Meyer and Gurevych study concept-map-based multi-document summarization. The goal is to turn a topic-focused document collection into a small, connected labeled graph that captures the key concepts and their relations. Earlier systems had three weaknesses: they left duplicate concepts unmerged, picked important elements with only simple unsupervised heuristics, and did not scale to large inputs.

The proposed pipeline has three stages:

  1. Extraction. Propositions are obtained with Open IE, then filtered and post-processed.
  2. Concept merging. A pairwise log-linear classifier using lexical and semantic similarity features predicts which mentions corefer. A greedy partitioning search then groups them into unique concepts.
  3. Selection. A supervised ranking SVM scores concept importance, and an ILP chooses the optimal connected subgraph within the size limit.

The model is evaluated on two datasets, EDUC and WIKI. EDUC uses roughly 40 documents per topic. It beats several previous approaches on METEOR and ROUGE, and its ILP is far more efficient than earlier graph-summarization formulations. This makes structured summaries feasible for realistically large collections.

Key points

  • EDUC document sets are about 10–15 times bigger than classic DUC/TAC corpora, so scalability is a central concern. Code is released at github.com/UKPLab/ijcnlp2017-cmaps.
  • Coreference features are kept cheap because the number of mention pairs grows quadratically:
    • normalized Levenshtein distance;
    • Jaccard overlap of stemmed words;
    • embedding cosine similarity;
    • LSA- and WordNet-based measures from Semilar.
  • Replacing the coreference component with lemma matching (coref=lem) or CoreNLP chains merged across documents (coref=doc) lowered scores on both datasets.
  • Subgraph selection on EDUC:
    • The ILP of Li et al. (2016) needs about 37M variables and 2670.61 s per topic.
    • The authors' ILP needs about 22k variables and 7.31 s, or 5.61 s when solved per connected component.
    • A greedy heuristic recovered the best possible subgraph in just 35% of topics.
  • The strongest scoring features were document frequency, idf-weighted concept frequency, topic relatedness and PageRank.
  • Error analysis shows that importance estimation is the main bottleneck. On EDUC, recall falls from 60.27 at the concept step to 16.53 in the final subgraph.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{falke2017concept,
  title = {{Concept-Map-Based Multi-Document Summarization using Concept Coreference Resolution and Global Importance Optimization}},
  author = {Falke, Tobias and Meyer, Christian M. and Gurevych, Iryna},
  booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2017},
  url = {https://aclanthology.org/I17-1081/}
}

References

40 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tobias Falke and Iryna Gurevych. 2017b. Graph-DocExplore: A Framework for the Experimental Comparison of Graph-based Document Exploration Techniques. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 19–24, Copenhagen, Denmark. find
  2. Michele Banko, Michael J. Cafarella, Stephen Soderland, Matt Broadhead, and Oren Etzioni. 2007. Open Information Extraction from the Web. In Proceedings of the 20th International Joint Conference on Artifical Intelligence, pages 2670–2676, Hyderabad, India. find our card
  3. Tobias Falke and Iryna Gurevych. 2017c. Utilizing Automatic Predicate-Argument Analysis for Concept Map Mining. In Proceedings of the 12th International Conference on Computational Semantics (IWCS), Montpellier, France. find
  4. Regina Barzilay and Mirella Lapata. 2006. Aggregation via set partitioning for natural language generation. In Proceedings of the Human Language Technology Conference of the NAACL, pages 359–366, New York, NY, USA. find
  5. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA Data Mining Software: An Update. SIGKDD Explorations, 11(1):10–18. find
  6. Geoffrey Briggs, David A. Shamma, Alberto J. Cañas, Roger Carff, Jeffrey Scargle, and Joseph D. Novak. 2004. Concept Maps Applied to Mars Exploration Public Outreach. In Concept Maps: Theory, Methodology, Technology. Proceedings of the First International Conference on Concept Mapping, pages 109–11… find
  7. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 133– 142, Edmonton, Canada. find
  8. Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2014. Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3):904–911. find
  9. Martin Klein and Michael L. Nelson. 2009. Correlation of Term Count and Document Frequency for Google N-Grams. In Advances in Information Retrieval, volume 5478 of Lecture Notes in Computer Science, pages 620–627. Springer, Berlin, Heidelberg. find
  10. Ziqiang Cao, Furu Wei, Li Dong, Sujian Li, and Ming Zhou. 2016. Ranking with Recursive Neural Networks and Its Application to Multi-document Summarization. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 2153– 2159, Phoenix, AZ, USA. find
  11. Juliana H. Kowata, Davidson Cury, and Maria Claudia Silva Boeres. 2010. Concept Maps Core Elements Candidates Recognition from Text. In Concept Maps: Making Learning Meaningful. Proceedings of the 4th International Conference on Concept Mapping, pages 120–127, Vina del Mar, Chile. find
  12. Marco Carvalho, Rattikorn Hewett, and Alberto J. Cañas. 2001. Enhancing Web Searches from Concept Map-based Knowledge Models. In Proceedings of the 5th World Multi-Conference on Systemics, Cybernetics and Informatics, pages 69–73, Orlando, FL, USA. find
  13. Wei Li, Lei He, and Hai Zhuge. 2016. Abstractive News Summarization based on Event Semantic Link Network. In Proceedings of the 26th International Conference on Computational Linguistics (COLING), pages 236–246, Osaka, Japan. find
  14. Max Coltheart. 1981. The MRC psycholinguistic database. The Quarterly Journal of Experimental Psychology Section A, 33(4):497–505. find
  15. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81, Barcelona, Spain. find
  16. Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391–407. find
  17. Fei Liu, Jeffrey Flanigan, Sam Thomson, Norman Sadeh, and Noah A. Smith. 2015. Toward Abstractive Summarization Using Semantic Representations. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 107… find
  18. Michael Denkowski and Alon Lavie. 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, MD, USA. find our card
  19. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 55–60, Baltimore, MD, USA. find our card
  20. Tobias Falke and Iryna Gurevych. 2017a. Bringing Structure into Summaries: Crowdsourcing a Benchmark Corpus of Concept Maps. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2969–2979, Copenhagen, Denmark. find
  21. Gabriel Stanovsky and Ido Dagan. 2016. Creating a Large Benchmark for Open Information Extraction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2300–2305, Austin, TX, USA. find
  22. Ryan McDonald. 2007. A Study of Global Inference Algorithms in Multi-Document Summarization. In Proceedings of the 29th European conference on IR research, pages 557–564, Rome, Italy. find
  23. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26, pages 3111–3119, Lake Tahoe, NV, USA. find
  24. Antoine Tixier, Fragkiskos Malliaros, and Michaelis Vazirgiannis. 2016. A Graph Degeneracy-based Approach to Keyword Extraction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1860–1870, Austin, TX, USA. find
  25. Joseph D. Novak and D. Bob Gowin. 1984. Learning How to Learn. Cambridge University Press, Cambridge. find
  26. Alejandro Valerio and David B. Leake. 2006. Jump-Starting Concept Map Construction with Knowledge Extracted from Documents. In Proceedings of the 2nd International Conference on Concept Mapping, pages 296–303, San José, Costa Rica. find
  27. Andrew Olney, Whitney Cade, and Claire Williams. 2011. Generating Concept Map Exercises from Textbooks. In Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, pages 111–119, Portland, OR, USA. find
  28. Alejandro Valerio, David B. Leake, and Alberto J. Cañas. 2012. Using Automatically Generated Concept Maps for Document Understanding: A Human Subjects Experiment. In Proceedings of the 5th International Conference on Concept Mapping, pages 438–445, Valetta, Malta. find
  29. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank Citation Ranking: Bringing Order to the Web: Technical Report. find our card
  30. Jorge J. Villalon. 2012. Automated Generation of Concept Maps to Support Writing. PhD Thesis, University of Sydney, Australia. find
  31. Iqbal Qasim, Jin-Woo Jeong, Jee-Uk Heu, and Dong-Ho Lee. 2013. Concept map construction from text documents using affinity propagation. Journal of Information Science, 39(6):719–736. find
  32. Yinfei Yang, Forrest Bao, and Ani Nenkova. 2017. Detecting (Un)Important Content for Single-Document News Summarization. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 707–712, Valencia, Spain. find
  33. Kanagasabai Rajaraman and Ah-Hwee Tan. 2002. Knowledge discovery from texts: A Concept Frame Graph Approach. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, pages 669–671, McLean, VA, USA. find
  34. Markus Zopf, Maxime Peyrard, and Judith Eckle-Kohler. 2016. The Next Step for Multi-Document Summarization: A Heterogeneous Multi-Genre Corpus Built with a Novel Construction Approach. In Proceedings of the 26th International Conference on Computational Linguistics (COLING), pages 1535– 1545, Osaka,… find
  35. Thomas Reichherzer and David Leake. 2006. Understanding the Role of Structure in Concept Maps. In Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society, pages 2004– 2009, Vancouver, Canada. find
  36. Amal Zouaq and Roger Nkambou. 2009. Evaluating the Generation of Domain Ontologies in the Knowledge Puzzle Project. IEEE Transactions on Knowledge and Data Engineering, 21(11):1559–1572. find
  37. Philip Resnik. 1995. Using Information Content to Evaluate Semantic Similarity in a Taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada. find our card
  38. Krunoslav Zubrinic, Ines Obradovic, and Tomo Sjekavica. 2015. Implementation of method for generating concept map from unstructured text in the Croatian language. In 23rd International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pages 220–223, Split, Croatia. find
  39. Ryan Richardson and Edward A. Fox. 2005. Using concept maps as a cross-language resource discovery tool for large documents in digital libraries. In Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries, page 415, Denver, CO, USA. find
  40. Vasile Rus, Mihai Lintean, Rajendra Banjade, Nobal Niraula, and Dan Stefanescu. 2013. SEMILAR: The Semantic Similarity Toolkit. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 163–168, Sofia, Bulgaria. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/falke-2017-concept-map-based-multi-document/