Elamur Research⌕
☰
home / papers / 2014 / Automatic Keyphrase Extraction: A Survey of the State of the Art

Automatic Keyphrase Extraction: A Survey of the State of the Art

Summary

This survey asks why automatic keyphrase extraction still performs far below many core NLP tasks, despite years of work. The authors review the corpora used for evaluation, which range from short abstracts and emails to full scientific papers, meeting transcripts and chats. They identify four corpus properties that make the task harder: document length, structural consistency, topic change, and topic correlation.

They then describe the usual two-stage pipeline. First, heuristics produce candidate phrases. Second, supervised or unsupervised methods choose the real keyphrases. For supervised methods, the survey covers both how the task is framed and which features are used.

Using a running example, the authors sort system mistakes into four kinds:

  • redundancy errors,
  • infrequency errors,
  • overgeneration errors,
  • evaluation errors.

They argue that background knowledge, for example Freebase topics, types and domains, could address most of these. The paper ends with three priorities for the field:

  • adding background knowledge,
  • coping with long documents,
  • fixing evaluation so systems are not penalised for semantically equivalent answers.

Key points

  • Corpora. The survey lists 13 evaluation corpora, many of them public. Examples include Inspec (2,000 abstracts), SemEval-2010 (284 scientific papers), DUC-2001 (308 news articles), ICSI meeting transcripts and the Enron email corpus (14,659 documents).
  • Effect of length. An Inspec abstract has about 10 gold keyphrases among 34 candidates. A scientific paper has hundreds of candidates, which makes the search space much larger.
  • KEA features. KEA relies on three features: tf*idf, relative position of first occurrence ("distance"), and supervised keyphraseness (how often a phrase was a keyphrase in training data). These features work consistently well across sources.
  • Ranking versus classification. Jiang et al. (2009) replaced binary classification with pairwise ranking, and it significantly outperformed KEA.
  • Syntactic features. Ablation studies on web pages and scientific articles found that PoS-tag and suffix sequences add little once other feature types are present.

Interesting ideas

  • I1. Rank candidates against each other. When the goal is to pick the "most representative" items, a pairwise ranker lets candidates compete, whereas an independent binary classifier cannot. This framing transfers to any top-k selection task, such as term extraction or tag suggestion.
  • I2. Use a knowledge base to merge aliases. Mapping candidates to knowledge-base entities shows when two surface forms are the same concept, as with Olympics and Olympic games. Keeping only one removes duplicate outputs, which is useful for entity-heavy or morphologically rich languages like Ukrainian.
  • I3. Boost rare candidates through related frequent ones. A phrase that appears rarely can still be important if a knowledge base links it to a prominent phrase through a shared domain. This avoids the limits of fixed co-occurrence windows.
  • I4. Use missing domain links as negative evidence. If a candidate has no knowledge-base topic in the document's dominant domain, that absence is evidence against selecting it. This is a cheap filter against phrases that merely contain an important word.
  • I5. Do not punish synonyms in evaluation. Ask annotators to mark semantically equivalent gold phrases, or build scorers that detect equivalence. This gives a fairer evaluation for any generation or extraction task with many acceptable answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

71 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 69 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p14-1119

@inproceedings{hasan2014automatic,
  title = {{Automatic Keyphrase Extraction: A Survey of the State of the Art}},
  author = {Hasan, Kazi Saidul and Ng, Vincent},
  booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2014},
  doi = {10.3115/v1/p14-1119},
  url = {https://doi.org/10.3115/v1/p14-1119}
}

References

64 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ken Barker and Nadia Cornacchia. 2000. Using noun phrase heads to extract document keyphrases. In Proceedings of the 13th Biennial Conference of the Canadian Society on Computational Studies of Intelligence, pages 40–52. find
  2. Gábor Berend. 2011. Opinion expression mining by exploiting keyphrase extraction. In Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1162–1170. find
  3. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993–1022. find our card
  4. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247–1250. find
  5. Adrien Bougouin, Florian Boudin, and Béatrice Daille. 2013. Topicrank: Graph-based topic ranking for keyphrase extraction. In Proceedings of the 6th International Joint Conference on Natural Language Processing, pages 543–551. find our card
  6. Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual Web search engine. Computer Networks, 30(1–7):107–117. find our card
  7. Mo Chen, Jian-Tao Sun, Hua-Jun Zeng, and Kwok-Yan Lam. 2005. A practical system of keyphrase extraction for web pages. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pages 277–278. find
  8. Zhuoye Ding, Qi Zhang, and Xuanjing Huang. 2011. Keyphrase extraction from online news using binary integer programming. In Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 165–173. find
  9. Mark Dredze, Hanna M. Wallach, Danny Puller, and Fernando Pereira. 2008. Generating summary keywords for emails using topics. In Proceedings of the 13th International Conference on Intelligent User Interfaces, pages 199–206. find
  10. Samhaa R. El-Beltagy and Ahmed A. Rafea. 2009. KP-Miner: A keyphrase extraction system for English and Arabic documents. Information Systems, 34(1):132–144. find
  11. Samhaa R. El-Beltagy and Ahmed Rafea. 2010. KP-Miner: Participation in SemEval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 190–193. find
  12. Eibe Frank, Gordon W. Paynter, Ian H. Witten, Carl Gutwin, and Craig G. Nevill-Manning. 1999. Domain-specific keyphrase extraction. In Proceedings of 16th International Joint Conference on Artificial Intelligence, pages 668–673. find
  13. Maria Grineva, Maxim Grinev, and Dmitry Lizorkin. 2009. Extracting key terms from noisy and multitheme documents. In Proceedings of the 18th International Conference on World Wide Web, pages 661–670. find
  14. Carl Gutwin, Gordon Paynter, Ian Witten, Craig Nevill-Manning, and Eibe Frank. 1999. Improving browsing in digital libraries with keyphrase indexes. Decision Support Systems, 27:81–104. find
  15. Khaled M. Hammouda, Diego N. Matute, and Mohamed S. Kamel. 2005. CorePhrase: Keyphrase extraction for document clustering. In Proceedings of the 4th International Conference on Machine Learning and Data Mining in Pattern Recognition, pages 265–274. find
  16. Kazi Saidul Hasan and Vincent Ng. 2010. Conundrums in unsupervised keyphrase extraction: Making sense of the state-of-the-art. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 365–373. find
  17. Chong Huang, Yonghong Tian, Zhi Zhou, Charles X. Ling, and Tiejun Huang. 2006. Keyphrase extraction using semantic networks structure analysis. In Proceedings of the 6th International Conference on Data Mining, pages 275–284. find
  18. Anette Hulth and Beáta B. Megyesi. 2006. A study on automatically extracted keywords in text categorization. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pages 537–544. find
  19. Anette Hulth, Jussi Karlgren, Anna Jonsson, Henrik Boström, and Lars Asker. 2001. Automatic keyword extraction using domain knowledge. In Proceedings of the 2nd International Conference on Computational Linguistics and Intelligent Text Processing, pages 472–482. find
  20. Anette Hulth. 2003. Improved automatic keyword extraction given more linguistic knowledge. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 216– 223. find our card
  21. Anette Hulth. 2004. Enhancing linguistically oriented automatic keyword extraction. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: Short Papers, pages 17– 20. find
  22. Xin Jiang, Yunhua Hu, and Hang Li. 2009. A ranking approach to keyphrase extraction. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 756–757. find
  23. Daniel Kelleher and Saturnino Luz. 2005. Automatic hypertext keyphrase detection. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, pages 1608–1609. find
  24. Su Nam Kim and Timothy Baldwin. 2012. Extracting keywords from multi-party live chats. In Proceedings of the 26th Pacific Asia Conference on Language, Information, and Computation, pages 199– 208. find
  25. Su Nam Kim and Min-Yen Kan. 2009. Re-examining automatic keyphrase extraction approaches in scientific articles. In Proceedings of the ACL-IJCNLP Workshop on Multiword Expressions, pages 9–16. find
  26. Su Nam Kim, Timothy Baldwin, and Min-Yen Kan. 2010a. Evaluating n-gram based evaluation metrics for automatic keyphrase extraction. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 572–580. find
  27. Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2010b. SemEval-2010 Task 5: Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 21–26. find our card
  28. Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2013. Automatic keyphrase extraction from scientific articles. Language Resources and Evaluation, 47(3):723–742. find
  29. Niraj Kumar and Kannan Srinathan. 2008. Automatic keyphrase extraction from scientific documents using n-gram filtration technique. In Proceedings of the 8th ACM Symposium on Document Engineering, pages 199–208. find
  30. Feifan Liu, Deana Pennell, Fei Liu, and Yang Liu. 2009a. Unsupervised approaches for automatic keyword extraction using meeting transcripts. In Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 620–… find
  31. Zhiyuan Liu, Peng Li, Yabin Zheng, and Maosong Sun. 2009b. Clustering to find exemplar terms for keyphrase extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 257–266. find
  32. Zhiyuan Liu, Wenyi Huang, Yabin Zheng, and Maosong Sun. 2010. Automatic keyphrase extraction via topic decomposition. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 366–376. find
  33. Zhiyuan Liu, Xinxiong Chen, Yabin Zheng, and Maosong Sun. 2011. Automatic keyphrase extraction by bridging vocabulary gap. In Proceedings of the 15th Conference on Computational Natural Language Learning, pages 135–144. find
  34. Zhiyuan Liu, Chen Liang, and Maosong Sun. 2012. Topical word trigger model for keyphrase extraction. In Proceedings of the 24th International Conference on Computational Linguistics, pages 1715–1730. find
  35. Patrice Lopez and Laurent Romary. 2010. HUMB: Automatic key term extraction from scientific articles in GROBID. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 248–251. find
  36. Yutaka Matsuo and Mitsuru Ishizuka. 2004. Keyword extraction from a single document using word co-occurrence statistical information. International Journal on Artificial Intelligence Tools, 13. find
  37. Olena Medelyan, Eibe Frank, and Ian H. Witten. 2009. Human-competitive tagging using automatic keyphrase extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1318–1327. find
  38. Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing order into texts. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 404–411. find
  39. Roberto Navigli and Simone Paolo Ponzetto. 2012. BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artificial Intelligence, 193:217– 250. find
  40. David Newman, Nagendra Koilada, Jey Han Lau, and Timothy Baldwin. 2012. Bayesian text segmentation for index term identification and keyphrase extraction. In Proceedings of the 24th International Conference on Computational Linguistics, pages 2077–2092. find
  41. Thuy Dung Nguyen and Min-Yen Kan. 2007. Keyphrase extraction in scientific publications. In Proceedings of the International Conference on Asian Digital Libraries, pages 317–326. find
  42. Chau Q. Nguyen and Tuoi T. Phan. 2009. An ontology-based approach for key phrase extraction. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing: Short Papers, pag… find
  43. Paul Over. 2001. Introduction to DUC-2001: An intrinsic evaluation of generic news text summarization systems. In Proceedings of the 2001 Document Understanding Conference. find
  44. Mari-Sanna Paukkeri, Ilari T. Nieminen, Matti Pöllä, and Timo Honkela. 2008. A language-independent approach to keyphrase extraction and evaluation. In Proceedings of the 22nd International Conference on Computational Linguistics: Companion Volume: Posters, pages 83–86. find
  45. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24(5):513– 523. find
  46. Min Song, Il-Yeol Song, and Xiaohua Hu. 2003. KPSpotter: A flexible information gain-based keyphrase extraction system. In Proceedings of the 5th ACM International Workshop on Web Information and Data Management, pages 50–53. find
  47. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. YAGO: A core of semantic knowledge. In Proceedings of the 16th International World Wide Web Conference, pages 697–706. find
  48. Takashi Tomokiyo and Matthew Hurst. 2003. A language model approach to keyphrase extraction. In Proceedings of the ACL Workshop on Multiword Expressions, pages 33–40. find
  49. Peter Turney. 1999. Learning to extract keyphrases from text. National Research Council Canada, Institute for Information Technology, Technical Report ERB-1057. find
  50. Peter Turney. 2000. Learning algorithms for keyphrase extraction. Information Retrieval, 2:303–336. find
  51. Peter Turney. 2003. Coherent keyphrase extraction via web mining. In Proceedings of the 18th International Joint Conference on Artificial Intelligence, pages 434–439. find
  52. Xiaojun Wan and Jianguo Xiao. 2008a. CollabRank: Towards a collaborative approach to single-document keyphrase extraction. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 969–976. find
  53. Xiaojun Wan and Jianguo Xiao. 2008b. Single document keyphrase extraction using neighborhood knowledge. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pages 855–860. find
  54. Xiaojun Wan, Jianwu Yang, and Jianguo Xiao. 2007. Towards an iterative reinforcement approach for simultaneous document summarization and keyword extraction. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 552–559. find
  55. Chen Wang and Sujian Li. 2011. CoRankBayes: Bayesian learning to rank under the co-training framework and its application in keyphrase extraction. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pages 2241–2244. find
  56. Ian H. Witten, Gordon W. Paynter, Eibe Frank, Carl Gutwin, and Craig G. Nevill-Manning. 1999. KEA: Practical automatic keyphrase extraction. In Proceedings of the 4th ACM Conference on Digital Libraries, pages 254–255. find
  57. Yi-Fang Brook Wu, Quanzhi Li, Razvan Stefan Bot, and Xin Chen. 2005. Domain-specific keyphrase extraction. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pages 283–284. find
  58. Wen-Tau Yih, Joshua Goodman, and Vitor R. Carvalho. 2006. Finding advertising keywords on web pages. In Proceedings of the 15th International Conference on World Wide Web, pages 213–222. find
  59. Wei You, Dominique Fontaine, and Jean-Paul Barthès. 2009. Automatic keyphrase extraction with a refined candidate set. In Proceedings of the IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pages 576–579. find
  60. Torsten Zesch and Iryna Gurevych. 2009. Approximate matching for evaluating keyphrase extraction. In Proceedings of the International Conference on Recent Advances in Natural Language Processing 2009, pages 484–489. find
  61. Hongyuan Zha. 2002. Generic summarization and keyphrase extraction using mutual reinforcement principle and sentence clustering. In Proceedings of 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 113–120. find
  62. Yongzheng Zhang, Nur Zincir-Heywood, and Evangelos Milios. 2004. World Wide Web site summarization. Web Intelligence and Agent Systems, 2:39–53. find
  63. Yongzheng Zhang, Nur Zincir-Heywood, and Evangelos Milios. 2005. Narrative text classification for automatic key phrase extraction in web document corpora. In Proceedings of the 7th ACM International Workshop on Web Information and Data Management, pages 51–58. find
  64. Xin Zhao, Jing Jiang, Jing He, Yang Song, Palakorn Achanauparp, Ee-Peng Lim, and Xiaoming Li. 2011. Topical keyphrase extraction from Twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 379–388. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hasan-2014-automatic-keyphrase-extraction-survey-state/