Elamur Research⌕
☰
home / papers / 2014 / Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut

Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut

Summary

The paper addresses the automatic identification of multiword expressions (MWEs) in running English text. It covers the full range of MWE types, from proper names and compounds to phrasal verbs, support-verb constructions and fixed phrases. Earlier work typically targeted one MWE subclass at a time and handled only contiguous sequences. The authors start from a corpus of web reviews fully annotated for MWEs, with discontinuous ("gappy") expressions and a strong/weak strength distinction. They restrict gaps with two linguistic constraints: projectivity and no nested gaps. Under these constraints, the segmentation can be encoded as an extended BIO-style chunking tag set, so efficient sequence-tagging inference becomes possible. On top of this encoding they train a feature-rich discriminative first-order sequence tagger. Its features draw on WordNet and other multiword lexicons as well as unsupervised word clusters. They also propose a matching evaluation measure. The tagger reaches close to 60% F1 and clearly outperforms segmentation by lexicon lookup alone. This makes it the first supervised, general-purpose MWE identifier, and both data and code are released openly.

Key points

  • The data is the MWE-annotated portion of the English Web Treebank (WTB) reviews: 723 documents, 3,812 sentences and about 55,000 tokens, with 3,483 MWE instances covering roughly 15% of tokens.
  • The representation allows gappy MWEs under projectivity and no-nesting constraints. It is formalized as an extended CFG and distinguishes strong (idiomatic) MWEs from weak (collocational) ones.
  • The model is a discriminative structured tagger whose only first-order feature is the previous label. Its features include lexical, POS, lemma and WordNet/multiword-lexicon features, plus word clusters.
  • MWE identification reaches nearly 60% F1, far above heuristic lexicon lookup.
  • The corpus and software are available at ark.cs.cmu.edu/LexSem (TACL 2, 2014).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cited by

63 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 54 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00176

@article{schneider2014discriminative,
  title = {{Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut}},
  author = {Schneider, Nathan and Danchik, Emily and Dyer, Chris and Smith, Noah A.},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2014},
  doi = {10.1162/tacl\_a\_00176},
  url = {https://doi.org/10.1162/tacl_a_00176}
}

References

61 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne Abeillé, Lionel Clément, and François Toussenel. 2003. Building a treebank for French. In Anne Abeillé and Nancy Ide, editors, Treebanks, volume 20 of Text, Speech and Language Technology, pages 165–187. find
  2. Kluwer Academic Publishers, Dordrecht, The Netherlands. Timothy Baldwin. 2005. Looking for prepositional verbs in corpus data. In Proc. of the Second ACL-SIGSEM Workshop on the Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications, pages 115–126… find
  3. Colchester, UK. Timothy Baldwin. 2008. A resource for evaluating the deep lexical acquisition of English verb-particle constructions. In Proc. of MWE, pages 1–2. find
  4. Marrakech, Morocco. Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, Second Edition. CRC Press, Taylor and Francis Group, Boca Raton, Florida, USA. Eduard Bejček, Pavel Straňák, and Pavel Pecina.… find
  5. Atlanta, Georgia, USA. Gábor Berend. 2011. Opinion expression mining by exploiting keyphrase extraction. In Proc. of 5th International Joint Conference on Natural Language Processing, pages 1162–1170. find
  6. Chiang Mai, Thailand. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012a. English Web Treebank. Technical Report LDC2012T13, Linguistic Data Consortium, Philadelphia, Pennsylvania, USA. find
  7. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012b. English Web Treebank. Technical Report LDC2012T13, Linguistic Data Consortium, Philadelphia, Pennsylvania, USA. find
  8. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. find
  9. O’Reilly Media, Inc., Sebastopol, California, USA. Phil Blunsom and Timothy Baldwin. 2006. Multilingual deep lexical acquisition for HPSGs via supertagging. In Proc. of EMNLP, pages 164–171. Sydney, Australia. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C… find
  10. Marine Carpuat and Mona Diab. 2010. Task-based evaluation of multiword expressions: a pilot study in statistical machine translation. In Proc. of NAACL-HLT, pages 242–245. find
  11. Los Angeles, California, USA. Kenneth Church, William Gale, Patrick Hanks, and Donald Hindle. 1991. Using statistics in lexical analysis. In Uri Zernik, editor, Lexical acquisition: exploiting on-line resources to build a lexicon, pages 115–164. find
  12. Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA. Massimiliano Ciaramita and Yasemin Altun. 2006. Broad-coverage sense disambiguation and information extraction with a supersense sequence tagger. In Proc. of EMNLP, pages 594–602. find
  13. Sydney, Australia. Michael Collins. 2002. Discriminative training methods for Hidden Markov Models: theory and experiments with perceptron algorithms. In Proc. of EMNLP, pages 1–8. find our card
  14. Philadelphia, Pennsylvania, USA. Matthieu Constant and Anthony Sigogne. 2011. MWU-aware part-of-speech tagging with a CRF model and lexical resources. In Proc. of the Workshop on Multiword Expressions: from Parsing and Generation to the Real World, pages 49–56. find
  15. Portland, Oregon, USA. Matthieu Constant, Anthony Sigogne, and Patrick Watrin. 2012. Discriminative strategies to integrate multiword expression recognition and parsing. In Proc. of ACL, pages 204–212. find
  16. Jeju Island, Korea. Paul Cook, Afsaneh Fazly, and Suzanne Stevenson. 2008. The VNC-Tokens dataset. In Proc. of MWE, pages 19–22. find
  17. Marrakech, Morocco. Hal Daumé, III. 2006. Practical structured learning techniques for natural language processing. Ph.D. dissertation, University of Southern California, Los Angeles, California, USA. URL http://hal3.name/docs/ daume06thesis.pdf . Mona Diab and Pravin Bhutada. 2009. Verb noun constr… ↗
  18. Suntec, Singapore. Nick C. Ellis, Rita Simpson-Vlach, and Carson Maynard. 2008. Formulaic language in native and second language speakers: psycholinguistics, corpus linguistics, and TESOL. TESOL Quarterly, 42(3):375–396. Christiane Fellbaum, editor. 1998. WordNet: an electronic lexical database. find
  19. MIT Press, Cambridge, Massachusetts, USA. Charles J. Fillmore, Paul Kay, and Mary Catherine O’Connor. 1988. Regularity and idiomaticity in grammatical constructions: the case of ‘let alone’. Language, 64(3):501–538. find
  20. Yoav Freund and Robert E. Schapire. 1999. Large margin classification using the perceptron algorithm. Machine Learning, 37(3):277–296. find
  21. Mahmoud Ghoneim and Mona Diab. 2013. Multiword expressions in the context of statistical machine translation. In Proc. of IJCNLP, pages 1181–1187. find
  22. Nagoya, Japan. Kevin Gimpel and Noah A. Smith. 2011. Generative models of monolingual and bilingual gappy patterns. In Proc. of WMT, pages 512–522. find
  23. Edinburgh, Scotland, UK. Adele E. Goldberg. 1995. Constructions: a construction grammar approach to argument structure. University of Chicago Press, Chicago, Illinois, USA. find
  24. Adele E. Goldberg. 2006. Constructions at work: the nature of generalization in language. find
  25. Oxford University Press, Oxford, UK. Edouard Grave, Guillaume Obozinski, and Francis Bach. 2013. Hidden Markov tree models for semantic class induction. In Proc. of CoNLL, pages 94–103. find
  26. Sofia, Bulgaria. Spence Green, Marie-Catherine de Marneffe, John Bauer, and Christopher D. Manning. 2011. Multiword expression identification with tree substitution grammars: a parsing tour de force with French. In Proc. of EMNLP, pages 725–735. find
  27. Edinburgh, Scotland, UK. Spence Green, Marie-Catherine de Marneffe, and Christopher D. Manning. 2012. Parsing models for identifying multiword expressions. Computational Linguistics, 39(1):195–227. Jan Hajič, Eva Hajičová, Jarmila Panevová, Petr Sgall, Silvie Cinková, Eva Fučíková, Marie Mikulová… find
  28. IGI Global, Hershey, Pennsylvania, USA. Chikara Hashimoto and Daisuke Kawahara. 2008. Construction of an idiom corpus and its application to idiom identification based on WSD incorporating idiom-specific features. In Proc. of EMNLP, pages 992–1001. find
  29. Honolulu, Hawaii, USA. Terry Koo, Xavier Carreras, and Michael Collins. 2008. Simple semi-supervised dependency parsing. In Proc. of ACL-08: HLT, pages 595–603. find our card
  30. Columbus, Ohio. Koenraad Kuiper, Heather McCann, Heidi Quinn, Therese Aitchison, and Kees van der Veer. 2003. SAID. Technical Report LDC2003T10, Linguistic Data Consortium, Philadelphia, Pennsylvania, USA. URL http://www.ldc.upenn.edu/Catalog/catalogEntry.jsp?catalogId=LDC2003T10 . John D. Lafferty,… ↗
  31. Percy Liang. 2005. Semi-supervised learning for natural language. Master’s thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA. URL http://people.csail.mit.edu/ pliang/papers/meng-thesis.pdf . Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building… ↗
  32. I. Dan Melamed. 1997. Automatic discovery of non-compositional compounds in parallel data. In Proc. of EMNLP, pages 97–108. find
  33. Providence, Rhode Island, USA. George A. Miller, Claudia Leacock, Randee Tengi, and Ross T. Bunker. 1993. A semantic concordance. In Proc. of HLT, pages 303–308. find
  34. Plainsboro, New Jersey, USA. Scott Miller, Jethran Guinness, and Alex Zamanian. 2004. Name tagging with word clusters and discriminative training. In Proc. of HLT-NAACL, pages 337–342. find our card
  35. Boston, Massachusetts, USA. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A. Smith. 2012. Recall-oriented learning of named entities in Arabic Wikipedia. In Proc. of EACL, pages 162–173. find
  36. Avignon, France. Begona Villada Moirón and Jörg Tiedemann. 2006. Identifying idiomatic expressions using automatic word-alignment. In Proc. of the EACL 2006 Workshop on Multi-word Expressions in a Multilingual Context, pages 33–40. find
  37. Trento, Italy. Rosamund Moon. 1998. Fixed expressions and idioms in English: a corpus-based approach. Oxford Studies in Lexicography and Lexicology. find
  38. Clarendon Press, Oxford, UK. David Newman, Nagendra Koilada, Jey Han Lau, and Timothy Baldwin. 2012. Bayesian text segmentation for index term identification and keyphrase extraction. In Proc. of COLING 2012, pages 2077–2092. find
  39. Mumbai, India. Geoffrey Nunberg, Ivan A. Sag, and Thomas Wasow. 1994. Idioms. Language, 70(3):491–538. find
  40. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. 2013. Improved part-of-speech tagging for online conversational text with word clusters. In Proc. of NAACL-HLT, pages 380–390. find our card
  41. Atlanta, Georgia, USA. Gerhard Paaß and Frank Reichartz. 2009. Exploiting semantic constraints for estimating supersenses with CRFs. In Proc. of the Ninth SIAM International Conference on Data Mining, pages 485–496. find
  42. Sparks, Nevada, USA. Pavel Pecina. 2010. Lexical association measures and collocation extraction. Language Resources and Evaluation, 44(1):137–158. find
  43. Carlos Ramisch. 2012. A generic and open framework for multiword expressions treatment: from acquisition to applications. Ph.D. dissertation, University of Grenoble and Federal University of Rio Grande do Sul, Grenoble, France. URL http://www.inf.ufrgs.br/~ceramisch/ download_files/thesis-getalp.pdf… ↗
  44. Jeju Island, Korea. Carlos Ramisch, Aline Villavicencio, and Christian Boitet. 2010. mwetoolkit: a framework for multiword expression identification. In Proc. of LREC, pages 662–669. find
  45. Valletta, Malta. Lance A. Ramshaw and Mitchell P. Marcus. 1995. Text chunking using transformation-based learning. In Proc. of the Third ACL Workshop on Very Large Corpora, pages 82–94. find
  46. Cambridge, Massachusetts, USA. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proc. of CoNLL, pages 147–155. find our card
  47. Boulder, Colorado, USA. Marta Recasens and Eduard Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(04):485–510. find
  48. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named entity recognition in tweets: an experimental study. In Proc. of EMNLP, pages 1524–1534. find
  49. Edinburgh, Scotland, UK. Ivan Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002. Multiword expressions: a pain in the neck for NLP. In Alexander Gelbukh, editor, Computational Linguistics and Intelligent Text Processing, volume 2276 of Lecture Notes in Computer Science, pag… find our card
  50. Springer, Berlin, Germany. Nathan Schneider, Spencer Onuffer, Nora Kazour, Emily Danchik, Michael T. Mordowanec, Henrietta Conrad, and Noah A. Smith. 2014. Comprehensive annotation of multiword expressions in a social web corpus. In Proc. of LREC. find our card
  51. Reykjavík, Iceland. Yutaro Shigeto, Ai Azuma, Sorami Hisamoto, Shuhei Kondo, Tomoya Kouse, Keisuke Sakaguchi, Akifumi Yoshimoto, Frances Yung, and Yuji Matsumoto. 2013. Construction of English MWE dictionary and its application to POS tagging. In Proc. of the 9th Workshop on Multiword Expressions, p… find our card
  52. Atlanta, Georgia, USA. James W. Thatcher. 1967. Characterizing derivation trees of context-free grammars through a generalization of finite automata theory. Journal of Computer and System Sciences, 1(4):317–322. find
  53. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proc. of HLT-NAACL, pages 173–180. find
  54. Edmonton, Alberta, Canada. Yulia Tsvetkov and Shuly Wintner. 2010. Extraction of multi-word expressions from small parallel corpora. In Coling 2010: Posters, pages 1256–1264. find
  55. Beijing, China. Yulia Tsvetkov and Shuly Wintner. 2011. Identification of multi-word expressions by combining multiple linguistic information sources. In Proc. of EMNLP, pages 836–845. find
  56. Edinburgh, Scotland, UK. Yuancheng Tu and Dan Roth. 2011. Learning English light verb constructions: contextual or statistical. In Proc. of the Workshop on Multiword Expressions: from Parsing and Generation to the Real World, pages 31–39. find
  57. Portland, Oregon, USA. Yuancheng Tu and Dan Roth. 2012. Sorting out the most confusing English phrasal verbs. In Proc. of *SEM, pages 65–69. find
  58. Montréal, Quebec, Canada. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proc. of ACL, pages 384–394. Uppsala, Sweden. Martin Čmejrek, Jan Cuřín, Jan Hajič, and Jiří Havelka. 2005. Prague Czech-English… find our card
  59. Budapest, Hungary. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proc. of MUC-6, pages 45–52. find our card
  60. Columbia, Maryland, USA. Veronika Vincze. 2012. Light verb constructions in the SzegedParalellFX English-Hungarian parallel corpus. In Proc. of LREC. find
  61. Istanbul, Turkey. Veronika Vincze, István Nagy T., and János Zsibrita. 2013. Learning to detect English and Hungarian light verb constructions. ACM Transactions on Speech and Language Processing, 10(2):6:1–6:25. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/schneider-2014-discriminative-lexical-semantic-segmentation-gaps/