Elamur Research⌕
☰
home / papers / 2020 / MAGPIE: A Large Corpus of Potentially Idiomatic Expressions

MAGPIE: A Large Corpus of Potentially Idiomatic Expressions

Summary

Research on automatic idiom processing has been limited by small annotated datasets. Earlier English resources contain at most a few thousand examples, cover only dozens of expressions, and often allow just one syntactic pattern. The authors address this by building MAGPIE, an English corpus of potentially idiomatic expressions (PIEs). A PIE is a phrase that can carry an idiomatic reading, whether or not it does so in a given context.

The idiom inventory comes from cross-checking three dictionaries. Candidate occurrences were pulled automatically from the British National Corpus, with a smaller set from the Parallel Meaning Bank. The candidates were then labelled by crowdworkers in a tightly controlled setup. Key elements of that setup were:

  • careful worker selection
  • detailed instructions
  • an interface that splits the judgement into small steps

The finished corpus holds over 50K sense-labelled instances, roughly ten times more than prior resources, and comes with rich metadata. The analysis shows that most expressions lean heavily toward either literal or idiomatic use. Genre also has a strong effect on that balance. The resource gives more realistic training and evaluation data for idiom disambiguation, and is freely released.

Key points

  • Idiom inventory: Idiom types were taken from Wiktionary, the Oxford Dictionary of English Idioms (ODEI) and UsingEnglish.com.
    • Every type had to appear in ODEI plus at least one of the other two dictionaries, giving 2,007 types.
    • Requiring all three dictionaries would have left only 591.
  • Candidate extraction: Candidates were extracted with the Haagsma et al. (2019) system, which reaches an F1 of almost 91% and handles inflection, passives, insertions and article changes. Run over the BNC, it produced about 200,000 hits.
  • Filtering:
    • Removing 40 noisy types (e.g. to go) left 1,967 types and cut more than 80,000 candidates.
    • Capping each type at 200 instances left 72,713 items for annotation, covering 1,781 types.
  • Annotation and size: Annotators chose among five sense labels, with crowdsourcing run on FigureEight. The final corpus exceeds 50K instances, compared with at most 4,350 instances and 65 types in earlier corpora.
  • Findings: The share of idiomatic uses ranges from above 85% to below 40% depending on genre.
    • Technical and instructional texts lean literal.
    • Argumentative, abstract writing leans idiomatic.
    • Truly balanced types are rare.
    • The corpus is available at github.com/hslh/magpie-corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

81 papers of our library cite it (of the 140673 whose bibliographies we read); 18 of them have a page here, 63 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{haagsma2020magpie,
  title = {{MAGPIE: A Large Corpus of Potentially Idiomatic Expressions}},
  author = {Haagsma, Hessel and Bos, Johan and Nissim, Malvina},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.35/}
}

References

15 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abzianidze, L., Bjerva, J., Evang, K., Haagsma, H., van Noord, R., Ludmann, P., Nguyen, D.-D., and Bos, J. (2017). The Parallel Meaning Bank: Towards a multilingual corpus of translations annotated with compositional meaning representations. In Proceedings of the 15th Conference of the European Chap… find our card
  2. Buchholz, S. and Latorre, J. (2011). Crowdsourcing preference tests, and how to detect cheating. In INTERSPEECH-2011, pages 3053–3056. find
  3. Cook, P., Fazly, A., and Stevenson, S. (2008). The VNC-Tokens dataset. In Proceedings of the LREC Workshop: Towards a shared task for Multiword Expressions, pages 19–22. find
  4. Fazly, A., Cook, P., and Stevenson, S. (2009). Unsupervised type and token identification of idiomatic expressions. Computational Linguistics, 35(1):61–103. Grégoire, N. (2009). Untangling Multiword Expressions: A study on the representation and variation of Dutch multiword expressions. Ph.D. thesi… find our card
  5. Haagsma, H., Nissim, M., and Bos, J. (2019). Casting a wide net: Robust extraction of potentially idiomatic expressions. arXiv:1911.08829. — [arXiv:1911.08829](https://arxiv.org/abs/1911.08829) find
  6. Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., and Weischedel, R. (2006). OntoNotes: The 90% solution. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 57–60, New York City, USA, June. Association for Computational Linguistics. find
  7. Kato, A., Shindo, H., and Matsumoto, Y. (2018). Construction of large-scale English verbal multiword expression annotated corpus. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (E… find
  8. Lee, D. (2000). Genres, registers, text types, domains and styles: Clarifying the concepts and navigating a path through the BNC jungle. In Proceedings of the Fourth International Conference on Teaching and Language Corpora, Language and Computers, pages 245– 292. Brill — Rodopi. find
  9. McCarthy, M. (1998). Spoken language and applied linguistics. find
  10. Cambridge University Press, Cambridge. Minugh, D. (2008). The college idiom: Idioms in the COLL corpus. ICAME Journal: Computers in English Linguistics, 32:115–138. find
  11. Moon, R. (1998). Fixed Expressions and Idioms in English: A Corpus-Based Approach. find
  12. Oxford University Press, Oxford. Simpson, R. C. and Mendis, D. (2003). A corpus-based study of idioms in academic speech. TESOL Quarterly, 37(3):419–441. find
  13. Sporleder, C. and Li, L. (2009). Unsupervised recognition of literal and non-literal use of idiomatic expressions. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 754– 762, Athens, Greece, March. Association for Computational Linguistics. find our card
  14. Sporleder, C., Li, L., Gorinski, P., and Koch, X. (2010). Idioms in context: The IDIX corpus. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta, May. European Language Resources Association (ELRA). find
  15. Language Resource References BNC Consortium. (2007). The British National Corpus. Distributed by Bodleian Libraries, University of Oxford, Version 3 (BNC XML Edition). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/haagsma-2020-magpie-large-corpus-potentially-idiomatic/