Elamur Research
☰
home / papers / 2013 / Squibs: What Is a Paraphrase?

Squibs: What Is a Paraphrase?

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J13-3001.pdf

Summary

Paraphrase recognition matters for NLP tasks such as question answering, information extraction, textual entailment and machine translation. However, there has been no agreed definition of a paraphrase. Logic demands identical meaning, while linguists accept looser, approximate equivalence. Bhagat and Hovy adopt the broad view and call such pairs "quasi-paraphrases." Drawing on earlier theories (Mel'cuk, Harris, Honeck) and on corpus analysis, they propose an inventory of 25 lexical operations that produce quasi-paraphrases. Examples include synonym, antonym and converse substitution, voice change, nominalization, tense or modality change, semantic implication and use of external knowledge.

They test the list in two ways:

  • Coverage: they annotate phrase pairs sampled from the MTC and MSR paraphrase corpora.
  • Precision: two annotators judge sentences rewritten with each operation, built from WordNet, DIRT and Gigaword.

The inventory covered every observed phrase pair. A small number of operations account for most real paraphrases, and which ones dominate depends on the corpus. The authors argue that useful paraphrase types are application-dependent. Their taxonomy gives a map for focusing future paraphrase research.

Key points

  • Sampling: 30 random sentence-level pairs were taken from each corpus, using manual alignments by Cohn, Callison-Burch, and Lapata (2008). This yielded 210 phrase pairs for MTC and 145 for MSR. No pair needed the "unknown" label.
  • Most frequent operations: synonym substitution (37% MTC, 19% MSR) and function word variations (37% MTC, 30% MSR). External knowledge was rare in MTC (6%) but common in MSR (32%).
  • Precision test design: for each operation, two words or phrases were chosen and five Gigaword sentences per item were rewritten, giving 10 test sentences per operation. Inter-annotator agreement was κ = 0.66.
  • Precision results: Repetition/Ellipsis and Verb/Noun conversion reached 100%. Manipulator/Device substitution was lowest at 30%. Synonym substitution and external knowledge both scored 95%.
  • Scope: the paper is a definitional squib and reports no system or application results.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

41 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 36 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00166

@article{bhagat2013squibs,
  title = {{Squibs: What Is a Paraphrase?}},
  author = {Bhagat, Rahul and Hovy, Eduard},
  journal = {Computational Linguistics},
  year = {2013},
  doi = {10.1162/coli\_a\_00166},
  url = {https://doi.org/10.1162/coli_a_00166}
}

References

13 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Clark, E. V. 1992. Conventionality and contrasts: Pragmatic principles with lexical consequences. In Andrienne Lehrer and Eva Feder Kittay, editors, Frame, Fields, and Contrasts: New Essays in Semantic Lexical Organization. Lawrence Erlbaum Associates, Hillsdale, NJ, pages 171–188. find
  2. Cohn, T., C. Callison-Burch, and M. Lapata. 2008. Constructing corpora for the development and evaluation of paraphrase systems. Computational Linguistics, 34(4):597–614. find
  3. De Beaugrande, R. and W. V. Dressler. 1981. Introduction to Text Linguistics. Longman, New York, NY. find
  4. Dolan, B., C. Quirk, and C. Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In Proceedings of the Conference on Computational Linguistics (COLING), pages 350–357, Geneva. find our card
  5. Fellbaum, C. 1998. An Electronic Lexical Database. MIT Press, Cambridge, MA. find
  6. Harris, Z. 1981. Co-occurence and transformation in linguistic structure. In Henry Hiz, editor, Papers on Syntax. D. Reidel Publishing Co., Dordrecht, pages 143–210. First published in 1957. find
  7. Hirst, G. 2003. Paraphrasing paraphrased. Invited talk at the ACL International Workshop on Paraphrasing, Sapporo. find
  8. Honeck, Richard P. 1971. A study of paraphrases. Journal of Verbal Learning and Verbal Behavior, 10(4):367–381. find
  9. Huang, S., D. Graff, and G. Doddington. 2002. Multiple-translation Chinese corpus. Linguistic Data Consortium, Philadelphia, PA. find
  10. Lin, D. and P. Pantel. 2001. Dirt: Discovery of inference rules from text. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 323–328, San Francisco, CA. find
  11. Mel’cuk, I. 1996. Lexical functions: A tool for description of lexical relations in a lexicon. In Leo Wanner, editor, Lexical Functions in Lexicography and Natural Language Processing. John Benjamins Publishing Co., Philadelphia, PA, pages 37–102. find
  12. Mel’cuk, I., 2012. Semantics: From Meaning to Text. John Benjamins Publishing Co., Philadelphia, PA. find
  13. Siegal, S. and N. J. Castellan, Jr. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Columbus, OH. find