Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources
Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/C04-1051.pdf
Summary
The paper tackles a shortage of large, broad-domain collections of sentence-level paraphrase pairs that could be used to train statistical machine translation (SMT) style models within a single language. The authors gathered 11,162 topically and temporally clustered news clusters (177,095 articles from thousands of web sources over eight months) and compared two unsupervised ways of pairing sentences within each cluster. The first keeps pairs with a small Levenshtein distance (the L12 corpus). The second relies on the journalistic habit of opening articles with summary sentences, so it pairs the first two sentences of articles in the same cluster, after light lexical and length filtering (the F2 corpus). They evaluated both corpora by training word alignment models and measuring alignment error rate (AER), and they also hand-analysed the types of paraphrase each corpus contains. L12 proved cleaner and easier to align. F2, although noisier, preserved far more of the complex rewordings and reorderings that make paraphrase interesting. This suggests that edit-distance extraction trades linguistic richness for precision.
Key points
- Corpus sizes: L12 has about 139K sentence pairs, with a maximum Levenshtein distance of 12 and a mean distance of 5.17. F2 has about 214K pairs. In a human-rated sample of 448 F2 pairs, 26.3% were full paraphrases and 33.7% were partial paraphrases.
- F2 filters: each pair must share at least 3 words of 4 or more characters, and the shorter sentence must be at least half as long as the longer one.
- Alignment results: L12-trained alignment reached 11.58% AER on a test set extracted the same way. On F2-style test data, the two training sets performed similarly, at 13.2% (L12) and 14.7% (F2).
- Manual analysis: the authors annotated 100 pairs per corpus. In F2, phrasal alternations were over 4× more frequent and reorderings over 20× more frequent than in L12, although F2 also contained more elaborations, meaning it is less parallel.
- Future work: the authors propose a paraphrase classifier combining edit distance, clustering, discourse, sentence-length and synonymy features. They intend to apply SMT methods to paraphrase recognition and generation.
Cited by
- Universal Conceptual Cognitive Annotation (UCCA)
- SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
- Squibs: What Is a Paraphrase?
- Generating Sentences from a Continuous Space
- SentEval: An Evaluation Toolkit for Universal Sentence Representations
- Automatically Constructing a Corpus of Sentential Paraphrases
- Language-agnostic BERT Sentence Embedding
- PPDB: The Paraphrase Database
- DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
- Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions
- Learning Distributed Representations of Sentences from Unlabelled Data
- An Analysis of Natural Language Inference Benchmarks through the Lens of Negation
- Unsupervised Syntactically Controlled Paraphrase Generation with Abstract Meaning Representations
- ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation
- An efficient framework for learning sentence representations
- Generating Phrasal and Sentential Paraphrases: A Survey of Data-Driven Methods
- Monolingual Machine Translation for Paraphrase Generation
- Learning to Generate Reviews and Discovering Sentiment
- From Paraphrase Database to Compositional Paraphrase Model and Back
- Paraphrasing for Style
- Multilingual Universal Sentence Encoder for Semantic Retrieval
- ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
Cite
@inproceedings{dolan2004unsupervised,
title = {{Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources}},
author = {Dolan, Bill and Quirk, Chris and Brockett, Chris},
booktitle = {COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics},
year = {2004},
url = {https://aclanthology.org/C04-1051/}
}References
- R. Barzilay and K. R. McKeown. 2001. Extracting Paraphrases from a parallel corpus. In Proceedings of the ACL/EACL. find our card
- R. Barzilay and L. Lee. 2003. Learning to Paraphrase: an unsupervised approach using multiple-sequence alignment. In Proceedings of HLT/NAACL. find our card
- P. Brown, S. A. Della Pietra, V.J. Della Pietra and R. L. Mercer. 1993. The Mathematics of Statistical Machine Translation. Computational Linguistics, 19(2): 263-311. find
- V. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physice-Doklady, 10:707-710. find
- D. Lin and P. Pantel. 2001. DIRT - Discovery of Inference Rules from Text. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining. find
- I. D. Melamed. 2001. Empirical Methods for Exploiting Parallel Texts. MIT Press. find
- R. Mihalcea and T. Pedersen. 2003 An Evaluation Exercise for Word Alignment. In Proceedings of the Workshop on Building and Using Parallel Texts: Data Driven Machine Translation and Beyond. May 31, 2003. Edmonton, Canada. find
- F. Och and H. Ney. 2000. Improved Statistical Alignment Models. In Proceedings of the 38th Annual Meeting of the ACL, Hong Kong, China. find
- F. Och and H. Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics, 29(1):19-52. find
- Y. Shinyama, S. Sekine and K. Sudo. 2002. Automatic Paraphrase Acquisition from News Articles. In Proceedings of NAACL-HLT. find
- S. Vogel, H. Ney and C. Tillmann. 1996. HMM-Based Word Alignment in Statistical Translation. In Proceedings of the Annual Meeting of the ACL, Copenhagen, Denmark. find