Elamur Research⌕
☰
home / papers / 2009 / Application-driven Statistical Paraphrase Generation

Application-driven Statistical Paraphrase Generation

Summary

Zhao et al. address paraphrase generation (PG) for downstream tasks. Earlier approaches were rule-based, thesaurus-based, NLG-based or SMT-based, and each had weaknesses: low coverage, synonym-only changes, a need for deep analysis, or scarce monolingual parallel data. The authors argue that PG is not simply monolingual MT. Its goal depends on the application, only part of a sentence needs rewriting, and reference-based metrics such as BLEU fit it poorly.

Their statistical paraphrase generation (SPG) framework has three stages:

  • Preprocessing: POS tagging and dependency parsing.
  • Paraphrase planning: candidate rewrites from several paraphrase tables (phrases, patterns, collocations) are filtered by whether they serve the target application.
  • Decoding: candidates are scored by log-linearly combining three components (paraphrasing, language modelling and application-specific usability), which correspond to adequacy, fluency and usability.

The same model was applied to sentence compression, simplification and similarity computation. Human judges rated the outputs on adequacy, fluency and usability. SPG produced useful paraphrases and beat a conventional Moses-based PG baseline. This suggests that a single model can be retargeted across tasks while pooling heterogeneous resources.

Key points

  • Three sub-models. The SPG model is built from three parts: paraphrase, language and usability models. Decoding is monotone, and paraphrase patterns with POS-typed slots (e.g. take [NN_1] into consideration) handle word reordering.
  • Multiple paraphrase tables. Resources are stored in several paraphrase tables (PTs) built from different corpora with different scoring functions. Examples are PT-2 from a monolingual comparable corpus and PT-4 from bilingual parallel corpora. About 2% of units occur in more than one PT; in that case, only the highest-scoring table counts.
  • Self-paraphrase table. This table maps any word to itself with a constant score of e^−1, so the model can leave parts of a sentence unchanged.
  • Application criteria.
    • Compression keeps only target units that are shorter in bytes; usability is the byte-length difference.
    • Simplification keeps only target units that the language model rates as more frequent.
    • Similarity computation is the third application.
  • Baselines.
    • Baseline-1: Moses with PT-2, following Quirk et al.
    • Baseline-2: all PTs, but with no planning stage and no usability model.

Interesting ideas

  • I1. Filter candidates by the task goal before decoding. Discarding rewrites that cannot help the target task shrinks the search space. The same generator can then serve different goals just by changing the filter, which is a cheap way to make one rewriting system multi-purpose.
  • I2. Score usability as its own model term. Adding "does this help the application" as a weighted factor alongside adequacy and fluency makes the trade-off explicit and tunable. The idea transfers to controlled generation, such as readability or length targets.
  • I3. Use an explicit identity option. A self-paraphrase table with a fixed score lets the model decide which spans to keep. This is useful wherever only a partial rewrite is desired, for example in editing or normalization.
  • I4. Pool heterogeneous resources with per-table weights. Phrases, slot-based patterns and dependency collocations can be combined, with the best-scoring table taken on overlap. This eases data scarcity, which is relevant for lower-resource languages like Ukrainian.
  • I5. Evaluate by human judgment on three axes. Rating adequacy, fluency and usability separately, instead of relying on overlap with references, makes sense when valid outputs vary widely and references cover few of them.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

62 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 53 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zhao2009application,
  title = {{Application-driven Statistical Paraphrase Generation}},
  author = {Zhao, Shiqi and Lan, Xiang and Liu, Ting and Li, Sheng},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  doi = {10.3115/1690219.1690263},
  url = {https://doi.org/10.3115/1690219.1690263}
}

References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Lillian Lee. 2003. Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment. In Proceedings of HLT-NAACL, pages 16-23. ↗ our card
  2. Igor A. Bolshakov and Alexander Gelbukh. 2004. Synonymous Paraphrasing Using WordNet and Internet. In Proceedings of NLDB, pages 312-323. find
  3. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (Meta-) Evaluation of Machine Translation. In Proceedings of ACL Workshop on Statistical Machine Translation, pages 136-158. find our card
  4. Jean Carletta. 1996. Assessing Agreement on Classification Tasks: The Kappa Statistic. In Computational Linguistics, 22(2): 249-254. ↗ our card
  5. John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, John Tait. 1999. Simplifying Text for Language-Impaired Readers. In Proceedings of EACL, pages 269-270. find our card
  6. Trevor Cohn and Mirella Lapata. 2008. Sentence Compression Beyond Word Deletion In Proceedings of COLING, pages 137-144. find
  7. Pablo Ariel Duboue and Jennifer Chu-Carroll. 2006. Answering the Question You Wish They Had Asked: The impact of paraphrasing for Question Answering. In Proceedings of HLT-NAACL, pages 33-36. find
  8. Jesus Gimenez and Lluis Marquez. 2004. SVMTool: A general POS tagger generator based on Support Vector Machines. In Proceedings of LREC, pages 43-46. find our card
  9. Hieu Hoang and Philipp Koehn. 2008. Design of the Moses Decoder for Statistical Machine Translation. In Proceedings of ACL Workshop on Software engineering, testing, and quality assurance for NLP, pages 58-65. find
  10. Lidija Iordanskaja, Richard Kittredge, and Alain Polguère. 1991. Lexical Selection and Paraphrase in a Meaning-Text Generation Model. In Cécile L. Paris, William R. Swartout, and William C. Mann (Eds.): Natural Language Generation in Artificial Intelligence and Computational Linguistics, pages 293-312. ↗
  11. David Kauchak and Regina Barzilay. 2006. Paraphrasing for Automatic Evaluation. In Proceedings of HLT-NAACL, pages 455-462. find our card
  12. Philipp Koehn, Franz Josef Och, Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of HLT-NAACL, pages 127-133. ↗
  13. Raymond Kozlowski, Kathleen F. McCoy, and K. Vijay-Shanker. 2003. Generation of single-sentence paraphrases from predicate/argument structure using lexico-grammatical resources. In Proceedings of IWP, pages 1-8. find
  14. J. R. Landis and G. G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. In Biometrics 33(1): 159-174. ↗
  15. De-Kang Lin and Patrick Pantel. 2001. Discovery of Inference Rules for Question Answering. In Natural Language Engineering 7(4): 343-360. ↗
  16. Ryan McDonald, Kevin Lerman, and Fernando Pereira. 2006. Multilingual Dependency Parsing with a Two-Stage Discriminative Parser. In Proceedings of CoNLL. find
  17. Kathleen R. McKeown. 1979. Paraphrasing Using Given and New Information in a Question-Answer System. In Proceedings of ACL, pages 67-72. find
  18. Masaki Murata and Hitoshi Isahara. 2001. Universal Model for Paraphrasing - Using Transformation Based on a Defined Criteria. In Proceedings of NL-PRS, pages 47-54. find
  19. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of ACL, pages 160-167. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of ACL, pages 311-318. find our card
  21. Richard Power and Donia Scott. 2005. Automatic generation of large-scale paraphrases. In Proceedings of IWP, pages 73-79. find
  22. Chris Quirk, Chris Brockett, and William Dolan. 2004. Monolingual Machine Translation for Paraphrase Generation. In Proceedings of EMNLP, pages 142-149. find our card
  23. Tetsuro Takahashi, Tomoyam Iwakura, Ryu Iida, Atsushi Fujita, Kentaro Inui. 2001. KURA: A Transfer-based Lexico-structural Paraphrasing Engine. In Proceedings of NLPRS, pages 37-46. find
  24. Shiqi Zhao, Cheng Niu, Ming Zhou, Ting Liu, and Sheng Li. 2008a. Combining Multiple Resources to Improve SMT-based Paraphrasing Model. In Proceedings of ACL-08:HLT, pages 1021-1029. find
  25. Shiqi Zhao, Haifeng Wang, Ting Liu, and Sheng Li. 2008b. Pivot Approach for Extracting Paraphrase Patterns from Bilingual Corpora. In Proceedings of ACL-08:HLT, pages 780-788. find
  26. Liang Zhou, Chin-Yew Lin, Dragos Stefan Munteanu, and Eduard Hovy. 2006. ParaEval: Using Paraphrases to Evaluate Summaries Automatically. In Proceedings of HLT-NAACL, pages 447-454. find
  27. Chengqing Zong, Yujie Zhang, Kazuhide Yamamoto, Masashi Sakamoto, Satoshi Shirai. 2001. Approach to Spoken Chinese Paraphrasing Based on Feature Extraction. In Proceedings of NLPRS, pages 551-556. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2009-application-driven-statistical-paraphrase-generation/