Elamur Research⌕
☰
home / papers / 2014 / Hybrid Simplification using Deep Semantics and Machine Translation

Hybrid Simplification using Deep Semantics and Machine Translation

Summary

Narayan and Gardent tackle automatic sentence simplification: rewriting a complex English sentence into one or more easier sentences that keep its meaning. Earlier systems worked either on the surface string or on a syntactic parse. This one starts from a deep semantic representation, the Discourse Representation Structure (DRS) that Boxer builds for the input, converted into a graph of variables and relations. The pipeline has two parts:

  • A probabilistic model on the semantic graph handles splitting and deletion.
  • A phrase-based monolingual SMT module with a language model handles lexical/phrasal substitution and reordering.

Both are trained on the PWKP corpus of aligned English Wikipedia and Simple English Wikipedia sentences. Edit-distance analysis and human judgments are used to benchmark the system against three prior approaches: Wubben et al. (2012), Woodsend and Lapata (2011), and Zhu et al. (2010). Its output is rated the simplest of all systems, and it stays grammatical and faithful to the input. The work matters because it shows that semantics makes splitting and argument handling more principled than purely syntactic rewriting.

Key points

  • Input and training data: the input is the Boxer DRS of the complex sentence, turned into a graph. Preprocessing inverts nn relations, maps named/timex expressions to unary predicates and adds nodes for orphan words. Training uses PWKP (about 108K/114K complex/simple sentences from EWKP/SWKP).

  • Deletion constraint: subformulae attached to a predicate through core thematic roles (e.g., agent, patient) are never candidates for deletion. This prevents obligatory arguments from being dropped.

  • Human evaluation setup: 20 test sentences, each with outputs from Gold, Zhu, Woodsend, Wubben and Hybrid, giving 100 pairs. 27 raters scored them on a 0–5 scale. Significance was tested with one-way ANOVA and Tukey HSD.

  • Human evaluation results:

    CriterionHybridGoldOther systems
    Simplicity3.373.571.73–2.84
    Fluency3.553.93Wubben 3.65
    Adequacy3.503.66—

    Only Hybrid was not significantly different from Gold on simplicity.

  • Edit analysis: Hybrid's outputs are closest to the references and match them exactly most often. Its average distance from the input is 6.32 edits, against Woodsend 8.63, Zhu 7.87 and Wubben 3.33. Only 6.1% of PWKP pairs involve splitting.

Interesting ideas

  • I1. Split on shared semantic entities: the system splits where one entity takes part in two separate events. Copying that entity into each new sentence handles completion (re-creating the subject in the second sentence) with fewer errors. This could help in other generation tasks that break text into atomic facts.
  • I2. Divide the work between models: substitution and reordering go to SMT, which handles them well. Splitting and deletion, which SMT barely handles, go to a dedicated model. Splitting a task by which operations each tool handles best is a reusable design pattern.
  • I3. Use semantic roles as hard constraints: forbidding deletion of core-role arguments keeps compression from damaging meaning. The same constraint would suit summarization or text compression.
  • I4. Measure edits in both directions: reporting edit distance to the input and to the reference shows whether a system is conservative (high fluency, little simplification, like Wubben) or aggressive. It is a cheap diagnostic that helps interpret human scores.
  • I5. Group systems by significance: clustering systems that are not significantly different (ANOVA + Tukey HSD) gives an honest ranking from a small human evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

93 papers of our library cite it (of the 140884 whose bibliographies we read); 19 of them have a page here, 74 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p14-1041

@inproceedings{narayan2014hybrid,
  title = {{Hybrid Simplification using Deep Semantics and Machine Translation}},
  author = {Narayan, Shashi and Gardent, Claire},
  booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2014},
  doi = {10.3115/v1/p14-1041},
  url = {https://doi.org/10.3115/v1/p14-1041}
}

References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL), pages 597– 604. Association for Computational Linguistics. find our card
  2. Regina Barzilay and Noemie Elhadad. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of the 2003 conference on Empirical Methods in Natural Language Processing (EMNLP), pages 25–32. Association for Computational Linguistics. find our card
  3. Stefan Bott, Horacio Saggion, and Simon Mille. 2012. Text simplification tools for spanish. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC), pages 1665– 1671. find
  4. Yvonne Margaret Canning. 2002. Syntactic simplification of Text. Ph.D. thesis, University of Sunderland. find
  5. John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, and John Tait. 1999. Simplifying text for language-impaired readers. In Proceedings of 9th Conference of the European Chapter of the Association for Computational Linguistics (EACL), volume 99, pages 269–270. Citeseer. find our card
  6. Raman Chandrasekar and Bangalore Srinivas. 1997. Automatic induction of rules for text simplification. Knowledge-Based Systems, 10(3):183–190. find
  7. Raman Chandrasekar, Christine Doran, and Bangalore Srinivas. 1996. Motivations and methods for text simplification. In Proceedings of the 16th International conference on Computational linguistics (COLING), pages 1041–1044. Association for Computational Linguistics. find our card
  8. William Coster and David Kauchak. 2011. Learning to simplify sentences using wikipedia. In Proceedings of the Workshop on Monolingual Text-To-Text Generation, pages 1–9. Association for Computational Linguistics. find our card
  9. James R Curran, Stephen Clark, and Johan Bos. 2007. Linguistically motivated large-scale NLP with C&C and Boxer. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL) on Interactive Poster and Demonstration Sessions, pages 33–36. Association for Computation… find
  10. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1–38. find
  11. Mark Dras. 1999. Tree adjoining grammar and the reluctant paraphrasing of text. Ph.D. thesis, Macquarie University NSW 2109 Australia. find
  12. Katja Filippova and Michael Strube. 2008. Dependency tree based sentence compression. In Proceedings of the Fifth International Natural Language Generation Conference (INLG), pages 25–32. Association for Computational Linguistics. find
  13. Hans Kamp. 1981. A theory of truth and semantic representation. In J.A.G. Groenendijk, T.M.V. Janssen, B.J. Stokhof, and M.J.B. Stokhof, editors, Formal methods in the study of language, number pt. 1 in Mathematical Centre tracts. Mathematisch Centrum. find
  14. Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization-step one: Sentence compression. In Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI) and Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI), pages 703–710. AAAI Press. find
  15. Eric Kow and Anja Belz. 2012. LG-Eval: A Toolkit for Creating Online Language Evaluation Experiments. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC), pages 4033–4037. find
  16. Advaith Siddharthan, Ani Nenkova, and Kathleen McKeown. 2004. Syntactic simplification for improving content selection in multi-document summarization. In Proceedings of the 20th International Conference on Computational Linguistics (COL-ING), page 896. Association for Computational Linguistics. find
  17. Advaith Siddharthan. 2002. An architecture for a text simplification system. In Proceedings of the Language Engineering Conference (LEC), pages 64–71. IEEE Computer Society. find
  18. Advaith Siddharthan. 2006. Syntactic simplification and text cohesion. Research on Language and Computation, 4(1):77–109. find
  19. Advaith Siddharthan. 2010. Complex lexico-syntactic reformulation of sentences using typed dependency representations. In Proceedings of the 6th International Natural Language Generation Conference (INLG), pages 125–133. Association for Computational Linguistics. find
  20. Advaith Siddharthan. 2011. Text simplification using typed dependencies: a comparison of the robustness of different generation strategies. In Proceedings of the 13th European Workshop on Natural Language Generation (ENLG), pages 2–11. Association for Computational Linguistics. find
  21. David A Smith and Jason Eisner. 2006. Quasi-synchronous grammars: Alignment by soft projection of syntactic dependencies. In Proceedings of the HLT-NAACL Workshop on Statistical Machine Translation, pages 23–30. Association for Computational Linguistics. find our card
  22. David Vickrey and Daphne Koller. 2008. Sentence simplification for semantic role labeling. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics (ACL) and the Human Language Technology Conference (HLT), pages 344–352. find
  23. Willian Massami Watanabe, Arnaldo Candido Junior, Vinı́cius Rodriguez Uzêda, Renata Pontin de Mattos Fortes, Thiago Alexandre Salgueiro Pardo, and Sandra Maria Aluı́sio. 2009. Facilita: reading assistance for low-literacy readers. In Proceedings of the 27th ACM international conference on Design of… find
  24. Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 409–420. Association for Computational Linguistics. find our card
  25. Sander Wubben, Antal van den Bosch, and Emiel Krahmer. 2012. Sentence simplification by monolingual machine translation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL): Long Papers-Volume 1, pages 1015–1024. Association for Computational Linguistics. find our card
  26. Kenji Yamada and Kevin Knight. 2001. A syntax-based statistical translation model. In Proceedings of the 39th Annual Meeting on Association for Computational Linguistics (ACL), pages 523–530. Association for Computational Linguistics. find our card
  27. Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pages 1353–1361, Stroudsburg, PA, USA. Association for Computational Linguistics… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/narayan-2014-hybrid-simplification-deep-semantics-machine/