Elamur Research⌕
☰
home / papers / 2015 / Simplifying Lexical Simplification: Do We Need Simplified Corpora?

Simplifying Lexical Simplification: Do We Need Simplified Corpora?

Summary

Lexical simplification replaces difficult words with easier synonyms to help readers such as children, non-native speakers and people with language impairments. Most recent systems learn from manually simplified corpora like Simple Wikipedia, or rely on WordNet. Such resources exist for only a few languages. The authors ask whether either is actually needed. Their unsupervised method, LIGHT-LS, uses only regular text. It retrieves candidate replacements from pretrained GloVe vectors and scores them with several features: word similarity, fit with the surrounding context, information content estimated from Google Books Ngrams counts, and n-gram language model probabilities. The final choice is based on the candidate's average rank across these features. LIGHT-LS was evaluated in three ways: the Horn et al. (2014) replacement dataset, the SemEval-2012 ranking task, and human judgments. It matched a supervised system trained on aligned simplified data and beat the best SemEval-2012 system. Compared with an unsupervised baseline built on simplified corpora, it produced noticeably simpler output. This makes lexical simplification feasible for languages without simplified corpora.

Key points

  • Candidates are the 10 nearest neighbours of each content word in 200-dimensional GloVe vectors trained on Wikipedia + Gigaword 5, with morphological derivations of the target word removed. The original word is swapped out only if the top-ranked candidate carries less information content than it does, and the chosen word is re-inflected with NodeBox Linguistics.
  • Replacement task (Horn et al., 2014 dataset):
    • LIGHT-LS reached 71.0 precision, 68.2 accuracy and changed 96.0% of target words.
    • The supervised Horn et al. (2014) system scored 76.1 / 66.3 / 86.3.
    • Biran et al. (2011) had 71.4 precision but only 3.4 accuracy, because it changed just 5.2% of targets.
  • SemEval-2012 Task 1 ranking (Cohen's kappa):
    • LIGHT-LS scored 0.540, significantly above the best participant, Jauhar and Specia (2012), at 0.496.
    • The frequency baseline reached 0.471 and the random baseline 0.013.
  • Human evaluation of simplicity:
    • LIGHT-LS output was rated 3.76, compared with 3.36 for the original sentences and 3.24 for Biran et al.
    • Grammaticality stayed close to Biran et al. (4.60 vs 4.63).
    • LIGHT-LS altered 68.6% of sentences, while Biran et al. altered 17.5%.
  • When only sentences that were actually changed are scored, LIGHT-LS also preserves meaning better (3.75 vs 3.57). Most of its meaning errors come from embeddings mixing up related words, or even antonyms, with true synonyms (e.g., "warm" → "cool").

Interesting ideas

  • I1. Combine heterogeneous features by averaging ranks instead of learning weights. Similarity scores, information-content differences and LM probabilities sit on different scales. Ranking candidates per feature and averaging the ranks merges them without training data, which suits low-resource settings.
  • I2. Fold the "should we simplify?" decision into the replacement step. A separate complex-word classifier is unnecessary if a substitution happens only when it lowers information content. This frequency-based threshold transfers easily to any language with a large raw corpus.
  • I3. Disambiguate senses with context vectors. Averaging a candidate's similarity to nearby content words pushes type-level embeddings toward the intended sense. This is a cheap trick for any substitution or paraphrasing task where polysemy is a concern.
  • I4. Adapt features when the evaluation format would give one option an unfair edge. In the SemEval ranking task the original word is itself a candidate, so the authors dropped the similarity feature and used plain information content instead. This reminds us to check whether features trivially favour one option under a given evaluation setup.
  • I5. Score humans only on outputs the system actually changed. A conservative system can look good on meaning preservation by doing nothing. Reporting the change rate and evaluating changed sentences separately makes comparisons fairer for any rewriting or normalisation system.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

98 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 92 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-2011

@inproceedings{glavas2015simplifying,
  title = {{Simplifying Lexical Simplification: Do We Need Simplified Corpora?}},
  author = {Glavaš, Goran and Štajner, Sanja},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-2011},
  url = {https://doi.org/10.3115/v1/p15-2011}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Susana Bautista, Pablo Gervás, and R. Ignacio Madrid. 2009. Feasibility analysis for semi-automatic conversion of text to improve readability. In Proceedings of the Second International Conference on Information and Communication Technology and Accessibility (ICTA), pages 33–40. find
  2. Or Biran, Samuel Brody, and Noémie Elhadad. 2011. Putting it simply: A context-aware approach to lexical simplification. In Proceedings of the ACL-HLT 2011, pages 496–501. ACL. ↗ our card
  3. John Carroll, Guido Minnen, Yvonne Canning, Siobhan Devlin, and John Tait. 1998. Practical simplification of english newspaper text to assist aphasic readers. In Proceedings of AAAI-98 Workshop on Integrating Artificial Intelligence and Assistive Technology, pages 7–10. find
  4. Jan De Belder and Marie-Francine Moens. 2010. Text simplification for children. In Proceedings of the SI-GIR Workshop on Accessible Search Systems, pages 19–26. find
  5. Siobhan Devlin and John Tait. 1998. The use of a psycholinguistic database in the simplification of text for aphasic readers. Linguistic Databases, pages 161–173. find
  6. Siobhan Devlin and Gary Unthank. 2006. Helping aphasic people process online information. In Proceedings of the 8th International ACM SIGAC-CESS Conference on Computers and Accessibility (ASSETS), pages 225–226. ACM. find
  7. Christiane Fellbaum. 1998. WordNet. Wiley Online Library. find
  8. Lijun Feng. 2009. Automatic readability assessment for people with intellectual disabilities. In ACM SIGACCESS Accessibility and Computing, number 93, pages 84–91. ACM. ↗
  9. Goran Glavaš and Sanja Štajner. 2013. Event-centered simplification of news stories. In Proceedings of the Student Workshop held in conjunction with RANLP, pages 71–78. find
  10. Colby Horn, Cathryn Manduca, and David Kauchak. 2014. Learning a lexical simplifier using wikipedia. In Proceedings of ACL 2014 (Short Papers), pages 458–463. find
  11. Sujay Kumar Jauhar and Lucia Specia. 2012. UOW-SHEF: SimpLex – lexical simplicity ranking based on contextual and psycholinguistic features. In Proceedings of the SemEval-2012, pages 477–481. ACL. find
  12. Juan Martos, Sandra Freire, Ana González, David Gil, and Maria Sebastian. 2012. D2.1: Functional requirements specifications and user preference survey. Technical report, FIRST project. find
  13. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, and Jon Orwant. 2011. Quantitative analysis of culture using millions of digitized books. Science, 331(6014):176–182. ↗
  14. Adam Pauls and Dan Klein. 2011. Faster and smaller n-gram language models. In Proceedings of ACL-HLT 2011, pages 258–267. ACL. find
  15. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of EMNLP 2014, pages 1532–1543. find
  16. Sarah E. Petersen and Mari Ostendorf. 2007. Text simplification for language learners: A corpus analysis. In Proceedings of Workshop on Speech and Language Technology for Education (SLaTE). find
  17. Luz Rello. 2012. DysWebxia: A Model to Improve Accessibility of the Textual Web for Dyslexic Users. In ACM SIGACCESS Accessibility and Computing., number 102, pages 41–44. ACM, New York, NY, USA, January. find
  18. Horacio Saggion, Sanja Štajner, Stefan Bott, Simon Mille, Luz Rello, and Biljana Drndarevic. 2015. Making it simplext: Implementation and evaluation of a text simplification system for spanish. ACM Transactions on Accessible Computing, 6(4):14. find
  19. Matthew Shardlow. 2014. Out in the open: Finding and categorising errors in the lexical simplification pipeline. In Proceedings of LREC 2014, pages 1583–1590. find
  20. Lucia Specia, Sujay Kumar Jauhar, and Rada Mihalcea. 2012. SemEval-2012 Task 1: English lexical simplification. In Proceedings of the SemEval 2012, pages 347–355. ACL. find our card
  21. Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of EMNLP 2011, pages 409–420. ACL. find our card
  22. Sander Wubben, Antal Van Den Bosch, and Emiel Krahmer. 2012. Sentence simplification by monolingual machine translation. In Proceedings of ACL 2012 (Long Papers), pages 1015–1024. ACL. find our card
  23. Mark Yatskar, Bo Pang, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2010. For the sake of simplicity: unsupervised extraction of lexical simplifications from Wikipedia. In Proceedings of NAACL 2010, pages 365–368. ACL. ↗
  24. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of COLING 2000, pages 947–953. ACL. ↗
  25. Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of the COLING 2010, pages 1353–1361. ACL. find our card

↑ back to top

This page in Markdown →