Elamur Research⌕
☰
home / papers / 2023 / Simplify: Automatic Arabic Sentence Simplification using Word Embeddings

Simplify: Automatic Arabic Sentence Simplification using Word Embeddings

Summary

The paper builds a lexical simplification system for Arabic, a language with few simplification tools and resources. The pipeline has three stages. It first finds hard words, then proposes easier alternatives, then picks one alternative in context. To find hard words, Farasa POS tags limit the check to nouns, verbs and adjectives. An SVC classifier then rates each word's difficulty. It was trained on a CEFR-labelled Arabic frequency list, enriched with Wordfreq frequencies, Farasa tags and stems. The classifier comes in two versions: a Binary model (easy/hard) and a six-level Multi-Comp model. Candidates were generated with FastText nearest neighbours and with AraBERT masked prediction. Selection used AraBERT confidence, Gensim similarity, and a rule that a replacement must be rated less complex than the original. Evaluation used BERTScore on the only known Arabic parallel simplification corpus, plus a small human survey. Multi-Comp scored better on the small and medium splits, and Binary scored better on the large one. Human judges saw a trade-off: Multi-Comp kept meaning and grammar better, while Binary simplified more. The work offers a baseline for Arabic simplification.

Key points

  • The identification data comes from the Kilgarriff et al. frequency list (8904 words with CEFR levels). After cleaning, it holds 4258 unique words with stem, POS tag, frequency and label.
  • In the SVC feature ablation, Stem/Frequency reached the best F1 (0.88). Word/Frequency scored 0.86, and adding POS tags lowered scores to 0.77–0.79.
  • The parallel corpus is from Al-Raisi et al. (2018), in three sizes: 8, 69 and 765 sentence pairs. Scoring used BERTScore with bert-base-multilingual-cased.
  • BERTScore F1 by split:
    • Large: Binary 0.867, Multi-Comp 0.862.
    • Medium: Multi-Comp ahead, 0.885 vs 0.868.
  • In the manual evaluation (20 random samples, rated 1–5 for Meaning Preservation, Grammaticality and Simplicity), Multi-Comp led on meaning (69%) and grammar (84%). Binary led on simplicity (79% of outputs simpler).

Interesting ideas

  • I1. A substitute is accepted only if its predicted difficulty is strictly lower than the original's; on a tie, the original word stays, which favours keeping meaning over making changes. — For Mova (back): During generation or the compressor's text rebuilding, any lexical swap could be gated by a lemma-difficulty score, with ties falling back to the source word so the change remains explainable.
  • I2. Difficulty was predicted better from the stem than from the inflected surface form, because different inflections otherwise look like unrelated words to the classifier. — For Mova (tokens): Frequency and difficulty attributes should be attached to lemmas from crate en rather than to word forms, keeping the lexicon smaller and more consistent.
  • I3. The masked LM sometimes ranks the original word as its top candidate, so the authors added an explicit check that drops identity predictions and re-ranks the remaining candidates by similarity. — For Mova (learn): When an LLM teacher or witness model proposes repairs, identity and trivial outputs should be filtered out before agreement is counted, so they cannot pass as confirmations.
  • I4. Nearest neighbours in FastText space often returned other forms of the same string, not true synonyms, which shows that embedding closeness is a weak signal for substitution. — For Mova (world): Synonym candidates should come from WordNet/FrameNet relations in the compiled knowledge, with embeddings used at most for ranking.
  • I5. Judging outputs separately on meaning preservation, grammaticality and simplicity revealed a trade-off that one automatic similarity score did not show. — For Mova (learn): Evaluation of generated or repaired text should report several independent axes, so a gain on one axis cannot hide losses on the others.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Topics: Semitic languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.35

@inproceedings{salaheldin2023simplify,
  title = {{Simplify: Automatic Arabic Sentence Simplification using Word Embeddings}},
  author = {SalahEldin, Yousef and Sabty, Caroline},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.35},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.35}
}

References

18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California. Association for Computational Linguistics. find our card
  2. Muhamed Al Khalil, Nizar Habash, and Hind Saddiki. 2017. Simplification of arabic masterpieces for extensive reading: A project overview. Procedia Computer Science, 117:192–198. find
  3. Fatima Al-Raisi, Weijian Lin, and Abdelwahab Bourai. 2018. A monolingual parallel corpus of arabic. Procedia computer science, 142:334–338. find
  4. Afnan A Al-Subaihin and Hend S Al-Khalifa. 2011. Albaseet: A proposed simplification authoring tool for the arabic language. In 2011 International Conference on Communications and Information Technology (ICCIT), pages 121–125. IEEE. find
  5. Suha S Al-Thanyyan and Aqil M Azmi. 2021. Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1–36. find
  6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16, page 9. find our card
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  8. Ali L Hatab, Caroline Sabty, and Slim Abdennadher. 2022. Enhancing deep learning with embedded features for arabic named entity recognition. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4904–4912. find
  9. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. Arabic word-level readability visualization for assisted text simplification. arXiv preprint arXiv:2210.10672. ↗ our card
  10. Nouran Abdelrahman Ahmed Khallaf. 2023. An Automatic Modern Standard Arabic Text Simplification System: A Corpus-Based Approach. Ph.D. thesis, University of Leeds. find
  11. Adam Kilgarriff, Frieda Charalabopoulou, Maria Gavrilidou, Janne Bondi Johannessen, Saussan Khalil, Sofie Johansson Kokkinakis, Robert Lew, Serge Sharoff, Ravikiran Vadlapudi, and Elena Volodina. 2014. Corpus-based vocabulary lists for language learners for nine languages. Language resources and evaluation, 48(1):121–163. find
  12. Philippe Laban, Tobias Schnabel, Paul Bennett, and Marti A Hearst. 2021. Keep it simple: Unsupervised simplification of multi-paragraph text. arXiv preprint arXiv:2107.03444. ↗
  13. Radim Rehurek and Petr Sojka. 2011. Gensim–python framework for vector space modelling. NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic, 3(2). find
  14. Luz Rello, Ricardo Baeza-Yates, Laura Dempere-Marco, and Horacio Saggion. 2013. Frequent words improve readability and short words improve understandability for people with dyslexia. In IFIP Conference on Human-Computer Interaction, pages 203– 219. Springer. find
  15. Horacio Rodríguez, David Farwell, Javi Farreres, Manuel Bertran, Musa Alkhalifa, M Antonia Martí, William Black, Sabri Elkateb, James Kirk, Adam Pease, et al. 2008. Arabic wordnet: Current state and future extensions. In Proceedings of The Fourth Global WordNet Conference, Szeged, Hungary, 387-405. find
  16. Matthew Shardlow. 2014. A survey of automated text simplification. International Journal of Advanced Computer Science and Applications, 4(1):58–70. find
  17. Punardeep Sikka and Vijay Mago. 2020. A survey on text simplification. arXiv preprint arXiv:2008.08612. ↗
  18. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/salaheldin-2023-simplify-automatic-arabic-sentence-simplification/