Elamur Research⌕
☰
home / papers / 2020 / Lexical Induction of Morphological and Orthographic Forms for Low-Resourced Languages

Lexical Induction of Morphological and Orthographic Forms for Low-Resourced Languages

Paper · 2020 · Taha Tobaili

Summary

Lebanese Arabizi is dialectal Arabic typed informally in Latin letters. It has rich morphology and no agreed spelling, so a single word can appear in a very large number of written forms. The available corpus, about 53MB, is too small to train models like BERT or GPT. Tobaili therefore turns the usual approach around. Instead of reducing variants to one canonical form, the method collects as many attested forms of each seed word as it can from raw text.

Spellings of consonant phonemes are first collapsed: ch/sh becomes $, gh becomes 8, kh becomes 5, and repeated letters are reduced to one. Each seed word's consonant skeleton is then matched against corpus words, with only certain affix letters allowed. Cosine similarity from word embeddings trained on the same corpus gives each candidate a sentiment score and settles cases where a form matched both a positive and a negative seed.

Starting from about 1.7K sentiment words, the method produced SenZi-Large, a lexicon of 175K forms. It is 7 times larger than the earlier SenZi, raises coverage on annotated tweets by 13%, and costs roughly 3% F1. The work shows that useful lexical resources can be built for low-resource languages without a standard spelling.

Key points

  • Data reused from Tobaili et al. (2019):
    • a seed lexicon of 2K words (600 positive, 1.4K negative);
    • a corpus of 1M Facebook comments;
    • a test set of 1.6K Arabizi tweets labelled for sentiment.
  • Previous expansion: the earlier SenZi lexicon (24.6K words, about 12 forms per word) was built purely from embedding nearest neighbours. The new rule-based matching yields 175K forms.
  • Normalisation scope: text is normalised only to decide whether two words match. The retrieved word is stored in its original spelling.
  • Manual step: seed words that pulled in irrelevant matches were filtered by hand, which is why the final seed list has 1.7K words. The author names this as the method's main limitation.
  • Error analysis (24 tweets drawn from a random sample of 100):
    • Missed sentiment words: 46% spelling variants, 37% code-switched English or French, 17% other.
    • Wrongly flagged words: 42% false neighbours (for example, "expert" matched "piece of information"), 58% other, such as sarcasm, multi-word expressions and polysemy.

Interesting ideas

  • I1. Expand the lexicon to cover attested variants instead of normalising the text. When no form is "correct", gathering every form seen in a corpus avoids forcing a standard that does not exist.
  • I2. Normalise only for the matching key and always keep the surface form. This allows aggressive collapsing of variants without losing the original text.
  • I3. Match on a consonant skeleton with a whitelist of allowed affix letters. This is a cheap, transparent rule for grouping spelling and inflection variants without training a model.
  • I4. Use embedding similarity to score and filter rule-generated candidates rather than to generate them. The rules provide explainable recall, while distributional similarity ranks the candidates and resolves conflicts between them.
  • I5. Evaluate the resource extrinsically and break the errors into named categories (spelling variant, code-switch, false neighbour). This shows clearly whether a larger lexicon trades recall for precision.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Lemma and lexicon entries could store harvested misspellings and informal spellings found in noisy English corpora, rather than trusting a spell-corrector to rewrite the input.
  • I2 (tokens): The tokenizer could compute a normalised lookup key alongside the untouched token, so later explanations can still point to the exact characters the user typed.
  • I3 (tokens): A skeleton-plus-affix rule could propose candidate lemmas for unknown English words, such as elongated or vowel-dropped forms, each with a readable reason for the match.
  • I4 (learn): Self-induced repair rules could propose candidates, and a similarity or witness-agreement score could decide which ones enter silver data, with rejected cases logged.
  • I5 (learn): Lexicon expansions could be judged by their effect on downstream task accuracy, with a typed error breakdown, so false-neighbour growth is caught as a regression.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Morphology · 420 worksLow-resource languages · 287 works

Cite

Cite as Taha Tobaili. 2020. Lexical Induction of Morphological and Orthographic Forms for Low-Resourced Languages. Proceedings of the Third Workshop on Multilingual Surface Realisation.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tobaili2020lexical,
  title = {{Lexical Induction of Morphological and Orthographic Forms for Low-Resourced Languages}},
  author = {Taha Tobaili},
  booktitle = {Proceedings of the Third Workshop on Multilingual Surface Realisation},
  year = {2020},
  url = {https://aclanthology.org/2020.msr-1.5/}
}

References

28 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mariam Aboelezz. 2009. Latinised arabic and connections to bilingual ability. In Lancaster University Postgraduate Conference in Linguistics & Language Teaching. Lancaster, UK, volume 3, pages 1–23. find
  2. Mohamed Al-Badrashiny, Ramy Eskander, Nizar Habash, and Owen Rambow. 2014. Automatic transliteration of romanized dialectal arabic. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 30–38.
  3. Jannis Androutsopoulos. 2012. Greeklish”: Transliteration practice and discourse in the context of computer-mediated digraphia. Orthography as social action: Scripts, spelling, identity and power, pages 359–392. find
  4. Timothy Baldwin, Marie-Catherine de Marneffe, Bo Han, Young-Bum Kim, Alan Ritter, and Wei Xu. 2015. Shared tasks of the 2015 workshop on noisy user-generated text: Twitter lexical normalization and named entity recognition. In Proceedings of the Workshop on Noisy User-generated Text, pages 126–135.
  5. Ramy Baly, Hazem Hajj, Nizar Habash, Khaled Bashir Shaban, and Wassim El-Hajj. 2017. A sentiment treebank and morphologically enriched recursive deep models for effective sentiment analysis in arabic. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):1–21. find
  6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  7. Danish Contractor, Tanveer A Faruquie, and L Venkata Subramaniam. 2010. Unsupervised cleansing of noisy text. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 189–196. Association for Computational Linguistics. find
  8. Kareem Darwish. 2014. Arabizi detection and conversion to arabic. ANLP 2014, page 217. https://tahatobaili.github.io/project-rbz/ ↗
  9. Leon Derczynski, Alan Ritter, Sam Clark, and Kalina Bontcheva. 2013. Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, pages 198–206.
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  11. Stephan Gouws, Dirk Hovy, and Donald Metzler. 2011. Unsupervised mining of lexical variants from noisy text. In Proceedings of the First workshop on Unsupervised Learning in NLP, pages 82–90. Association for Computational Linguistics. find
  12. Imane Guellil, Faiçal Azouaou, Mourad Abbas, and Sadat Fatiha. 2017. Arabizi transliteration of algerian arabic dialect into modern standard arabic. In Social MT 2017/First workshop on Social Media and User Generated Content Machine Translation. find
  13. Bo Han and Timothy Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a# twitter. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 368–378.
  14. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167. ↗
  15. Abir Masmoudi, Mariem Ellouze Khmekhem, Mourad Khrouf, and Lamia Hadrich Belguith. 2019. Transliteration of arabizi into arabic script for tunisian dialect. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(2):1–21. find
  16. Alice Millour and Karën Fort. 2020. Text corpora and the challenge of newly written languages. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), pages 111–120. find
  17. Sergiu Nisioi, Sanja Štajner, Simone Paolo Ponzetto, and Liviu P Dinu. 2017. Exploring neural text simplification models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 85–91.
  18. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. ↗
  19. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  20. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017), pages 502–518.
  21. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65:569–631. find
  22. Pingali Sailaja. 2011. Hinglish: code-switching in indian english. ELT journal, 65(4):473–480. find
  23. Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoı̂t Sagot, and Abhishek Srivastava. 2020. Building a user-generated content north-african arabizi treebank: Tackling hell. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1139–1150. find
  24. Radu Soricut and Franz Josef Och. 2015. Unsupervised morphology induction using word embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1627–1637. find
  25. Natalie Sullivan. 2017. Writing Arabizi: Orthographic Variation in Romanized Lebanese Arabic on Twitter. Ph.D. thesis. find
  26. Taha Tobaili, Miriam Fernandez, Harith Alani, Sanaa Sharafeddine, Hazem Hajj, and Goran Glavaš. 2019. Senzi: A sentiment analysis lexicon for the latinised arabic (arabizi). In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 1203–1211.
  27. Taha Tobaili. 2016. Arabizi identification in twitter data. In Proceedings of the ACL 2016 Student Research Workshop, pages 51–57.
  28. Mohammad Ali Yaghan. 2008. “arabizi”: A contemporary style of arabic slang. Design issues, 24(2):39–52. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tobaili-2020-lexical-induction-morphological-orthographic-forms/