Elamur Research⌕
☰
home / papers / 2018 / Approche Hybride pour la translitération de l'Arabizi Algérien : une étude préliminaire (A hybrid approach for the transliteration of Algerian Arabizi: A primary study)

Approche Hybride pour la translitération de l'Arabizi Algérien : une étude préliminaire (A hybrid approach for the transliteration of Algerian Arabizi: A primary study)

Paper · 2018 · Imane Guellil, Azouaou Faical, Fodil Benali et al.

Summary

Algerian dialect speakers often write online in Arabizi, a spontaneous romanized script that mixes Latin letters, digits and symbols. Most dialect-processing tools expect Arabic script, so this text has to be transliterated first. The task is hard for four reasons: vowels may map to several Arabic letters or to nothing, single letters are ambiguous (for example 't'), some words are ambiguous in context, and French or English words get mixed in.

The authors propose a hybrid method with four stages:

  1. Build and normalize an Arabic-script corpus of Algerian social-media comments.
  2. Apply a letter-mapping table plus hand-written, position-dependent rules.
  3. Generate every candidate Arabic spelling for each word.
  4. Pick the best candidate by corpus frequency and by a language model over that monolingual corpus.

No parallel Arabizi–Arabic data is needed. On three test corpora the best precision is 75.11%, which the authors consider competitive with general Arabizi transliteration work. The approach matters because it relies only on cheap monolingual data.

Key points

  • The Arabic corpus was scraped from 133 Algerian Facebook pages (telecom operators such as Ooredoo and Djezzy, drink producers, press). It was supplemented with terms from PADIC and an Algerian dialect dictionary.
  • Preprocessing collapses exaggerated letter repetition (e.g. hiaaaaati → hiati) on both the corpus and the input.
  • Each word gets many candidates: "kraht" yields 32 and "hiati" yields 16. The rules can map vowels to an empty character, which recovers correct forms that earlier candidate generators missed (e.g. '3afsa').
  • The best candidate is chosen in two steps: an occurrence count in the corpus, then the highest-probability candidate under a model built on the corpus.
  • The best precision is 75.11% across three test corpora. The remaining errors come from ignoring sentence context and from words of French origin (e.g. 'lafichage').

Interesting ideas

  • I1. Generate all candidates, then rank them with a monolingual corpus: an over-generating rule system can be disambiguated by checking which outputs occur in real target-side text, which avoids costly parallel data.
  • I2. Allowing "delete this symbol" as an explicit rewrite option makes the candidate space cover forms that one-to-one mappings can never reach, which is a cheap fix for recall gaps in rule-based normalizers.
  • I3. Collapsing exaggerated character repetition on both the reference corpus and the input keeps lookup consistent and shrinks the candidate explosion.
  • I4. The authors suggest running the transliterator semi-supervised to produce a parallel corpus, which would then train a statistical model and bootstrap from rules to data.
  • I5. Their error analysis splits failures into context-blind and foreign-word categories, which shows exactly where word-level decisions break and points to n-gram candidates as the next step.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Noisy or misspelled English tokens could be expanded into candidate normalizations by rules, then ranked by frequency in a clean corpus before tagging and lemmatization.
  • I2 (tokens): Spelling repair rules for informal English could include deletion of letters or apostrophes as explicit options, so contractions and elongations map back to dictionary forms.
  • I3 (tokens): Applying the same elongation normalization ("sooooo" → "so") to training and runtime text would keep tagger and lexicon lookups aligned.
  • I4 (learn): Rule-produced outputs that a corpus check confirms could serve as silver training pairs, kept explainable because each pair traces to a specific rule and a frequency check.
  • I5 (learn): Evaluations could tag each error by cause (missing context or out-of-language token) so that repair rules target the right stage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Imane Guellil, Azouaou Faical, Fodil Benali, Ala Eddine Hachani and Houda Saadane. 2018. Approche Hybride pour la translitération de l'Arabizi Algérien : une étude préliminaire (A hybrid approach for the transliteration of Algerian Arabizi: A primary study). Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{guellil2018approche,
  title = {{Approche Hybride pour la translitération de l'Arabizi Algérien : une étude préliminaire (A hybrid approach for the transliteration of Algerian Arabizi: A primary study)}},
  author = {Imane Guellil, Azouaou Faical, Fodil Benali, Ala Eddine Hachani and Houda Saadane},
  booktitle = {Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN},
  year = {2018},
  url = {https://aclanthology.org/2018.jeptalnrecital-court.35/}
}

References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al-Badrashiny M., Eskander R., Habash N., & Rambow, O. (2014). Automatic Transliteration of Romanized Dialectal Arabic. Paper presented at the CoNLL.
  2. Cotterell R., Renduchintala A., Saphra N., &Callison-Burch, C. (2014).An algerianarabic-french code-switched corpus. Paper presented at the Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools Workshop Programme. find
  3. Darwish K. (2013). Arabizi detection and conversion to Arabic.arXiv preprint arXiv:1306.6755. ↗
  4. Guellil I., & Azouaou F. (2017). ASDA: Analyseur Syntaxique du Dialecte Algérien dans un but d'analyse sémantique. arXivpreprint arXiv:1707.08998. ↗
  5. Guellil I., Azouaou F., & Abbas M. (2017). Comparison between Neural and Sta-tistical translation after translitera-tion of Algerian Arabic Dialect. Paper presented at the WiNLP: Women & Underrepresented Minorities in Natural Language Processing (co-located withACL 2017). find
  6. Guellil I., Azouaou F., Abbas M., & Fatiha, S. (2017a). Arabizi transliteration of Algerian Arabic dialect into Modern Standard Arabic. Paper presented at the Social MT 2017/First workshop on Social Media and User Generated Content Machine Translation. find
  7. Guellil I., AZOUAOU F. (2017b). Neural Vs Statistical Translation of Algerian Arabic Dialect written with Arabizi and Arabic letter. The 31st Pacific Asia Conference on Language, Information and Computation PACLIC. find
  8. Guellil., I et AZOUAOU F. (2017). Bilingual Lexicon for Algerian Arabic Dialect Treatment in Social Media. WiNLP: Women & Underrepresented Minorities in Natural Language Processing (co-located with ACL 2017) find
  9. Habash N., Soudi A., &Buckwalter, T. (2007). On arabic transliteration Arabic computational morphology (pp. 15-22): Springer. find
  10. Josan G. S., &Lehal, G. S. (2010).A Punjabi to Hindi Machine Transliteration System.Computational Linguistics and Chinese Language Processing, 15(2), 77-102. find
  11. Kaur K., & Singh P. (2014).Review of Machine Transliteration Techniques.International Journal of Computer Applications, 107(20). find
  12. May J., Benjira Y., &Echihabi, A. (2014). An Arabizi-English social media statistical machine translation system. Paper presented at the Proceedings of the 11th Conference of the Association for Machine Translation in the Americas.
  13. Meftouh K., Harrat S., Jamoussi S., Abbas, M., &Smaili, K. (2015). Machine translation experiments on PADIC: A parallel Arabic dialect corpus. Paper presented at the The 29th Pacific Asia conference on language, information and computation.
  14. Saâdane H., &Habash, N. (2015).A conventional orthography for Algerian Arabic.In Proceedings of the Second Workshop on Arabic Natural Language Processing (pp. 69-79).
  15. Saâdane H., & Semmar N. (2012).Utilisation de la transliteration arabe pour l'amélioration de l'alignement de mots à partir de corpus parallèles français-arabe (UsingArabic Transliteration to Improve Word Alignment from French-ArabicParallelCorpora)[in French].Paper presented at the Proceedings of the Joint Conference JEP-TALN-RECITAL 2012, volume 2: TALN. find
  16. Saâdane H., Nouvel,. D., Seffih H., &Fluhr, C. (2017). Une approche linguistique pour la détection des dialectes arabes. Paper presented at the 24e Conférence sur le Traitement Automatique des Langues Naturelles (TALN). find
  17. Rosca M., &Breuel T. (2016). Sequence-to-sequence neural network models for transliteration. arXiv preprint arXiv:1610.09565. ↗
  18. Saâdane H., Guidere M., &Fluhr C. (2013).La reconnaissance automatique des dialectes arabes à l’écrit. Paper presented at the Colloque International Traduction et Champs Connexes, Quelle Place Pour La Langue Arabe Aujourd’hui. van der Wees, M., Bisazza A., &Monz C. (2016). A Simple but Effective Approach to Improve Arabizi-to-English Statistical Machine Translation. WNUT 2016, 43.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guellil-2018-approche-hybride-pour-la-transliteration/