Elamur Research⌕
☰
home / papers / 2022 / GeezSwitch: Language Identification in Typologically Related Low-resourced East African Languages

GeezSwitch: Language Identification in Typologically Related Low-resourced East African Languages

Summary

Online services such as machine translation often mix up closely related languages that share the Ge'ez (Fidel) script. Tigrinya, for example, is regularly processed as if it were Amharic. Widely used identifiers such as LANGDETECT lack these languages, and langid.py covers only Amharic. The authors build GeezSwitch, a language-identification dataset for five related Afro-Asiatic languages: Amharic, Blin, Ge'ez, Tigre and Tigrinya. They frame the task as five-way text classification. The pipeline is almost fully automatic. It gathers text from existing corpora or scrapes new sources, normalises it, keeps only Ge'ez-script characters and filters samples by length. Humans are involved only to verify the evaluation portion. The dataset was added to an existing open-source LI tool and used to fine-tune several Transformer-based pre-trained models, and all setups scored very highly. The work matters for three reasons: it provides the first NLP task for Blin and Tigre, it offers a cheap and repeatable recipe for other under-resourced languages, and it gives a basic benchmark for languages that have no NLU test suites.

Key points

  • The five languages are Amharic (amh, ~32M speakers), Blin (byn, ~120k, Cushitic), Ge'ez (gez, liturgical only), Tigre (tig, ~1M) and Tigrinya (tir, ~9M), with over 42M speakers in total.
  • Data sources:
    • Amharic was sampled from earlier embedding and semantic-model datasets (Mersha and Wu 2020; Yimam et al. 2021).
    • Tigrinya came from the TLMD corpus.
    • Blin was scraped from Daberi.org.
    • Ge'ez came from church-published Bible and religious texts.
    • Tigre came from the newspaper Eritrea Haddas.
    • Much of the Blin and Tigre material was in PDFs with unreadable or non-standard encodings.
  • Preprocessing proceeds in three steps:
    • The text is normalised, unifying quote and whitespace variants.
    • Every character outside the Ge'ez Unicode block (U+1200) is dropped, except the hyphen and apostrophe.
    • Only samples of 4–30 tokens are kept.
  • From the cleaned data, 3000 samples per language were drawn for train, validation and test splits, and only the evaluation part was checked by hand.
  • A training-size study found that a large training set is not needed for good LI. Pre-trained models did weaken when examples per language were very few, so the authors suggest LI as a probe of model ability for languages without benchmarks.

Interesting ideas

  • I1. Strip all language-neutral material (digits, URLs, symbols, foreign-script words) before classifying, so the model can only rely on genuinely language-bearing signal. — For Mova (tokens): A script/character-class filter in crate en could flag spans that carry no English evidence, such as numbers, code and foreign script, before tagging, so they get dedicated handling instead of being guessed at.
  • I2. Set both a lower and an upper length bound on samples; this removes ambiguous fragments and run-on noise and also evens out token and vocabulary statistics across classes. — For Mova (learn): Apply a sentence-length window when selecting silver sentences that witness parsers must agree on, so the training pool is not skewed by fragments or extreme run-ons.
  • I3. Build training data fully automatically, but spend scarce human effort only on verifying the held-out evaluation split. — For Mova (learn): Keep silver data automatic, but hand-check a small gold test slice so reported parser and frame scores rest on trusted ground truth.
  • I4. An off-the-shelf identifier that knows only one language of a family labels all its relatives as that language, which is a telling failure mode to test. — For Mova (learn): Use negative controls made of out-of-coverage input (dialects, non-English, code) to check that each stage reports "not mine" instead of confidently producing a tree or answer.
  • I5. Plot performance against the number of training examples per class to see how little data is actually enough. — For Mova (learn): Run learning curves for perceptron versus gparse training on growing treebank slices to decide where extra silver data stops paying off.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

11 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gaim2022geezswitch,
  title = {{GeezSwitch: Language Identification in Typologically Related Low-resourced East African Languages}},
  author = {Gaim, Fitsum and Yang, Wonsuk and Park, Jong C.},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/3n56owmf2dq2},
  url = {https://doi.org/10.63317/3n56owmf2dq2}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Cavnar, W. B. and Trenkle, J. M. (1994). N-gram-based text categorization. In Proceedings of 3rd Annual Symposium on Document Analysis and Information Retrieval (SDAIR’94), volume 161175, page 161–175. find
  2. Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In ACL. Association for Computational Linguistics. find
  3. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL. find
  4. Eberhard, D., Simons, G., and Fennig, C. (2019). Ethnologue: Languages of the World, Twenty-Second Edition. Ethnologue Series. SIL International, Global Publishing. find
  5. Elworthy, D. (1998). Language identification with confidence limits. In Proceedings of the 6th Annual Workshop on Very Large Corpora, page 94–101. find
  6. Gaim, F., Yang, W., and Park, J. C. (2021a). Monolingual Pre-trained Language Models for Tigrinya. In WiNLP co-located with EMNLP 2021. find
  7. Gaim, F., Yang, W., and Park, J. C. (2021b). TLMD: Tigrinya Language Modeling Dataset. TLMD. Zenodo: https://doi.org/10.5281/zenodo.5139094, July. ↗
  8. Garg, A., Gupta, V., and Jindal, M. K. (2014). A survey of language identification techniques and applications. Journal of Emerging Technologies in Web Intelligence, 6:388–400. find
  9. Gold, E. M. (1967). Language identification in the limit. Information and Control, 10(5):447–474. find
  10. Jauhiainen, T., Lui, M., Zampieri, M., Baldwin, T., and Lindén, K. (2019). Automatic language identification in texts: A survey. ArXiv, abs/1804.08186. ↗ our card
  11. Loshchilov, I. and Hutter, F. (2018). Fixing weight decay regularization in adam. In ICLR. find
  12. Lui, M. and Baldwin, T. (2012). langid.py: An off-the-shelf language identification tool. In ACL. Association for Computational Linguistics. find our card
  13. Mersha, A. and Wu, S. (2020). Morphology-rich alphasyllabary embeddings. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2590–2595. find our card
  14. Ogueji, K., Zhu, Y., and Lin, J. (2021). Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic, November. Association for Computational Linguistics. find our card
  15. Radford, A. and Narasimhan, K. (2018). Improving language understanding by generative pre-training. find
  16. Shuyo, N. (2014). Language detection library for java. https://github.com/shuyo/language-detection. ↗
  17. Tiedemann, J. and Ljubesic, N. (2012). Efficient discrimination between closely related languages. In COLING. find
  18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, pages 5998– 6008. find
  19. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online, October. Association for Computational Linguistics. find our card
  20. Yimam, S. M., Ayele, A. A., Venkatesh, G., Gashaw, I., and Biemann, C. (2021). Introducing various semantic models for amharic: Experimentation and evaluation with multiple tasks and datasets. Future Internet, 13(11). find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gaim-2022-geezswitch-language-identification-typologically-related/