Elamur Research⌕
☰
home / papers / 2026 / Kyrgyz Text Normalization: A Comparative Study of Neural and Rule-Based Approaches

Kyrgyz Text Normalization: A Comparative Study of Neural and Rule-Based Approaches

Paper · 2026 · Zarina Uvalieva, Bektemir Kumarbai Uulu, Adilet Metinov et al.

Summary

Kyrgyz is a morphologically rich, low-resource Turkic language. Its web text routinely lacks punctuation, proper casing and standard spelling, which hurts downstream tools such as MT, NER and speech synthesis. The authors say no earlier work has tackled automatic normalization for Kyrgyz, so they build the first benchmark for it. They collected about 1.67M noisy–clean pairs from YouTube, Instagram and Telegram, with clean targets produced by Gemini 3 Pro. Two native speakers checked the entire 1,000-pair test set, with adjudication, while the training data was only spot-checked. Five systems are compared:

  • a deliberately minimal rule-based lower bound,
  • zero-shot mT5,
  • fine-tuned mT5-small,
  • mT5-small continually pre-trained on a 538 MB news-and-books corpus and then fine-tuned,
  • zero-shot Gemma 4.

Fine-tuned mT5-small achieves the lowest CER and beats the much larger Gemma 4 used without tuning. Extra span-corruption pre-training gives no gain, largely because the model hallucinates content. The study provides a reproducible starting point and deployable small models for Kyrgyz preprocessing pipelines.

Key points

  • Data: 1,672,715 training and 1,000 test pairs, with zero exact-match overlap between them. Sources are YouTube (45%), Telegram (30%) and Instagram (25%). Punctuation restoration is needed in 84.9% of test examples. In a 400-pair training spot-check, 84% of pairs were valid (Wilson 95% CI ≈ 80–87%).

  • CER results:

    • fine-tuned mT5-small: 0.0796
    • pre-train + fine-tune: 0.0825 (difference p = 0.06, inconclusive)
    • zero-shot Gemma 4 (~32× more parameters): 0.1620
    • rule-based: 0.2029
    • zero-shot mT5: 0.9887

  • Human evaluation: Two native speakers each rated 200 outputs per system. Both fine-tuned variants were judged correct in 99.8% of cases. Agreement is reported with Cohen's κ, PABAK and Gwet's AC1.

  • Robustness and error analysis: Hallucination explains 35 of 40 inspected pre-training failures (87.5%, CI 74–95%). On a 50-example reference-bias probe, mT5's CER shifts by only 0.012 and the system ranking stays the same.

  • Released resources:

    • code;
    • checkpoints Zarinaaa/mt5-small-kyrgyz-normalization and -ptft;
    • a 20k-pair training subset plus the full test set at Zarinaaa/kyrgyz-text-normalization.

    The following are left for future work: ByT5, an FST-based rule pipeline (Washington et al., 2012), and fine-tuned LLMs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Barcelona Digital Centro Tecnológico.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Zarina Uvalieva, Bektemir Kumarbai Uulu, Adilet Metinov, Tynchtykbek Tashbaltaev and Nurtilek Alibekov. 2026. Kyrgyz Text Normalization: A Comparative Study of Neural and Rule-Based Approaches. Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.mellm-1.5

@inproceedings{uvalieva2026kyrgyz,
  title = {{Kyrgyz Text Normalization: A Comparative Study of Neural and Rule-Based Approaches}},
  author = {Zarina Uvalieva, Bektemir Kumarbai Uulu, Adilet Metinov, Tynchtykbek Tashbaltaev and Nurtilek Alibekov},
  booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.mellm-1.5},
  url = {https://doi.org/10.18653/v1/2026.mellm-1.5}
}

References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anton Alekseev, Sergey I. Nikolenko, and Gulnara Kabaeva. 2023. Benchmarking multilabel topic classification in the Kyrgyz language. arXiv preprint arXiv:2308.15952. — [arXiv:2308.15952](https://arxiv.org/abs/2308.15952) find
  2. Alvan R Feinstein and Domenic V Cicchetti. 1990. High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6):543–549. find
  3. Asiye Tuba Koksal, Ozge Bozal, Emre Yürekli, and Gizem Gezici. 2020. \#TurkishTweets: A benchmark dataset for Turkish text correction. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4303–4315. Association for Computational Linguistics. find
  4. Anne-Marie Lutgen, Alistair Plum, Christoph Purschke, and Barbara Plank. 2025. Neural text normalization for Luxembourgish using real-life variation data. In Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2025), pages 94–106. Association for Computatio…
  5. Dilara Torunoğlu and Gülşen Eryiğit. 2014. A cascaded approach for social media text normalization of Turkish. In Proceedings of the Workshop on Language Analysis for Social Media, pages 62–70. Association for Computational Linguistics. find
  6. Jonathan N. Washington, Mirlan Ipasov, and Francis M. Tyers. 2012. A finite-state morphological transducer for Kyrgyz. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 934–940. find
  7. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. find
  8. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
  9. Andrew Zupon, Evan Crew, and Sandy Ritchie. 2021. Text normalization for low-resource languages of Africa. arXiv preprint arXiv:2103.15845. — [arXiv:2103.15845](https://arxiv.org/abs/2103.15845) find
  10. Talha Çolakoğlu, Umut Sulubacak, and Ahmet Cüneyd Tantuğ. 2019. Normalizing non-canonical Turkish texts using machine translation approaches. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 267–272. Association for Computat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/uvalieva-2026-kyrgyz-text-normalization-comparative-study/