Elamur Research⌕
☰
home / papers / 2025 / Neural Text Normalization for Luxembourgish Using Real-Life Variation Data

Neural Text Normalization for Luxembourgish Using Real-Life Variation Data

Paper · 2025 · Anne-Marie Lutgen, Alistair Plum, Christoph Purschke et al.

Summary

Written Luxembourgish varies widely in spelling. Its official orthography is still being standardized, and most writers never learned it at school. Normalization tools are hard to build because there is almost no annotated or parallel standard/non-standard text. The authors work around this by creating parallel data from real usage. Clean sentences come from professionally produced transcriptions of the Luxembourg Chamber of Deputies. Their words are then swapped for spelling variants that users actually typed on Spellchecker.lu, sampled according to how often each variant occurs. With this data they fine-tune ByT5 and mT5, the first sequence-to-sequence normalizers for the language. They compare these against prompted GPT-4o and Llama 3.1 and against the word-based pipeline spellux. Evaluation combines quantitative metrics such as error reduction rate (ERR) with a qualitative test set organized by Luxembourgish orthographic rules. GPT-4o scores best overall. ByT5 base reaches a competitive ERR of 0.26, and both ByT5 and spellux handle certain phenomena especially well. The work shows that realistic, frequency-aware noise is a practical route to normalization for languages with little parallel data.

Key points

  • The Spellchecker.lu variant dictionary covers 138,802 lemmas. It is built from logged pairs of what users entered and what they corrected it to, aggregated with usage frequencies.
  • The clean source text is Chamber of Deputies transcriptions from 2002–2012 and 2019–2020. Sequential, frequency-based word replacement turns them into about 833,000 sentence pairs, with an average of 19 tokens changed per pair.
  • The best ByT5 base model (582M parameters) used batch size 16, learning rate 1e-4, sequence length 256 and 3 epochs. ByT5 large (1.23B) and mT5 base were also tried, under tighter settings because of GPU limits.
  • Baselines were GPT-4o (gpt-4o-2024-08-06), Llama-3.1-8B-Instruct prompted to act as a Luxembourgish teacher, and the spellux pipeline. GPT-4o was strongest, and ByT5 base reached an ERR of 0.26.
  • The authors note that the variants reflect only Spellchecker.lu users, not the full variation space of the language.

Interesting ideas

  • I1. Instead of random character edits, noise is injected by replacing words with variants that people really wrote, at the rates they really wrote them, so the training distribution matches real errors.
  • I2. A spellchecker's log of user corrections can be repurposed as a lexicon of non-standard forms tied to each lemma, which avoids costly manual annotation.
  • I3. The same frequency-weighted corruption could be applied to sentences whose gold analysis is already known, producing robustness test material without new annotation.
  • I4. Alongside aggregate scores, the authors use a qualitative test set grouped by specific orthographic rules, reporting success per phenomenon to expose each system's strengths and weaknesses.
  • I5. ERR scores a normalizer against the trivial baseline of changing nothing, so a system that damages already-correct words is penalized.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Clean UD-tagged English text could be corrupted using a list of attested misspellings with frequencies, to train the tagger and lemmatizer to map noisy forms back to their lemmas.
  • I2 (tokens): Correction pairs of this kind could feed a variant-to-lemma lookup in crate en, so that unknown tokens get an explainable normalization before tagging.
  • I3 (tree): Corrupted copies of treebank sentences could test whether the perceptron parser and gparse keep the same tree under realistic misspellings, with the original trees serving as gold.
  • I4 (learn): Evaluation could report accuracy per linguistic phenomenon, which shows exactly which repair rules or teacher tiers help or hurt instead of hiding them in one number.
  • I5 (learn): Self-induced repair rules could be scored with an ERR-style measure against a no-op baseline, acting as a built-in negative control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations🎓University of LuxembourgLuxembourg · Luxembourg27 works · 337 citations

Also on the byline: LMU Klinikum.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Anne-Marie Lutgen, Alistair Plum, Christoph Purschke and Barbara Plank. 2025. Neural Text Normalization for Luxembourgish Using Real-Life Variation Data. Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lutgen2025neural,
  title = {{Neural Text Normalization for Luxembourgish Using Real-Life Variation Data}},
  author = {Anne-Marie Lutgen, Alistair Plum, Christoph Purschke and Barbara Plank},
  booktitle = {Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2025},
  url = {https://aclanthology.org/2025.vardial-1.9/}
}

References

41 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martine Adda-Decker, Thomas Pellegrini, Eric Bilinski, and Gilles Adda. 2008. Developments of “Lëtzebuergesch” Resources for Automatic Speech Processing and Linguistic Studies. In Proceedings of LREC 08, Marrakech, Morocco. ELRA. find
  2. François Conrad. 2017. Variation durch Sprachkontakt. Peter Lang Verlag, Berlin, Germany. find
  3. François Conrad. 2023. Deutsch-luxemburgischer Sprachkontakt in Luxemburg, pages 53–88. Olms Verlag. find
  4. Daniela Gierschek. 2022. Detection of Sentiment in Luxembourgish User Comments. Ph.D. thesis, University of Luxembourg. find
  5. Peter Gilles. 2006. Phonologie der n -Tilgung im Mo-selfränkischen (’Eifler Regel’). Ein Beitrag zur di-alektologischen Prosodieforschung. In Perspektiven einer linguistischen Luxemburgistik. Studien zur Diachronie und Synchronie. Winter. find
  6. Peter Gilles. 2014. Phonological domains in Luxembourgish and their relevance for the phonological system. In Syllable and Word Languages. de Gruyter. ↗
  7. Peter Gilles. 2015. From Status to Corpus: Codification and Implementation of Spelling Norms in Luxembourgish. In Language Planning and Microlinguis-tics: From Policy to Interaction and Vice Versa, pages 128–149. Springer. ↗
  8. Peter Gilles. 2019. 39. Komplexe Überdachung II: Luxemburg. Die Genese Einer Neuen Nationalsprache, pages 1039–1060. De Gruyter Mouton, Berlin, Boston. find
  9. Peter Gilles. 2023. Luxembourgish. In The Oxford Encyclopedia of Germanic Linguistics. Oxford University Press. find
  10. Peter Gilles, Léopold Edem Ayité Hillah, and Nina Hosseini Kivanani. 2023a. ASRLUX: AUTO-MATIC SPEECH RECOGNITION FOR THE LOW-RESOURCE LANGUAGE LUXEMBOURGISH. In Proceedings of the 20th International Congress of Phonetic Sciences. Guarant International. find
  11. Peter Gilles, Nina Hosseini Kivanani, and Léopold Edem Ayité Hillah. 2023b. LUX-ASR: Building an ASR system for the Luxembourgish language. In Proceedings - 2022 IEEE Spoken Language Technology Workshop (SLT). find
  12. Peter Gilles and Jürgen Trouvain. 2013. Luxembourgish. Journal of the International Phonetic Association, 43(1):67–74. find
  13. Dimitra Anastasiou. 2022. ENRICH4ALL: A First Luxembourgish BERT Model for a Multilingual Chatbot. In Proceedings of ELRA/ISCA Special Interest Group on Under-Resourced Languages, Marseille, France. ELRA. find
  14. Peter Gilles and Jürgen Trouvain. 2015. Closure durations in stops and grammatical encoding: On definite articles in Luxembourgish. In Proceedings of the 18th International Congress of Phonetic Sciences. Glasgow, UK: the University of Glasgow. find
  15. Laura Bernardy. 2022. A Luxembourgish GPT-2 Approach Based on Transfer Learning. Master’s thesis, University of Trier. find
  16. Bo Han and Timothy Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a #twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 368–378, Portland, Oregon, USA. Association for Computational Linguistics.
  17. Marcel Bollmann. 2018. Normalization of historical texts with neural network models. Doctoral thesis, Ruhr-Universität Bochum, Universitätsbibliothek. ↗
  18. Dilek Küçük and Ralf Steinberger. 2014. Experiments to improve named entity recognition on Turkish tweets. In Proceedings of the 5th Workshop on Language Analysis for Social Media (LASM), pages 71– 78, Gothenburg, Sweden. Association for Computational Linguistics. find
  19. Olli Kuparinen, Aleksandra Miletić, and Yves Scherrer. 2023. Dialect-to-Standard Normalization: A Large-Scale Multilingual Evaluation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13814–13828, Singapore. Association for Computational Linguistics. find
  20. Thomas Lavergne, Gilles Adda, Martine Adda-Decker, and Lori Lamel. 2014. Automatic language identity tagging on word and sentence-level in multilingual text sources: a case-study on Luxembourgish. In Proceedings of LREC 2014, Reykjavik, Iceland. ELRA. find
  21. Cedric Lothritz, Bertrand Lebichot, Kevin Allix, Lisa Veiber, Tegawende Bissyande, Jacques Klein, Andrey Boytsov, Clément Lefebvre, and Anne Goujon. 2022. LuxemBERT: Simple and Practical Data Augmentation in Language Model Pre-Training for Luxembourgish. In Proceedings of LREC 2022, Marseille, France. European Language Resources Association. find
  22. Massimo Lusetti, Tatyana Ruzsics, Anne Göhring, Tanja Samardžić, and Elisabeth Stark. 2018. Encoder-decoder methods for text normalization. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 18–28, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  23. Saul B. Needleman and Christian D. Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48(3):443–453. ↗
  24. Fred Philippy, Shohreh Haddadan, and Siwen Guo. 2024. Forget NLI, use a dictionary: Zero-shot topic classification for low-resource languages with application to Luxembourgish. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages LREC-COLING 2024, Torino, Italia. ELRA and ICCL. find
  25. Barbara Plank, Kristian Nørgaard Jensen, and Rob van der Goot. 2020. DaN+: Danish Nested Named Entities and Lexical Normalization. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6649–6662, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  26. Christoph Purschke. 2020. Attitudes Toward Multilingualism in Luxembourg. A Comparative Analysis of Online News Comments and Crowdsourced Questionnaire Data. Frontiers in AI, 3. find
  27. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140). ↗
  28. Tharindu Ranasinghe, Alistair Plum, Christoph Purschke, and Marcos Zampieri. 2023. Publish or hold? Automatic comment moderation in Luxembourgish news articles. In Proceedings of RANLP 2023, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria. find
  29. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902– 4912, Online. Association for Computational Linguistics. ↗
  30. Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A Simple Recipe for Multilingual Grammatical Error Correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 702–707, Online. Association for Computational Linguistics. find
  31. David Samuel and Milan Straka. 2021. ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 483–492, Online. Association for Computational Linguistics. find
  32. Joshgun Sirajzade, Daniela Gierschek, and Christoph Schommer. 2020. An Annotation Framework for Luxembourgish Sentiment Analysis. In Proceedings of SLTU-CCURL 2020, Marseille. LREC. find
  33. Natalie D. Snoeren, Martine Adda-Decker, and Gilles Adda. 2010. The study of writing variants in an under-resourced language: Some evidence from mobile n-deletion in Luxembourgish. In Proceedings of LREC 2010), Valletta, Malta. ELRA. find
  34. Alistair Plum, Caroline Döhmer, Emilia Milano, Anne-Marie Lutgen, and Christoph Purschke. 2024a. LuxBank: The First Universal Dependency Treebank for Luxembourgish. Preprint, arXiv:2411.04813. ↗
  35. Rob van der Goot. 2019. MoNoise: A Multi-lingual and Easy-to-use Lexical Normalization Tool. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 201–206, Florence, Italy. Association for Computational Linguistics. find
  36. Alistair Plum, Tharindu Ranasinghe, and Christoph Purschke. 2024b. Text generation models for luxembourgish with limited data: A balanced multilingual strategy. Preprint, arXiv:2412.09415. ↗
  37. Rob van der Goot and Özlem Çetinoğlu. 2021. Lexical normalization for code-switched data and its effect on POS tagging. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2352–2365, Online. Association for Computational Linguistics. find
  38. Rob van der Goot, Alan Ramponi, Arkaitz Zubiaga, Barbara Plank, Benjamin Muller, Iñaki San Vicente Roncal, Nikola Ljubešić, Özlem Çetinoğlu, Rahmad Mahendra, Talha Çolakoğlu, Timothy Baldwin, Tommaso Caselli, and Wladimir Sidorenko. 2021. MultiLexNorm: A Shared Task on Multilingual Lexical Normalization. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 493–509, Online. Association for Computational Linguistics. find
  39. Rob Matthijs van der Goot. 2019. Normalization and parsing algorithms for uncertain input. Ph.D. thesis, University of Groningen. find
  40. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. ↗
  41. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lutgen-2025-neural-text-normalization-luxembourgish-real/