Elamur Research⌕
☰
home / papers / 2026 / A Subword Embedding Approach for Variation Detection in Luxembourgish User Comments

A Subword Embedding Approach for Variation Detection in Luxembourgish User Comments

Summary

Written Luxembourgish, especially in informal online settings, shows extensive spelling and lexical diversity. NLP pipelines usually normalise this diversity away. The authors argue that it carries linguistic information. They propose a way to discover families of variant forms directly from raw text, without seed lexicons, predefined variant lists or normalisation rules.

The pipeline has three steps:

  1. Embeddings. FastText subword embeddings are trained on a large, minimally cleaned corpus of comments that users posted on one online platform.
  2. Grouping. Candidate variants are linked by combining embedding cosine similarity with character n-gram Jaccard overlap. Connected components of the resulting graph become variant families.
  3. Filtering and review. Families are pruned by size, cohesion, frequency ratio and spread across users. The surviving clusters are then reviewed and categorised manually.

Applied to Luxembourgish, the method yields roughly 800 variant families in seven categories. They reflect systematic orthographic and lexical patterns consistent with dialectological and sociolinguistic research, and many are undocumented in existing resources. The work offers a transparent, reproducible framework for studying variation in small, low-resource languages. The authors note it appears less effective for highly standardised languages.

Key points

  • Training setup: FastText skip-gram with 100-dimensional vectors, window 5, min_count 10, 10 epochs and character n-grams of length 3–7.
  • Strict grouping mode (used in the analysis): up to 100 neighbours per seed, cosine threshold 0.73, DEGREE_CAP 200 and minimum family size 2. Cohesion is the harmonic mean of average cosine and average Jaccard scores.
  • Comparison dimension: user_id. Filters include MIN_USERS = 3, MAX_FREQ_RATIO = 25 and a minimum token length of 3. Outputs are a JSONL file of families plus a CSV summary.
  • Main result: about 800 variant families across seven analytically defined categories. Many of these patterns are absent from existing resources such as Gilles' Variation Atlas.
  • Availability and limitations: the code is public (github.com/plumaj/vadamt).
    • The data come from one platform only.
    • Rare variants may be missed.
    • Clusters cannot by themselves separate linguistic variation from author-specific or temporal effects, so qualitative validation is still needed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.vardial-1.9

@inproceedings{lutgen2026subword,
  title = {{A Subword Embedding Approach for Variation Detection in Luxembourgish User Comments}},
  author = {Lutgen, Anne-Marie and Plum, Alistair and Purschke, Christoph},
  booktitle = {Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2026},
  doi = {10.18653/v1/2026.vardial-1.9},
  url = {https://doi.org/10.18653/v1/2026.vardial-1.9}
}

References

32 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Martine Adda-Decker, Thomas Pellegrini, Eric Bilinski, and Gilles Adda. 2008. Developments of “Lëtzebuergesch” Resources for Automatic Speech Processing and Linguistic Studies. In Proceedings of LREC. find
  2. Khetam Al Sharou, Zhenhao Li, and Lucia Specia. 2021. Towards a Better Understanding of Noise in Natural Language Processing. In Proceedings of RANLP. find
  3. Dimitra Anastasiou. 2022. ENRICH4ALL: A First Luxembourgish BERT Model for a Multilingual Chatbot. In Proceedings of SIGUL. ELRA. find
  4. Alistair Baron and Paul Rayson. 2008. VARD2: A tool for dealing with spelling variation in historical corpora. find
  5. Laura Bernardy. 2022. A Luxembourgish GPT-2 Approach Based on Transfer Learning. Master’s thesis, University of Trier. find
  6. This study uses publicly accessible user comments, but they remain sensitive textual data. All pro-Piotr Bojanowski, Edouard Grave, Armand Joulin, and cessing follows the terms of use of the platform Tomas Mikolov. 2017. Enriching word vectors with from which the comments were collected. No at-subword information. In Transactions of ACL. tempt is made to identify individual users, and the Steven Cao, Nikita Kitaev, and Dan Klein. 2020. Multi-analysis relies only on aggregated patterns such as lingual alignment of contextual word representations. variant frequencies and distribution across dimen-In Proceedings of ICLR. sions. Even though user identifiers are present in Jacob Eisenstein. 2013. What to do about bad language the raw data, they are treated only as categorical on the internet. In Proceedings of NAACL-HLT. variables and are not (and could not be) interpreted as personal attributes. The dataset represents a self- Nathalie Entringer, Peter Gilles, Sara Martin, and selected set of online participants whose linguistic Christoph Purschke. 2021. Schnëssen. surveying 121 language dynamics in luxembourgish with a mobile research app. Linguistics Vanguard, 7. find
  7. Pieter Fivez, Simon Šuster, and Walter Daelemans. 2017. Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings. Computational Linguistics in the Netherlands Journal. ↗
  8. Daniela Gierschek. 2022. Detection of Sentiment in Luxembourgish User Comments. Ph.D. thesis, University of Luxembourg. find
  9. Peter Gilles. 1999. Dialektausgleich im Lëtze-buergeschen: Zur phonetisch-phonologischen Fokussierung einer Nationalsprache. Niemeyer, Tübingen, Germany. find
  10. Peter Gilles. 2019. 39. Komplexe Überdachung II: Luxemburg. Die Genese Einer Neuen Nationalsprache. In Joachim Herrgen and Jürgen Erich Schmidt, editors, Sprache und Raum - Ein internationales Handbuch der Sprachvariation. Volume 4 Deutsch. De Gruyter Mouton, Berlin, Boston. find
  11. Peter Gilles. 2021. Variatiounsatlas vum lëtze-buergeschen. https://infolux.uni.lu/ variatiounsatlas. Accessed: 15.10.2025. ↗
  12. Peter Gilles, Léopold Edem Ayité Hillah, and Nina Hosseini Kivanani. 2023a. ASRLUX: Automatic Speech Recognition for the Low-Resource Language Luxembourgish. In Proceedings of the International Congress of Phonetic Sciences. find
  13. Cedric Lothritz, Bertrand Lebichot, Kevin Allix, Lisa Veiber, Tegawende Bissyande, Jacques Klein, Andrey Boytsov, Clément Lefebvre, and Anne Goujon. 2022. LuxemBERT: Simple and Practical Data Augmentation in Language Model Pre-Training for Luxembourgish. In Proceedings of LREC. find
  14. Anne-Marie Lutgen, Alistair Plum, Christoph Purschke, and Barbara Plank. 2025. Neural text normalization for Luxembourgish using real-life variation data. In Proceedings of VarDial (COLING). ↗
  15. Robert Munro and Christopher D. Manning. 2010. Subword Variation in Text Message Classification. In Proceedings of NAACL-HLT. find
  16. Dong Nguyen and Jack Grieve. 2020. Do Word Embeddings Capture Spelling Variation? In Proceedings of ICCL. ↗
  17. Nhi Pham, Lachlan Pham, and Adam Meyers. 2024. Towards Better Inclusivity: A Diverse Tweet Corpus of English Varieties. In Proceedings of LAW (ACL). find
  18. Fred Philippy, Shohreh Haddadan, and Siwen Guo. 2024. Forget NLI, Use a Dictionary: Zero-Shot Topic Classification for Low-Resource Languages with Application to Luxembourgish. In Proceedings of SIGUL (LREC-COLING). find
  19. Alistair Plum, Caroline Döhmer, Emilia Milano, Anne-Marie Lutgen, and Christoph Purschke. 2024. LuxBank: The first Universal Dependency treebank for Luxembourgish. In Proceedings of TLT. ↗
  20. Alistair Plum, Tharindu Ranasinghe, and Christoph Purschke. 2025. Text generation models for Luxembourgish with limited data: A balanced multilingual strategy. In Proceedings of VarDial (COLING). ↗
  21. Peter Gilles, Nina Hosseini Kivanani, and Léopold Edem Ayité Hillah. 2023b. LUX-ASR: Building an ASR system for the Luxembourgish language. In Proceedings of IEEE Spoken Language Technology Workshop. find
  22. Jack Grieve, Chris Montgomery, Andrea Nini, Akira Murakami, and Diansheng Guo. 2019. Mapping Lexical Dialect Variation in British English Using Twitter. Frontiers in AI, 2. ↗
  23. Hang Jiang, Haoshen Hong, Yuxing Chen, and Vivek Kulkarni. 2020. DialectGram: Automatic Detection of Dialectal Changes with Multi-geographic Resolution Analysis. In Proceedings of the Society for Computation in Linguistics. find
  24. Julie Kallini, Dan Jurafsky, Christopher Potts, and Martijn Bartelds. 2025. False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models. arXiv preprint. ArXiv:2509.18750 [cs]. ↗
  25. Thomas Lavergne, Gilles Adda, Martine Adda-Decker, and Lori Lamel. 2014. Automatic language identity tagging on word and sentence-level in multilingual text sources: a case-study on Luxembourgish. In Proceedings of LREC. find
  26. Christoph Purschke. 2020. Attitudes Toward Multilingualism in Luxembourg. A Comparative Analysis of Online News Comments and Crowdsourced Questionnaire Data. Frontiers in AI, 3. find
  27. Tharindu Ranasinghe, Alistair Plum, Christoph Purschke, and Marcos Zampieri. 2023. Publish or Hold? Automatic Comment Moderation in Luxembourgish News Articles. In Proceedings of RANLP. find
  28. Joshgun Sirajzade, Daniela Gierschek, and Christoph Schommer. 2020. An Annotation Framework for Luxembourgish Sentiment Analysis. In Proceedings of SLTU-CCURL (LREC). find
  29. Natalie D. Snoeren, Martine Adda-Decker, and Gilles Adda. 2010. The Study of Writing Variants in an Under-resourced Language: Some Evidence from Mobile N-Deletion in Luxembourgish. In Proceedings of LREC. find
  30. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37. ↗
  31. Omer Levy and Yoav Goldberg. 2014. Neural word embedding as implicit matrix factorization. In Proceedings of NeurIPS. find
  32. Zenter fir d’Lëtzebuerger Sprooch, editor. 2019. D’Lëtzebuerger Orthografie. Zenter fir d’Lëtzebuerger Sprooch, Stroossen. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/lutgen-2026-subword-embedding-approach-variation-detection/