Elamur Research⌕
☰
home / papers / 2026 / Stylistic Evolution and LLM Neutrality in Singlish Language

Stylistic Evolution and LLM Neutrality in Singlish Language

Paper · 2026 · Linus Tze En Foo, Weihan Angela Ng, Wenkai Li et al.

Summary

The paper asks two questions about Singlish, the English-based contact variety of Singapore. First, how has its written style changed over ten years of informal messaging? Second, can LLMs produce Singlish that reflects the variety's stable core rather than the habits of one particular year?

The authors use CoSEM messages from 2012–2021. They represent the text in two ways: handcrafted features (structural measures, discourse particles, LIWC-22 scores pooled per year) and all-MiniLM-L6-v2 embeddings. A Gradient Boosted Classifier tries to tell two sets of texts apart. Its accuracy is converted into a similarity score, and the spread of that score across years serves as a measure of temporal neutrality.

Year pairs become easier to separate as the time gap between them widens. The shift comes mostly from length and complexity, while pragmatic particles remain fairly stable. When generated text is checked against a null-distribution baseline, most LLMs show a trade-off. Models that sound authentic carry year-specific bias, and models that are neutral across years sound generic. SeaLLM is a partial exception.

The work turns temporal neutrality into a diagnostic for whether a model has actually grounded a sociolect.

Key points

  • Data: CoSEM messages from 2012–2021 (2022 excluded for low volume). Ten years give 45 unordered year pairs with gaps of 1 to 9 years. An LDA check found that topic drift across years was negligible.
  • Metric: S = 1 − 2|acc − 0.5|, using an 80:20 split and three repetitions per pair. Temporal neutrality is the standard deviation of S across years. Logistic Regression on the embeddings was run as a robustness check and reproduced the same gap trend.
  • Models tested: Qwen-7B-Chat, Mistral-7B-v0.1, SeaLLM-7B-v2 and DeepSeek-R1-Distill-Qwen-1.5B. Qwen-FT and Mistral-FT were LoRA-tuned in LLaMAFactory on all CoSEM years. Four prompting schemes were compared: Baseline, CoT, Diverse-Decoding and Self-Consistency.
  • Authenticity vs. neutrality: Qwen-FT scored the highest similarity (about 0.67 under Baseline and CoT) but kept clear temporal bias. DeepSeek-R1 looked neutral mostly because its output was generic. Training on all years did not remove era-specific traits; Mistral-FT actually became more variable across years than base Mistral.
  • Direction of change: Older texts read as compressed, basilectal Singlish. Newer ones move toward Standard-English syntax while keeping their identity markers.

Interesting ideas

  • I1. Separability as distance. Train a simple classifier to distinguish two text sets and fold its accuracy around chance into a 0–1 similarity score; this gives a model-agnostic distance between any two corpora.
  • I2. Pair low variance with a realism score. Low variance across conditions only counts as success if the output also stays close to real data, because a bland generator can look "stable" trivially.
  • I3. Rule out topic before blaming style. Run a topic-invariance check such as LDA before attributing a classifier's success to style, so content drift is excluded first.
  • I4. Pool unstable features. Lexicon-based features that are noisy on very short texts are aggregated over larger units instead of computed per message.
  • I5. Group comparisons by gap. Organising comparisons by the size of the gap rather than by individual pairs separates a cumulative trend from one-off quirks of a single period.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Measure how easily a classifier can tell Mova's text → graph → text outputs from the original sentences, using S as a fluency/fidelity gauge.
  • I2 (learn): Add this as a negative control so that parsers or teachers with suspiciously uniform scores are also checked against real-data similarity before being trusted.
  • I3 (tree): When comparing UD treebanks or dialect conversions, verify that genre or topic differences are not what makes the parser's accuracy diverge.
  • I4 (frames): Speech-act or irony cues from short utterances could be pooled per speaker or per document before frames rely on them.
  • I5 (tree): Order treebanks or genres along some axis such as formality, and test whether parser LAS drops steadily with distance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓ETH ZurichZurich · Switzerland72 works · 5,176 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHuman-Computer Interaction · 141 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Linus Tze En Foo, Weihan Angela Ng, Wenkai Li and Lynnette Hui Xian Ng. 2026. Stylistic Evolution and LLM Neutrality in Singlish Language. arXiv:2601.06580.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2601.06580 · arXiv 2601.06580

@misc{foo2026stylistic,
  title = {{Stylistic Evolution and LLM Neutrality in Singlish Language}},
  author = {Linus Tze En Foo, Weihan Angela Ng, Wenkai Li and Lynnette Hui Xian Ng},
  year = {2026},
  doi = {10.48550/arxiv.2601.06580},
  eprint = {2601.06580},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2601.06580}
}

References

34 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Meysam Alizadeh, Maël Kubli, Zeynab Samei, Shirin Dehghani, Mohammadmasiha Zahedivafa, Juan D Bermeo, Maria Korobeynikova, and Fabrizio Gilardi. 2025. Open-source llms for text annotation: a practical guide for model setting and fine-tuning. Journal of Computational Social Science, 8(1):17. ↗
  2. Rajiv Bajpai, Danyuan Ho, and Erik Cambria. 2016. Developing a concept-level knowledge base for sentiment analysis in singlish. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 347–361. Springer.
  3. Wenda Chen, Nancy F Chen, Boon Pang Lim, and Bin Ma. 2015. Corpus-based pronunciation variation rule analysis for singapore english. In SLaTE, pages 35– 40. find
  4. Siew Yeng Chow, Chang-Uk Shin, and Francis Bond. 2024. This word mean what: Constructing a singlish dictionary with chatgpt. In Proceedings of the 2nd Workshop on Resources and Technologies for Indigenous, Endangered and Lesser-resourced Languages in Eurasia (EURALI)@ LREC-COLING 2024, pages 41–50.
  5. Dagmar Deuber, Jakob Leimgruber, and Andrea Sand. 2018. Singaporean internet chit chat compared to informal spoken language: Linguistic variation and indexicality in a language contact situation. Journal of Pidgin and Creole Languages, 33:48–90. ↗
  6. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2014. Diffusion of lexical change in social media. PloS one, 9(11):e113114. ↗
  7. Linus Tze En Foo and Lynnette Hui Xian Ng. 2024. Disentangling singlish discourse particles with task-driven representation. In Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops, pages 1–6.
  8. Wilkinson Daniel Wong Gonzales, Mie Hiramoto, Jakob RE Leimgruber, and Jun Jie Lim. 2023. The corpus of singapore english messages (cosem). World Englishes, 42(2):371–388. find
  9. Amalesh Gope, Anusuya Pal, Sekholu Tetseo, Tulika Gogoi, Dinkur Borah, and 1 others. 2024. Multi-class identification of tonal contrasts in chokri using supervised machine learning algorithms. Humanities and Social Sciences Communications, 11(1):1–13. find
  10. Jack Grieve, Andrea Nini, and Diansheng Guo. 2018. Mapping lexical innovation on american social media. Journal of English Linguistics, 46(4):293–319. ↗
  11. Liangjie Hong and Brian D. Davison. 2010. Empirical study of topic modeling in twitter. In Proceedings of the First Workshop on Social Media Analytics, SOMA ’10, page 80–88, New York, NY, USA. Association for Computing Machinery. find
  12. Lee-Hsun Hsieh, Nam-Chew Chua, Agus Trisnajaya Kwee, Pei-Chi Lo, Yang-Yin Lee, and Ee-Peng Lim. 2022. Singlish checker: A tool for understanding and analysing an english creole language. In International Conference on Asian Digital Libraries, pages 115–124. Springer. find
  13. Wesley Lee. 2016. In defense of singlish: A cultural interpretation of singapore english. The Morningside Review, 12. find
  14. Zhengyuan Liu, Shikang Ni, Aiti Aw, and Nancy Chen. 2022. Singlish message paraphrasing: A joint task of creole translation and text normalization. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3924–3936. find
  15. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30. ↗
  16. R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Linguistics.
  17. Ho Mian-Lian and John T Platt. 1993. Dynamics of a contact continuum: Singaporean english. find
  18. Lynnette Hui Xian Ng and Luo Qi Chan. 2024. What talking you?: Translating code-mixed messaging texts to english. arXiv preprint arXiv:2411.05253. ↗
  19. Lynnette Hui Xian Ng, Iain J Cruickshank, and Roy Lee. 2025. Examining the influence of political bias on large language model performance in stance classification. In Proceedings of the International AAAI Conference on Web and Social Media, volume 19, pages 1315–1328. ↗
  20. Ri Chi Ng, Nirmalendu Prakash, Ming Shan Hee, Kenny Tsu Wei Choo, and Roy Ka-Wei Lee. 2024. Sghatecheck: Functional tests for detecting hate speech in low-resource languages of singapore. arXiv preprint arXiv:2405.01842. ↗
  21. Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Lidong Bing. 2024. SeaLLMs - large language models for Southeast Asia. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 294–304, Bangkok, Thailand. Association for Computational Linguistics. ↗
  22. Nourma Silvia Ningsih and Fadhlur Rahman. 2023. Exploring the unique morphological and syntactic features of singlish (singapore english). Journal of English in Academic and Professional Communication, 9(2):72–80. ↗
  23. Francesco Periti and Stefano Montanelli. 2024. Lexical semantic change through large language models: a survey. ACM Computing Surveys, 56(11):1–38. ↗
  24. Nigora Satibaldieva. 2024. Dynamics of linguistic evolution in the technological epoch. Western European Journal of Linguistics and Education, 2(1):9–11. find
  25. Charlene J. Sato. 1994. Language change in a creole continuum: decreolization?, page 122–144. Cambridge University Press. find
  26. Charlene Tan. 2005. English or singlish? the syntactic influences of chinese and malay on the learning of english in singapore. Journal of Language and Learning, 3(1):156–179. find
  27. Ying-Ying Tan. 2023. The curious case of nomenclatures: Singapore english, singlish, and singaporean english. English Today, 39(3):194–206. ↗
  28. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54. ↗
  29. Jock Wong. 2004. The particles of singapore english: A semantic and cultural interpretation. Journal of Pragmatics, 36(4):739–793. ↗
  30. Tao Wu, Mengze Li, Jingyuan Chen, Wei Ji, Wang Lin, Jinyang Gao, Kun Kuang, Zhou Zhao, and Fei Wu. 2024. Semantic alignment for multimodal large language models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3489– 3498. find
  31. Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Indra Winata, Lintang Sutawika, Jan Christian Blaise Cruz, and 1 others. 2023. Prompting multilingual large language models to generate code-mixed texts: The case of south east asian languages. arXiv preprint arXiv:2303.13592. ↗
  32. Nan Zhao, Dongdong Jiao, Shuotian Bai, and Tingshao Zhu. 2016. Evaluating the validity of simplified chinese version of liwc in detecting psychological expressions in short texts on social network services. PLoS One, 11(6):e0157947. ↗
  33. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372. ↗
  34. Debra Ziegeler. 2020. Changes in the functions of already in singapore english: A grammaticalization approach. Journal of Pidgin and Creole Languages, 35(2):293–331. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/foo-2026-stylistic-evolution-llm-neutrality-singlish/