Elamur Research⌕
☰
home / papers / 2025 / Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses

Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses

Paper · 2025 · Dong Nguyen and Laura Rosseel

Summary

The paper asks whether large language models attach the same social impressions to non-standard English spelling as people do. For example, does "funnnn" or "gr8" make a tweet seem less formal, its author less careful, or younger? The authors adapt the speaker evaluation paradigm from sociolinguistics. They build a small, tightly controlled set of tweets in which each item exists in a conventional and an unconventional spelling, and only the targeted word differs. Human participants recruited via Prolific rate formality and carefulness on a slider and estimate the writer's age. LLMs answer the same questions through prompts that closely mirror the human instructions.

Averaged over items, human and LLM ratings correlate strongly. The agreement weakens when one looks at the shape of the rating distributions or at specific categories of spelling variation. A prompt that shows both spellings side by side, as humans saw them, matches human judgments better than rating each tweet in isolation. The work matters because social meaning is a neglected part of language understanding. It also shows that LLMs are only partly reliable as stand-ins for human participants.

Key points

  • Three social attributes are measured:
    • formality, on a 0–100 scale;
    • carefulness of the writer, on a 0–100 scale;
    • estimated writer age, in years.
  • Seven variation types are covered, with five words each, giving 35 words and two tweets per word:
    • g-dropping (doin);
    • vowel lengthening (cooool);
    • consonant lengthening (funnnn);
    • vowel omission (wknd);
    • letter/number homophones (gr8, 2morrow);
    • transposition typos (becuase);
    • adjacent-key typos (shpuld).
  • Target words are marked with <W>…</W> tags so that both raters and models focus on the manipulated form.
  • Two LLM prompting setups are compared:
    • an independent setup, where one tweet is rated per prompt;
    • a paired setup, where versions A and B are rated together.
    • The paired setup, closer to the human task, yields higher correlations.
  • Human participants were restricted to the UK. The dataset is to be released under CC-BY.

Interesting ideas

  • I1. Build minimal pairs that differ in exactly one surface form while keeping meaning constant, so any change in judgment can be attributed to that form alone. This avoids topic confounds that plagued earlier author-profiling perception studies.
  • I2. Distinguish informal-but-intentional variants (lengthening, g-dropping, number homophones) from accidental typos (transpositions, adjacent keys), since readers interpret them differently.
  • I3. Deliberate non-standard spelling carries social meaning (informality, youth, emotional emphasis) on top of the referential content, rather than being mere noise.
  • I4. When using an LLM as a stand-in judge, mirror the human task format closely, for example by showing contrastive pairs side by side. Even then, compare whole rating distributions with human data, not only correlations.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation sets could use such meaning-equivalent pairs (gr8/great, doin/doing) as negative controls, checking that parses and frames stay identical when only spelling changes.
  • I2 (tokens): The tokenizer/lemmatizer could normalize both kinds to the same lemma but keep a feature recording whether the variant looks deliberate or like a slip.
  • I3 (frames): The prag crate could turn lengthening like "funnnn" or "helpppp" into an intensity/emphasis or stance cue on the speech act, instead of discarding it after normalization.
  • I4 (learn): LLM teacher tiers could be prompted with paired alternatives, and their outputs audited for distributional bias, before being accepted as silver labels.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksHuman-Computer Interaction · 160 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEnglish · 2,379 works

Cite

Cite as Dong Nguyen and Laura Rosseel. 2025. Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses. arXiv:2511.23041.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2511.23041 · arXiv 2511.23041

@misc{nguyen2025social,
  title = {{Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses}},
  author = {Dong Nguyen and Laura Rosseel},
  year = {2025},
  doi = {10.48550/arxiv.2511.23041},
  eprint = {2511.23041},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2511.23041}
}

References

124 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdurahman, Suhaib, Mohammad Atari, Farzan Karimi-Malekabadi, Mona J Xue, Jackson Trager, Peter S Park, Preni Golazizian, Ali Omrani, and Morteza Dehghani. 2024. Perils and opportunities in using large language models in psychological research. PNAS Nexus, 3(7):pgae245. find
  2. Agnew, William, A. Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jaylen Pittman, Shakir Mohamed, and Kevin R. McKee. 2024. The illusion of artificial inclusion. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, Association for Computing Machinery, New York, NY, USA. find
  3. Aher, Gati V, Rosa I. Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 337–371, PMLR. find
  4. Anderson, Tyler Kimball and Almeida Jacqueline Toribio. 2007. Attitudes towards lexical borrowing and intra-sentential code-switching among Spanish-English bilinguals. Spanish in Context, 4(2):217–240. find
  5. Androutsopoulos, Jannis K. 2000. Non-standard spellings in media texts: The case of German fanzines. Journal of Sociolinguistics, 4(4):514–533. find
  6. Anthis, Jacy Reese, Ryan Liu, Sean M Richardson, Austin C Kozlowski, Bernard Koch, Erik Brynjolfsson, James Evans, and Michael S Bernstein. 2025. Position: LLM social simulations are a promising research method. In Forty-second International Conference on Machine Learning Position Paper Track. find
  7. Argamon, Shlomo, Moshe Koppel, James W. Pennebaker, and Jonathan Schler. 2009. Automatically profiling the author of an anonymous text. Commun. ACM, 52(2):119–123. find
  8. Argyle, Lisa P., Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.
  9. Bai, Xuechunzi, Angelina Wang, Ilia Sucholutsky, and Thomas L. Griffiths. 2025. Explicitly unbiased large language models still form biased associations. Proceedings of the National Academy of Sciences, 122(8):e2416228122. find
  10. Bamman, David, Jacob Eisenstein, and Tyler Schnoebelen. 2014. Gender identity and lexical variation in social media. Journal of Sociolinguistics, 18(2):135–160. find
  11. Beck, Tilman, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. 2024. Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2589–2615, Association for Computational Linguistics, St. Julian’s, Malta.
  12. Binz, Marcel, Elif Akata, Matthias Bethge, Franziska Brändle, Fred Callaway, Julian Coda-Forno, Peter Dayan, Can Demircan, Maria K Eckstein, Noémi Éltető, et al. 2025. A foundation model to predict and capture human cognition. Nature, pages 1–8. find
  13. Blodgett, Su Lin, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Association for Computational Linguistics, Online.
  14. Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
  15. Brysbaert, Marc, Gonzalo Martínez, and Pedro Reviriego. 2024. Moving beyond word frequency based on tally counting: AI-generated familiarity estimates of words and phrases are an interesting additional index of language knowledge. Behavior Research Methods, 57(1):28. find
  16. Buchstaller, Isabelle. 2006. Social stereotypes, personality traits and regional perception displaced: Attitudes towards the ‘new’ quotatives in the U.K. Journal of Sociolinguistics, 10(3):362–381. find
  17. Bui, Minh Duc, Carolin Holtermann, Valentin Hofmann, Anne Lauscher, and Katharina von der Wense. 2025. Large language models discriminate against speakers of German dialects. In EMNLP 2025. find
  18. Busch, Florian. 2021. Enregistered spellings in interaction: Social indexicality in digital written communication. Zeitschrift für Sprachwissenschaft, 40(3):297–323. find
  19. Campbell-Kibler, Kathryn. 2009. The nature of sociolinguistic perception. Language Variation and Change, 21(1):135–156. find
  20. Campbell-Kibler, Kathryn. 2010a. The effect of speaker information on attitudes toward (ing). Journal of Language and Social Psychology, 29(2):214–223. find
  21. Campbell-Kibler, Kathryn. 2010b. Sociolinguistics and perception. Language and Linguistics Compass, 4(6):377–389. find
  22. Campbell-Kibler, Kathryn. 2014. Accent, (ing), and the social logic of listener perceptions. American Speech, 82(1):2–64. find
  23. Chen, Hongyu, Neele Falk, Michael Roth, and Agnieszka Falenskaa. 2025. “Feels feminine to me”: Understanding perceived gendered style through human annotations. In EMNLP2025. find
  24. Cheng, Myra, Sunny Yu, and Dan Jurafsky. 2025. HumT DumT: Measuring and controlling human-like language in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25983–26008, Association for Computational Linguistics, Vienna, Austria. find
  25. Cheung, Vanessa, Maximilian Maier, and Falk Lieder. 2025. Large language models show amplified cognitive biases in moral decision-making. Proceedings of the National Academy of Sciences, 122(25):e2412015122. find
  26. Choudhury, Monojit, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal of Document Analysis and Recognition (IJDAR), 10:157–174. find
  27. Conde, Javier, Miguel González Saiz, María Grandury, Pedro Reviriego, Gonzalo Martínez, and Marc Brysbaert. 2025. Psycholinguistic word features: a new approach for the evaluation of LLMs alignment with humans. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²), pages 8–17, Association for Computational Linguistics, Vienna, Austria and virtual meeting. find
  28. Contractor, Danish, Tanveer A. Faruquie, and L. Venkata Subramaniam. 2010. Unsupervised cleansing of noisy text. In Coling 2010: Posters, pages 189–196, Coling 2010 Organizing Committee, Beijing, China. find
  29. Cook, Paul and Suzanne Stevenson. 2009. An unsupervised model for text message normalization. In Proceedings of the Workshop on Computational Approaches to Linguistic Creativity, pages 71–78, Association for Computational Linguistics, Boulder, Colorado. find
  30. Darics, Erika. 2013. Non-verbal signalling in digital discourse: The case of letter repetition. Discourse, Context & Media, 2(3):141–148. find
  31. Deas, Nicholas, Jessica Grieser, Shana Kleiner, Desmond Patton, Elsbeth Turcan, and Kathleen McKeown. 2023. Evaluation of African American language bias in natural language generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6805–6824, Association for Computational Linguistics, Singapore. find
  32. DeepSeek-AI. 2025. Deepseek-V3 technical report. Arxiv preprint arXiv:2412.19437. ↗
  33. Dillion, Danica, Niket Tandon, Yuling Gu, and Kurt Gray. 2023. Can AI language models replace human participants? Trends in Cognitive Sciences. find
  34. Duncan, Daniel. 2024. Does ChatGPT have sociolinguistic competence? Journal of Computer-Assisted Linguistic Research, 8:51–75. find
  35. Durmus, Esin, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the representation of subjective global opinions in language models. In First Conference on Language Modeling.
  36. Eckert, Penelope. 2008. Variation and the indexical field. Journal of Sociolinguistics, 12(4):453–476. find
  37. Eckert, Penelope. 2012. Three waves of variation study: The emergence of meaning in the study of sociolinguistic variation. Annual Review of Anthropology, 41(1):87–100. find
  38. Eisenstein, Jacob. 2015. Systematic patterning in phonologically-motivated orthographic variation. Journal of Sociolinguistics, 19(2):161–188. find
  39. Ersoy, Asım, Gerson Vizcarra, Tahsin Mayeesha, and Benjamin Muller. 2023. In what languages are generative language models the most formal? Analyzing formality distribution across languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2650–2666, Association for Computational Linguistics, Singapore. find
  40. Faisal, Fahim, Orevaoghene Ahia, Aarohi Srivastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios Anastasopoulos. 2024. DIALECTBENCH: An NLP benchmark for dialects, varieties, and closely-related languages. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14412–14454, Association for Computational Linguistics, Bangkok, Thailand. find
  41. Fleisig, Eve, Genevieve Smith, Madeline Bossi, Ishita Rustagi, Xavier Yin, and Dan Klein. 2024. Linguistic bias in ChatGPT: Language models reinforce dialect discrimination. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13541–13564, Association for Computational Linguistics, Miami, Florida, USA.
  42. Flekova, Lucie, Jordan Carpenter, Salvatore Giorgi, Lyle Ungar, and Daniel Preoţiuc-Pietro. 2016. Analyzing biases in human perception of user age and gender from text. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 843–854, Association for Computational Linguistics, Berlin, Germany. find
  43. Flekova, Lucie, Daniel Preoţiuc-Pietro, and Lyle Ungar. 2016. Exploring stylistic variation with age and income on Twitter. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 313–319, Association for Computational Linguistics, Berlin, Germany. find
  44. Fuchs, Susanne, Egor Savin, Stephanie Solt, Cornelia Ebert, and Manfred Krifka. 2019. Antonym adjective pairs and prosodic iconicity: Evidence from letter replications in an English blogger corpus. Linguistics Vanguard, 5(1):20180017. find
  45. Ganuza, Natalia and Maria Rydell. 2024. Turning talk into text: The representation of contemporary urban vernaculars in Swedish fiction. Text & Talk, 45(3):319–339. find
  46. Gao, Leo, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. ↗
  47. Garimella, Aparna, Carmen Banea, Dirk Hovy, and Rada Mihalcea. 2019. Women’s syntactic resilience and men’s grammatical luck: Gender-bias in part-of-speech tagging and dependency parsing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3493–3498, Association for Computational Linguistics, Florence, Italy. find
  48. Garrett, Peter. 2010. Attitudes to Language. Key Topics in Sociolinguistics. Cambridge University Press. find
  49. Garrett, Peter, Angie Williams, and Betsy Evans. 2005. Accessing social meanings: Values of keywords, values in keywords. Acta Linguistica Hafniensia, 37(1):37–54. Gemma Team and others. 2024. Gemma 2: Improving open language models at a practical size. find
  50. Goldshtein, Maria, Jaclyn Ocumpaugh, Andrew Potter, and Rod D. Roscoe. 2024. The social consequences of language technologies and their underlying language ideologies. In Universal Access in Human-Computer Interaction, pages 271–290, Springer Nature Switzerland, Cham. van der Goot, Rob, Rik van Noord, and Gertjan van Noord. 2018. A taxonomy for in-depth evaluation of normalization for user generated content. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), European Language Resources Association (ELRA), Miyazaki, Japan. van der Goot, Rob, Alan Ramponi, Arkaitz Zubiaga, Barbara Plank, Benjamin Muller, Iñaki San Vicente Roncal, Nikola Ljubešić, Özlem Çetinoğlu, Rahmad Mahendra, Talha Çolakoğlu, Timothy Baldwin, Tommaso Caselli, and Wladimir Sidorenko. 2021. MultiLexNorm: A shared task on multilingual lexical normalization. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 493–509, Association for Computational Linguistics, Online. find
  51. Grattafiori, Aaron, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, et al. 2024. The Llama 3 herd of models.
  52. Grondelaers, Stefan, Roeland van Hout, Hans van Halteren, and Esther Veerbeek. 2023. Why do we say them when we know it should be they? Twitter as a resource for investigating nonstandard syntactic variation in The Netherlands. Language Variation and Change, 35(2):223–245. find
  53. Grondelaers, Stefan, Dirk Speelman, Chloé Lybaert, and Paul van Gent. 2020. Getting a (big) data-based grip on ideological change. evidence from Belgian Dutch. Journal of Linguistic Geography, 8(1):49–65. find
  54. Hilte, Lisa, Reinhild Vandekerckhove, and Walter Daelemans. 2019. Adolescents’ perceptions of social media writing: Has non-standard become the new standard? European Journal of Applied Linguistics, 7(2):189–224. find
  55. Hinrichs, Lars and Jessica White-Sustaíta. 2011. Global Englishes and the sociolinguistics of spelling: A study of Jamaican blog and email writing. English World-Wide, 32(1):46–73. find
  56. Hofmann, Valentin, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. AI generates covertly racist decisions about people based on their dialect. Nature, 633(8028):147–154. find
  57. Holliday, Nicole and Marie Tano. 2021. “It’s a whole vibe”: testing evaluations of grammatical and ungrammatical AAE on Twitter. Linguistics Vanguard, 7(1):20200095. find
  58. Honeybone, Patrick and Kevin Watson. 2013. Salience and the sociolinguistics of scouse spelling: Exploring the phonology of the contemporary humorous localised dialect literature of Liverpool. Journal of Varieties of English, 34(3):305–340. find
  59. Hu, Jennifer, Sammy Floyd, Olessia Jouravlev, Evelina Fedorenko, and Edward Gibson. 2023. A fine-grained comparison of pragmatic language understanding in humans and language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4194–4213, Association for Computational Linguistics, Toronto, Canada.
  60. Ilbury, Christian. 2020. “Sassy Queens”: Stylistic orthographic variation in Twitter and the enregisterment of AAVE. Journal of Sociolinguistics, 24(2):245–264. find
  61. Ilbury, Christian, Jack Grieve, and David Hall. 2024. Using social media to infer the diffusion of an urban contact dialect: A case study of Multicultural London English. Journal of Sociolinguistics, 28(3):45–70. find
  62. Jaffe, Alexandra, Jannis Androutsopoulos, Mark Sebba, and Sally Johnson (eds.). 2012. Orthography as Social Action: Scripts, Spelling, Identity and Power. De Gruyter Mouton. find
  63. Jin, Di, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155–205. find
  64. Joshi, Aditya, Raj Dabre, Diptesh Kanojia, Zhuang Li, Haolan Zhan, Gholamreza Haffari, and Doris Dippold. 2025. Natural language processing for dialects of a language: A survey. ACM Comput. Surv., 57(6). find
  65. Kang, Dongyeop and Eduard Hovy. 2021. Style is NOT a single variable: Case studies for cross-stylistic language understanding. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2376–2387, Association for Computational Linguistics, Online. find
  66. Karamolegkou, Antonia, Mostafa Abdou, and Anders Søgaard. 2023. Mapping brains with language models: A survey. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9748–9762, Association for Computational Linguistics, Toronto, Canada. find
  67. Keuleers, Emmanuel and Marc Brysbaert. 2010. Wuggy: A multilingual pseudoword generator. Behavior research methods, 42(3):627–633. find
  68. Kircher, Ruth and Lena Zipp. 2022. Research Methods in Language Attitudes. Cambridge University Press. find
  69. Labov, Willian. 1972. Sociolinguistic patterns. Philadelphia University Press. find
  70. Lambert, Wallace E, Richard C Hodgson, Robert C Gardner, and Samuel Fillenbaum. 1960. Evaluational reactions to spoken languages. The journal of abnormal and social psychology, 60(1):44. find
  71. Lauscher, Anne, Federico Bianchi, Samuel R. Bowman, and Dirk Hovy. 2022. SocioProbe: What, when, and where language models learn about sociodemographics. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7901–7918, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates. find
  72. Leigh, Daisy. 2018. Expecting a performance: Listener expectations of social meaning in social media. Presented at New Ways of Analyzing Variation (NWAV) 47. find
  73. Lin, Fangru, Shaoguang Mao, Emanuele La Malfa, Valentin Hofmann, Adrian de Wynter, Xun Wang, Si-Qing Chen, Michael J. Wooldridge, Janet B. Pierrehumbert, and Furu Wei. 2025. Assessing dialect fairness and robustness of large language models in reasoning tasks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6317–6342, Association for Computational Linguistics, Vienna, Austria. find
  74. Liu, Fei, Fuliang Weng, Bingqing Wang, and Yang Liu. 2011. Insertion, deletion, or substitution? Normalizing text messages without pre-categorization nor supervision. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 71–76, Association for Computational Linguistics, Portland, Oregon, USA. find
  75. Llamas, Carmen and Dominic Watt. 2014. Scottish, English, British?: Innovations in attitude measurement. Language and Linguistics Compass, 8(11):610–617. find
  76. Loureiro-Rodríguez, Verónica and Elif Fidan Acar. 2022. The matched-guise technique. In Ruth Kircher and LenaEditors Zipp, editors, Research Methods in Language Attitudes. Cambridge University Press, page 185–202. find
  77. Lourentzou, Ismini, Kabir Manghnani, and ChengXiang Zhai. 2019. Adapting sequence to sequence models for text normalization in social media. Proceedings of the International AAAI Conference on Web and Social Media, 13(01):335–345. find
  78. Lucy, Li, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, and Alexandra Olteanu. 2024. “One-size-fits-all”? Examining expectations around what constitute “fair” or “good” NLG system behaviors. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1054–1089, Association for Computational Linguistics, Mexico City, Mexico. find
  79. Lutz, Marlene, Indira Sen, Georg Ahnert, Elisa Rogers, and Markus Strohmaier. 2025. The prompt makes the person(a): A systematic evaluation of sociodemographic persona prompting for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23212–23237, Association for Computational Linguistics, Suzhou, China. find
  80. Malik, Manuj, Jing Jiang, and Kian Ming A. Chai. 2024. An empirical analysis of the writing styles of persona-assigned LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19369–19388, Association for Computational Linguistics, Miami, Florida, USA. find
  81. Martínez, Gonzalo, Juan Diego Molero, Sandra González, Javier Conde, Marc Brysbaert, and Pedro Reviriego. 2024. Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal. Behavior Research Methods, 57(1):5. find
  82. McCoy, R. Thomas, Shunyu Yao, Dan Friedman, Mathew D. Hardy, and Thomas L. Griffiths. 2024. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences, 121(41):e2322420121. find
  83. Nguyen, Dong. 2025. Collaborative growth: When large language models meet sociolinguistics. Language and Linguistics Compass, 19(2):e70010. find
  84. Nguyen, Dong, A. Seza Doğruöz, Carolyn P. Rosé, and Franciska de Jong. 2016. Computational sociolinguistics: A survey. Computational Linguistics, 42(3):537–593.
  85. Nguyen, Dong and Jack Grieve. 2020. Do word embeddings capture spelling variation? In Proceedings of the 28th International Conference on Computational Linguistics, pages 870–881, International Committee on Computational Linguistics, Barcelona, Spain (Online). find
  86. Nguyen, Dong, Laura Rosseel, and Jack Grieve. 2021. On learning and representing social meaning in NLP: a sociolinguistic perspective. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 603–612, Association for Computational Linguistics, Online. find
  87. Nguyen, Dong, Dolf Trieschnigg, A. Seza Doğruöz, Rilana Gravel, Mariët Theune, Theo Meder, and Franciska de Jong. 2014. Why gender and age prediction from tweets is hard: Lessons from a crowdsourcing experiment. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1950–1961, Dublin City University and Association for Computational Linguistics, Dublin, Ireland. find
  88. Nielsen, Elizabeth, Christo Kirov, and Brian Roark. 2023. Spelling convention sensitivity in neural language models. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1334–1346, Association for Computational Linguistics, Dubrovnik, Croatia. OpenAI team. 2023. GPT-4 technical report. ArXiv preprint arXiv:2303.08774. ↗
  89. Pavlick, Ellie and Joel Tetreault. 2016. An empirical analysis of formality in online communication. Transactions of the Association for Computational Linguistics, 4:61–74.
  90. Pennell, Deana and Yang Liu. 2011. A character-level machine translation approach for normalization of SMS abbreviations. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 974–982, Asian Federation of Natural Language Processing, Chiang Mai, Thailand. find
  91. Petrov, Aleksandar, Emanuele La Malfa, Philip Torr, and Adel Bibi. 2023. Language model tokenizers introduce unfairness between languages. In Thirty-seventh Conference on Neural Information Processing Systems.
  92. Preoţiuc-Pietro, Daniel, Sharath Chandra Guntuku, and Lyle Ungar. 2017. Controlling human perception of basic user traits. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2335–2341, Association for Computational Linguistics, Copenhagen, Denmark. find
  93. Rao, Sudha and Joel Tetreault. 2018. Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129–140, Association for Computational Linguistics, New Orleans, Louisiana.
  94. Russ, Brice. 2012. Social meaning in social media: Perceptual judgments of orthographic variation on Facebook and Twitter. The Ohio State University Second Qualifying Paper Talk. find
  95. Rácz, Peter, Jennifer B. Hay, and Janet B. Pierrehumbert. 2017. Social salience discriminates learnability of contextual cues in an artificial language. Frontiers in Psychology, 8. find
  96. Samara, Anna, Kenny Smith, Helen Brown, and Elizabeth Wonnacott. 2017. Acquiring variation in an artificial language: Children and adults are sensitive to socially conditioned linguistic variation. Cognitive Psychology, 94:85–114. find
  97. Sandoval, Sandra Camille, Christabel Acquaye, Kwesi Adu Cobbina, Mohammad Nayeem Teli, and Hal Daumé Iii. 2025. My LLM might mimic AAE - but when should it? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5277–5302, Association for Computational Linguistics, Albuquerque, New Mexico.
  98. Santurkar, Shibani, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org.
  99. Schwartz, H. Andrew, Johannes C. Eichstaedt, Margaret L. Kern, Lukasz Dziurzynski, Stephanie M. Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin E. P. Seligman, and Lyle H. Ungar. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PLOS ONE, 8(9):1–16. find
  100. Sclar, Melanie, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations. find
  101. Sebba, Mark. 2007. Spelling and society: The culture and politics of orthography around the world. Cambridge University Press. find
  102. Shortis, Timothy Francis John. 2016. Orthographic practices in SMS text messaging as a case signifying diachronic change in linguistic and semiotic resources. Ph.D. thesis, UCL (University College London). find
  103. Shu, Bangzhao, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don‘t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5263–5281, Association for Computational Linguistics, Mexico City, Mexico. find
  104. Soldaini, Luca, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. 2024. Dolma: an open corpus of three trillion tokens for language model pretraining research. find
  105. Soukup, Barbara. 2013. On matching speaker (dis)guises – revisiting a methodological tradition. Novus. find
  106. Sourati, Zhivar, Filip Ilievski, Pia Sommerauer, and Yifan Jiang. 2024. ARN: Analogical reasoning on narratives. Transactions of the Association for Computational Linguistics, 12:1063–1086.
  107. Sourati, Zhivar, Farzan Karimi-Malekabadi, Meltem Ozcan, Colin McDaniel, Alireza Ziabari, Jackson Trager, Ala Tak, Meng Chen, Fred Morstatter, and Morteza Dehghani. 2025. The shrinking landscape of linguistic diversity in the age of large language models. find
  108. Speed, Laura J., Ewelina Wnuk, and Asifa Majid. 2017. Studying Psycholinguistics out of the Lab, chapter 10. John Wiley & Sons, Ltd. find
  109. Squires, Lauren. 2010. Enregistering internet language. Language in Society, 39(4):457–492. find
  110. Sterner, Igor and Simone Teufel. 2025. Minimal pair-based evaluation of code-switching. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18575–18598, Association for Computational Linguistics, Vienna, Austria.
  111. Tagg, Caroline. 2009. A corpus linguistics study of SMS text messaging. Ph.D. thesis, University of Birmingham. find
  112. Tjuatja, Lindia, Valerie Chen, Tongshuang Wu, Ameet Talwalkwar, and Graham Neubig. 2024. Do LLMs exhibit human-like response biases? A case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011–1026.
  113. Verheijen, Lieke. 2018. Orthographic principles in computer-mediated communication: The SUPER-functions of textisms and their interaction with age and medium. Written Language & Literacy, 7:111–145. find
  114. Vosters, Rik, Gijsbert Rutten, Marijke Van der Wal, and Wim Vandenbussche. 2012. Spelling and identity in the Southern Netherlands (1750–1830), chapter 6. De Gruyter Mouton. find
  115. Waldis, Andreas, Yotam Perlitz, Leshem Choshen, Yufang Hou, and Iryna Gurevych. 2024. Holmes: A benchmark to assess the linguistic competence of language models. Transactions of the Association for Computational Linguistics, 12:1616–1647. find
  116. Walker, Abby, Christina García, Yomi Cortés, and Kathryn Campbell-Kibler. 2014. Comparing social meanings across listener and speaker groups: The indexical field of Spanish /s/. Language Variation and Change, 26(2):169–189. find
  117. Wang, Angelina, Jamie Morgenstern, and John P Dickerson. 2025. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence, 7:pages 400––411. find
  118. Warstadt, Alex, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377–392.
  119. Wong, Andrew D. 2013. Brand names and unconventional spelling: A two-pronged analysis of the orthographic construction of brand identity. Written Language & Literacy, 16(2):115–145. find
  120. Yang, Yi and Jacob Eisenstein. 2013. A log-linear model for unsupervised text normalization. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 61–72, Association for Computational Linguistics, Seattle, Washington, USA. find
  121. Zacharopoulos, Christos, Théo Desbordes, and Mathias Sablé-Meyer. 2023. Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16081–16090, Association for Computational Linguistics, Singapore.
  122. Zenner, Eline, Laura Rosseel, and Dirk Speelman. 2021. Starman or Sterrenman: An acquisitional perspective on the social meaning of English in Flanders. International Journal of Bilingualism, 25(3):568–591. find
  123. Zhang, Sheng, Xin Zhang, Weiming Zhang, and Anders Søgaard. 2021. Sociolectal analysis of pretrained language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4581–4588, Association for Computational Linguistics, Online and Punta Cana, Dominican Republic. find
  124. Ziems, Caleb, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. Multi-VALUE: A framework for cross-dialectal English NLP. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Association for Computational Linguistics, Toronto, Canada.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2025-social-perceptions-english-spelling-variation/