Elamur Research⌕
☰
home / papers / 2024 / This Word Mean What: Constructing a Singlish Dictionary with ChatGPT

This Word Mean What: Constructing a Singlish Dictionary with ChatGPT

Paper · 2024 · Siew Yeng Chow, Chang-Uk Shin and Francis Bond

Summary

Singlish, Singapore's English-based contact variety, draws vocabulary from Hokkien, Malay, Cantonese, Tamil, Mandarin and others. Existing dictionaries for it are small, closed-licensed or no longer maintained. The authors built an open Singlish dictionary through a hybrid pipeline. First, they scraped only headwords, not descriptions or examples, from the Dictionary of Singlish and Singapore English (DSSE), Wiktionary and the Red Dot Baby Talk wordlist, which kept the result free of licensing restrictions. Next, one fixed ChatGPT prompt per word drafted a definition, part of speech, pronunciation, example and alternate spellings. A native-speaker annotator then corrected each draft against existing sources and her own intuition, normalising spellings and writing pronunciations in X-SAMPA, with tone marked for particles. Four further annotators labelled the generated responses so that agreement could be measured, and disagreement turned out to be high. The resulting dictionary has 1,783 entries under CC-BY-SA. It is presented as a seed resource for Singlish NLP and linguistic research, and the public can extend it.

Key points

  • Final resource: 1,783 entries, each with definition, example sentence, POS, X-SAMPA pronunciation, and where applicable alternate spellings and source language. Licence: CC-BY-SA.
  • Headword sources were DSSE, Wiktionary and Red Dot Baby Talk. Their example texts and descriptions were deliberately left out to keep the licence open.
  • Comparison with earlier resources:
    • Coxford Singlish Dictionary: 809 items.
    • DSSE: over 1,000 entries, but closed and last updated in 2016.
    • singlish.net: about 140 entries.
    • OED: 27 Singlish words as of 2022.
    • None of these is both large and open, and none records particle tones.
  • Drafting used ChatGPT through its API with a single template prompt. Post-editing included changes such as normalising spelling to the standard form (e.g. chochok → cocok).
  • Four extra annotators labelled each response to compute agreement. The high disagreement is attributed to variation in speakers' Singlish knowledge, since the variety is not formally taught.

Interesting ideas

  • I1. Use an LLM only as a draft writer, with a human editor verifying each entry against existing sources and intuition; this speeds up lexicon building without trusting model output blindly.
  • I2. Harvest only headwords from restrictively licensed resources and regenerate all descriptive content, so the derived lexicon can be released openly.
  • I3. Treat high inter-annotator disagreement as a signal of real speaker variation in a non-standardised variety, not just as annotation noise.
  • I4. Identify each word's language of origin in a mixed variety while making clear that such text is not code-switching, because speakers often do not know the source languages.
  • I5. Combine several prompts or several LLMs into competing descriptive hypotheses, then merge them into one entry (a direction the authors propose).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): This fits the LLM-teacher tiers, where the LLM proposes entries or labels and a human or several witness checks accept them before they enter silver data.
  • I2 (world): Compiled knowledge such as slang or dialect lexicons could be built this way without inheriting licence restrictions from source dictionaries.
  • I3 (learn): Evaluation of dialect or informal-text items could flag low-agreement cases as variable rather than forcing a single gold label.
  • I4 (tokens): The tagger and lemmatiser could carry an origin or variety feature for loanwords in informal English, instead of treating them as unknown or foreign tokens.
  • I5 (learn): This mirrors agreement among multiple witnesses and could be applied to LLM-generated glosses for rare words, keeping only the content the hypotheses agree on.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Siew Yeng Chow, Chang-Uk Shin and Francis Bond. 2024. This Word Mean What: Constructing a Singlish Dictionary with ChatGPT. Proceedings of the 2nd Workshop on Resources and Technologies for Indigenous, Endangered and Lesser-resourced Languages in Eurasia (EURALI) @ LREC-COLING 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{chow2024this,
  title = {{This Word Mean What: Constructing a Singlish Dictionary with ChatGPT}},
  author = {Siew Yeng Chow, Chang-Uk Shin and Francis Bond},
  booktitle = {Proceedings of the 2nd Workshop on Resources and Technologies for Indigenous, Endangered and Lesser-resourced Languages in Eurasia (EURALI) @ LREC-COLING 2024},
  year = {2024},
  doi = {10.63317/4z959t34f8vz},
  url = {https://doi.org/10.63317/4z959t34f8vz}
}

References

25 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francis Bond and Ryan Foster. 2013. Linking and extending an open multilingual wordnet. In 51st Annual Meeting of the Association for Computational Linguistics: ACL-2013, pages 1352–1362, Sofia.
  2. Francesco Cavallaro and Bee Chin Ng. 2009. Between status and solidarity in Singapore. World Englishes, 28(2):143–159. ↗
  3. Tao Chen and Min-Yen Kan. 2012. Creating a live, public short message service corpus: the NUS SMS corpus. Language Resources and Evaluation. ↗
  4. Renae Cheng. 2021. 10 bizarre things singaporeans do that the rest of the world won’t understand. [Online; posted 14-June-2021]. find
  5. Siew Yeng Chow and Francis Bond. 2022. Singlish where got rules one? Constructing a computational grammar for Singlish. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5243–5250.
  6. Gilles-Maurice de Schryver. 2023. Generative AI and Lexicography: The Current State of the Art Using ChatGPT. International Journal of Lexicography, 36(4):355–387. ↗
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. ↗
  8. Micha Elsner and Jordan Needle. 2023. Translating a low-resource language using GPT-3 and a human-readable dictionary. In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 1–13, Toronto, Canada. Association for Computational Linguistics. ↗
  9. Christine Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
  10. Colin Goh. 2002. The Coxford Singlish dictionary, second edition. Angsana Books. find
  11. David Huddart. 2014. Declarations of Linguistic Independence: The Postcolonial Dictionary. In Involuntary Associations. Liverpool University Press. ↗
  12. Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and Sivanesan Sangeetha. 2021. Ammus : A survey of transformer-based pretrained models in natural language processing. ↗
  13. Jack Tsen Ta Lee. 2004. A Dictionary of Singlish and Singapore English. find
  14. Jakob RE Leimgruber. 2016. Bah in Singapore English. World Englishes, 35(1):78–97. find
  15. Zhuoyang Li. 2021. An analysis of the linguistic characteristics of Singlish. Journal of Contemporary Educational Research, 5(3). ↗
  16. Lisa Lim. 2011. Tone in Singlish: Substrate features from Sinitic and Malay. Substrate Features in Creole Languages, pages 271–288. find
  17. Ying Qi Soh, Junwen Lee, and Ying-Ying Tan. 2022. Ethnicity and Tone Production on Singlish Particles. Languages, 7(3):243. ↗
  18. Teresa Rebecca Tan. 2010. Singlish. Singapore Infopedia. find
  19. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models.
  20. John C Wells. 1995. Computer-coding the IPA: a proposed extension of SAMPA. University College of London. find
  21. Steven Wilson, Walid Magdy, Barbara McGillivray, Kiran Garimella, and Gareth Tyson. 2020. Urban dictionary embeddings for slang NLP applications. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4764–4773, Marseille, France. European Language Resources Association. ↗
  22. Jock Wong. 2005. “Why you so Singlish one?” A semantic and cultural interpretation of the Singapore English particle one. Language in Society, 34(2):239–275. 10. find
  23. Language Resource References Tao Chen and Min-Yen Kan. 2015. The National University of Singapore SMS Corpus. ICE-Singapore. International corpus of english — singapore. find
  24. OpenAI. 2023. ChatGPT: Large-scale GPT-3.5 for Conversational AI. find
  25. Fei Ting Woon and Suzy Styles. 2021. Red-Dot Baby-Talk Quiz. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chow-2024-this-word-mean-what-constructing/