Elamur Research⌕
☰
home / papers / 2024 / Disentangling Singlish Discourse Particles with Task-Driven Representation

Disentangling Singlish Discourse Particles with Task-Driven Representation

Paper · 2024 · Linus Tze En Foo and Lynnette Hui Xian Ng

Summary

Singlish (Colloquial Singapore English) relies heavily on sentence-final discourse particles whose pragmatic meaning shifts with context. One particle can carry several functions at once. The authors take an early computational step toward separating the functions of three particles: lah, meh and hor. They start from SingBERT and fine-tune it on two objectives. The first is Next Sentence Prediction (NSP), which picks the true follow-up utterance among candidates. The second is Particle Prediction (P-Pred), which recovers a masked sentence-final particle. Training data combines a subset of the National Speech Corpus with Singlish messages from NUS-SMS. To isolate what a particle contributes, they propose Linear-Isolation of Representation (LIR). It subtracts the embedding obtained when the particle is masked from the embedding of the original utterance. The resulting vectors are reduced with t-SNE and clustered with DBSCAN, and the clusters are scored by silhouette. LIR clusters far more cleanly than naive word embeddings. A small GPT-2-based Singlish-to-English translator, GPT-Singlish, is also built. Its high BLEU hides poor translations, which shows that overlap-based metrics mislead for creoles.

Key points

  • Fine-tuned SingBERT results: NSP reaches R10@1 = 0.483, P-Pred reaches 0.790 accuracy, and the NSP model gets R3@1 = 0.767 on a test where particles are swapped.
  • Silhouette scores with LIR are 0.436 for Baseline LIR-word, 0.413 for NSP LIR-word and 0.395 for P-Pred LIR-sentence. Naive word embeddings score only -0.0224, 0.032 and 0.275.
  • The translation corpus has 80,000 Singlish texts from Singapore subreddits and HardwareZone. The authors, who are native speakers, hand-translated 400 of them to create manual-subset-eng.
  • Back-translation with Moses produced 4,500 synthetic pairs. After 2 iterations, BLEU reached 65, yet human inspection found many translations wrong.
  • Sentiment agreement with the human translations (FastText, 100 texts) is 68.1% after translation versus 66.7% with no translation. Tonal information was not modeled, which the authors name as a limitation.

Interesting ideas

  • I1. Mask-and-subtract (LIR): encode a sentence with and without one token and treat the difference vector as that token's isolated contribution, then cluster these vectors to discover its functional uses.
  • I2. Minimal-pair hard negatives: build test items by replacing only the particle in the correct continuation, so the model must use the particle itself rather than topic cues.
  • I3. Three-way candidate labels (true next, topically related, random) make negatives graded in difficulty instead of trivially easy.
  • I4. Extrinsic meaning check against a do-nothing baseline: score whether a property like sentiment survives translation, and compare with skipping translation entirely, because BLEU inflates when source and target share most vocabulary.
  • I5. Tune DBSCAN for silhouette while requiring every particle to own at least one cluster, a simple guard against degenerate unsupervised solutions.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Apply this to English discourse markers and tag questions ("well", "actually", "right?") to propose candidate speech-act or implicature senses that prag can then name with explicit rules.
  • I2 (learn): Use single-function-word substitutions (e.g. "even", "only", "but") as negative controls to check whether a component actually depends on that word.
  • I3 (answer): Generate distractor answers for fable QA and school tests at similar graded difficulty to probe whether readers truly track the story.
  • I4 (back): Evaluate the text → graph → text compressor by whether frames or attributes survive the round trip, with identity copying as the control.
  • I5 (world): When inducing verb-class or entity-type groupings without labels, add a coverage constraint so each known seed category must appear in some cluster.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓University of EdinburghEdinburgh · United Kingdom359 works · 77,352 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Discourse analysis · 466 works

Cite

Cite as Linus Tze En Foo and Lynnette Hui Xian Ng. 2024. Disentangling Singlish Discourse Particles with Task-Driven Representation. arXiv:2409.20366.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2409.20366 · arXiv 2409.20366

@misc{foo2024disentangling,
  title = {{Disentangling Singlish Discourse Particles with Task-Driven Representation}},
  author = {Linus Tze En Foo and Lynnette Hui Xian Ng},
  year = {2024},
  doi = {10.48550/arxiv.2409.20366},
  eprint = {2409.20366},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2409.20366}
}

References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tiago A Almeida, José María G Hidalgo, and Akebo Yamakami. 2011. Contributions to the study of SMS spam filtering: new collection and results. , 259–262 pages. find
  2. Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad, Kazi Samin, Md Saiful Islam, Anindya Iqbal, M Sohel Rahman, and Rifat Shahriyar. 2021. BanglaBERT: Language model pretraining and benchmarks for low-resource language understanding evaluation in Bangla. ↗
  3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching Word Vectors with Subword Information. ↗
  4. Dipam Chakraborty. 2024. Fine Tuning The H2O Danube2 LLM for The Singlish Language. https://h2o.ai/blog/2024/Fine-Tuning-the-H2O-Danube2-LLM-for-the-Singlish-language/ ↗
  5. Tao Chen and Min-Yen Kan. 2013. Creating a live, public short message service corpus: the NUS SMS corpus. Language Resources and Evaluation 47, 2 (01 Jun 2013), 299–335. https://doi.org/10.1007/s10579-012-9197-9 ↗
  6. Wenda Chen, Nancy F Chen, Boon Pang Lim, and Bin Ma. 2015. Corpus-based pronunciation variation rule analysis for singapore English. , 35–40 pages. find
  7. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding Back-Translation at Scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, Brussels, Belgium, 489–500. https://doi.org/10.18653/v1/D18-1045 ↗
  8. James Fodor, Simon De Deyne, and Shinsuke Suzuki. 2023. The Importance of Context in the Evaluation of Word Embeddings: The Effects of Antonymy and Polysemy. , 155–172 pages. https://aclanthology.org/2023.iwcs-1.17 ↗
  9. Wilkinson Daniel Wong Gonzales, Mie Hiramoto, Jakob R. E. Leimgruber, and Jun Jie Lim. 2023. Is it in Colloquial Singapore English: What variation can tell us about its conventions and development. English Today 39, 4 (2023), 243–256. https://doi.org/10.1017/S0266078422000141 ↗
  10. Anthea Fraser Gupta. 1992. The pragmatic particles of Singapore colloquial English. Journal of Pragmatics 18, 1 (1992), 31–57. https://doi.org/10.1016/0378-2166(92)90106-L ↗
  11. Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko, and Jungyun Seo. 2021. Fine-grained Post-training for Improving Retrieval-based Dialogue Systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, 1549–1558. https://www.aclweb.org/anthology/2021.naacl-main.122 ↗
  12. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, Sophia Ananiadou (Ed.). Association for Computational Linguistics, Prague, Czech Republic, 177–180. https://aclanthology.org/P07-2045 ↗
  13. Jia Koh, Aqilah Mislan, Kevin Khoo, Brian Ang, Wilson Ang, Charmaine Ng, and Ying-Ying Tan. 2019. Building the Singapore English National Speech Corpus. , 321-325 pages. https://doi.org/10.21437/Interspeech.2019-1525 ↗
  14. Vivien Ler Soon Lay. 2005. An in-depth study of discourse particles in Singapore English. find
  15. Lisa Lim. 2007. Mergers and acquisitions: On the ages and origins of Singapore English particles. World Englishes 26 (2007), 446–473. find
  16. Zane Lim. 2020. Zanelim/Singbert · hugging face. https://huggingface.co/zanelim/singbert ↗
  17. Zhengyuan Liu, Shikang Ni, Aiti Aw, and Nancy Chen. 2022. Singlish message paraphrasing: A joint task of creole translation and text normalization. , 3924– 3936 pages. find
  18. Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579–2605. http://jmlr.org/papers/v9/vandermaaten08a.html ↗
  19. William Merrill, Zhaofeng Wu, Norihito Naka, Yoon Kim, and Tal Linzen. 2024. Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment. find
  20. Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training data quality and quantity for a low-resource language: New corpus and BERT models for Maltese.
  21. R Parimala and R Nallaswamy. 2012. A Study on Analysis of SMS Classification Using Document Frequency Thresold. International Journal of Information Engineering and Electronic Business 4, 1 (2012), 44. ↗
  22. John Platt. 1987. Communicative functions of particles in Singapore English. https://www.jbe-platform.com/content/books/9789027286239-z.lt2.37pla ↗
  23. Vijay V Raghavan, Peter Bollmann, and Gwang S Jung. 1989. Retrieval system evaluation using recall and precision: Problems and answers. , 59–68 pages. find
  24. Wei Shi and Vera Demberg. 2019. Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 5790–5796. https://doi.org/10.18653/v1/D19-1586 ↗
  25. Maali Tars, Andre Tättar, and Mark Fišel. 2021. Extremely low-resource machine translation for closely related languages. arXiv:2105.13065 [cs.CL] https://arxiv.org/abs/2105.13065 ↗
  26. A Vaswani. 2017. Attention is all you need.
  27. Zihan Wang, K Karthikeyan, Stephen Mayhew, and Dan Roth. 2020. Extending Multilingual BERT to Low-Resource Languages. , 2649–2656 pages. find
  28. Lionel Wee. 2018. The Singlish Controversy: Language, Culture and Identity in a Globalizing World. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/foo-2024-disentangling-singlish-discourse-particles-task/