Elamur Research⌕
☰
home / papers / 2026 / Cross-Lingual Emotion Recognition in Balinese Text using Multilingual-LLMs under Peer-Collaborations Settings

Cross-Lingual Emotion Recognition in Balinese Text using Multilingual-LLMs under Peer-Collaborations Settings

Paper · 2026 · Putu Kussa Laksana Utama, Jilles Steeve Dibangoye and Tsegaye Misikir Tashu

Summary

The paper addresses cross-lingual emotion recognition (CLER) for Balinese, an Austronesian language with almost no labelled data. Earlier multilingual encoders handle such languages poorly. The authors built the first Balinese emotion benchmark, a multi-label corpus. Experts in Balinese linguistics and psychology annotated it using Ekman's six basic emotions. They also propose Multi-Agent Peer Collaboration (MAPC), which splits multi-label prediction into one binary decision per emotion. Several different LLMs make initial judgements, critique and revise each other's answers, and then reach a majority vote. Few-shot prompts use examples from related languages (Indonesian, Javanese, Sundanese) taken from BRIGHTER. LLMs clearly beat the LaBSE encoder. MAPC achieved the best overall macro-F1 of 63.95, ahead of every single-model baseline in both zero-shot and few-shot settings. The gain over the strongest single model is small and not statistically significant per category. Still, the work provides a public resource and shows that collaborating LLMs can help ultra-low-resource languages.

Key points

  • Dataset: BaliEmoV1 is released on Hugging Face (kyo21/BaliEmoV1). It is multi-label and covers anger, disgust, fear, joy, sadness and surprise.
  • Models compared: LaBSE versus Qwen3-32b, Kimi K2-Instruct-0905 and Llama-3.3-70b, evaluated with macro-F1.
  • Model size and prompting: Larger models did not adapt better to few-shot prompting.
    • Llama-3.3-70b led in zero-shot (63.34) but fell to 58.29 with few-shot examples.
    • Qwen3-32b rose to 63.81 with few-shot examples.
  • Effect of review and revision: Each agent's macro-F1 improved:
    • Qwen3-32b: 63.52 → 67.72
    • Kimi K2: 40.13 → 62.47
    • Llama-3.3: 57.43 → 61.66
  • Significance and limitations:
    • McNemar's tests were not significant at p < 0.05 for any emotion; anger came closest (p = 0.07).
    • Majority voting can lock in errors that most agents share.
    • Running multiple agents raises computational cost and latency.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,894 worksSentiment & emotion · 1,035 worksMultilingual & cross-lingual NLP · 882 works

Cite

Cite as Putu Kussa Laksana Utama, Jilles Steeve Dibangoye and Tsegaye Misikir Tashu. 2026. Cross-Lingual Emotion Recognition in Balinese Text using Multilingual-LLMs under Peer-Collaborations Settings. Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.loreslm-1.21

@inproceedings{utama2026cross,
  title = {{Cross-Lingual Emotion Recognition in Balinese Text using Multilingual-LLMs under Peer-Collaborations Settings}},
  author = {Putu Kussa Laksana Utama, Jilles Steeve Dibangoye and Tsegaye Misikir Tashu},
  booktitle = {Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.loreslm-1.21},
  url = {https://doi.org/10.18653/v1/2026.loreslm-1.21}
}

References

53 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AliNadia, TubaishatAbdallah, Al-ObeidatFeras, ShabazMohammad, WaqasMuhammad, HalimZa-hid, RidaImad, and AnwarSajid. 2023. Towards enhanced identification of emotion from resource-constrained language through a novel multilingual bert approach. ACM Transactions on Asian and Low-Resource Language Info… find
  2. Iqra Ameer, Necva Bölücü, Muhammad Hammad Fahim Siddiqui, Burcu Can, Grigori Sidorov, and Alexander Gelbukh. 2023. Multi-label emotion classification in texts using transfer learning. Expert Systems with Applications, 213:118534. find
  3. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, 34:555–596. find
  4. Mobin Barfi, Sajjad Mehrpeyma, and Nasser Mozayani. 2025. Unclelm at semeval-2025 task 11: Rag-based few-shot learning and fine-tuned encoders for multilingual emotion detection. find
  5. Wei Bi and James T Kwok. 2013. Efficient multi-label classification with many labels. find
  6. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology,…
  7. Qingyu Chen, Yan Hu, Xueqing Peng, Qianqian Xie, Qiao Jin, Aidan Gilson, Maxwell B. Singer, Xuguang Ai, Po Ting Lai, Zhizheng Wang, Vipina K. Keloth, Kalpana Raja, Jimin Huang, Huan He, Fongci Lin, Jingcheng Du, Rui Zhang, W. Jim Zheng, Ron A. Adelman, and 2 others. 2025. Benchmarking large language… find
  8. Nature Communications 2025 16:1, 16:3280–. find
  9. Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart De Castilho, and Iryna Gurevych. 2018. The inception platform: Machine-assisted and knowledge-oriented interactive annotation.
  10. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. Proceedings of the Annual Meeting of the Association for Comput…
  11. Klaus Krippendorff. 2011. Computing krippendorff’s alpha-reliability. find
  12. Claudiu Creanga, Teodor George Marchitan, and Liviu P. Dinu. 2025. Team unibuc - nlp at semeval-2025 task 11: Few-shot text-based emotion detection. find
  13. Jian Mao, Kai Huang, and Jinming Liu. 2024. Mlawsmote: Oversampling in imbalanced multi-label classification with missing labels by learning label correlation matrix. International Journal of Computational Intelligence Systems, 17:205–. find
  14. Paul Ekman. 1992. An argument for basic emotions. Cognition & Emotion, 6:169–200.
  15. Viktor Hangya, Hossain Shaikh Saadi, and Alexander Fraser. 2022. Improving low-resource languages in pre-trained multilingual language models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, pages 11993–12006. find
  16. Sabit Hassan, Shaden Shaar, and Kareem Darwish. 2022. Cross-lingual emotion detection. Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6948–6958. find
  17. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with mace. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130.
  18. Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting. Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12365–12…
  19. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations.
  20. Mohammed Jabreel, Antonio Moreno, Mohammed Jabreel, and Antonio Moreno. 2019. A deep learning-based approach for multi-label emotion classification in tweets. find
  21. Applied Sciences 2019, Vol. 9,, 9. find
  22. Ram Mohan Rao Kadiyala. 2024. Cross-lingual emotion detection through large language models. find
  23. WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop, pages 464– 469. find
  24. Xuan Liu, Tianyi Shi, Guohui Zhou, Mingzhe Liu, Zhengtong Yin, Lirong Yin, and Wenfeng Zheng. 2023. Emotion classification for short texts: an improved multi-label method. find
  25. Humanities and Social Sciences Communications 2023 10:1, 10:306–. find
  26. Saif M. Mohammad. 2024. Worrywords: Norms of anxiety association for over 44k english words.
  27. EMNLP 2024 - 2024 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pages 16261–16278. find
  28. Shamsuddeen Hassan Muhammad, Nedjma Ousidhoum, Idris Abdulmumin, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine de Kock, Nirmal Surange, Daniela Teodorescu, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino D. M. A. Ali, Ilseyar Alimova, Vladimir Araujo, Nikolay Babako… find
  29. Muljono, Anisa Sri Winarsih, and Catur Supriyanto. 2016. Evaluation of classification methods for indonesian text emotion detection. Proceedings - 2016 International Seminar on Application of Technology for Information and Communication, ISEMANTIC 2016, pages 130–133. find
  30. Ercong Nie, Sheng Liang, Helmut Schmid, and Hinrich Schütze. 2023. Cross-lingual retrieval augmented prompt for low-resource languages. Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 8320–8340. find
  31. Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12:484–506.
  32. Wongi Park, Inhyuk Park, Sungeun Kim, and Jongbin Ryu. 2023. Robust asymmetric loss for multi-label long-tailed learning. find
  33. Vaidehi Patil, Partha Talukdar, and Sunita Sarawagi. 2022. Overlap-based vocabulary generation improves cross-lingual transfer among related languages. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1:219– 233. find
  34. Robert Plutchik. 1991. The emotions. Bloomsbury Publishing PLC. find
  35. Association for Computational Linguistics, Findings of EACL 2023, pages 1092–1106. find
  36. Oddy Virgantara Putra, Fathin Muhammad Wasmanson, Triana Harmini, and Shoffin Nahwa Utama. 2020. Sundanese twitter dataset for emotion classification. find
  37. CENIM 2020 - Proceeding: International Conference on Computer Engineering, Network, and Intelligent Multimedia 2020, pages 391–395. find
  38. Yuanchi Zhang, Yile Wang, Zijun Liu, Shuo Wang, Xiaolong Wang, Peng Li, Maosong Sun, and Yang Liu. 2024. Enhancing multilingual capabilities of large language models through self-distillation from resource-rich languages. Proceedings of the Annual Meeting of the Association for Computational Linguis…
  39. Gaurish Thakkar, Nives Mikelić Preradović, Marko Tadić, Gaurish Thakkar, Nives Mikelić Preradović, and Marko Tadić. 2024. Transferring sentiment cross-lingually within and across same-family languages. find
  40. Applied Sciences 2024, Vol. 14,, 14. find
  41. Jiaan Wang, Yunlong Liang, Fandong Meng, Beiqi Zou, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2023. Zero-shot cross-lingual summarization via large language models. find
  42. NewSumm 2023 - Proceedings of the 4th New Frontiers in Summarization Workshop, Proceedings of EMNLP Workshop, pages 12–23. find
  43. Jakub Šmíd, Pavel Přibáň, and Pavel Král. 2025. Advancing cross-lingual aspect-based sentiment analysis with llms and constrained decoding for sequence-to-sequence models. International Conference on Agents and Artificial Intelligence, 2:757–766. find
  44. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models.
  45. Trans. Mach. Learn. Res., 2022. find
  46. Feng Xiong, Jun Wang, Geng Tu, and Ruifeng Xu. 2024. Hitsz-hlt at wassa-2024 shared task 2: Language-agnostic multi-task learning for explainability of cross-lingual emotion detection. find
  47. WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop, pages 476–482. find
  48. Zhenran Xu, Senbao Shi, Baotian Hu, Jindi Yu, Dongfang Li, Min Zhang, and Yuxiang Wu. 2023. Towards reasoning in large language models via multi-agent peer review collaboration. Preprint, arXiv:2311.08152.
  49. Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, and Xin Liu. 2025a. JNLP at SemEval-2025 Task 11: Cross-Lingual Multi-Label Emotion Detection Using Generative Models. pages 20–27. find
  50. Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, and Xin Liu. 2025b. Jnlp at semeval-2025 task 11: Cross-lingual multi-label emotion detection using generative models. find
  51. Daniel Yohanes, Jessen Surya Putra, Kenneth Filbert, Kristien Margi Suryaningrum, and Hanis Amalia Saputri. 2023. Emotion detection in textual data using deep learning. Procedia Computer Science, 227:464– 473. find
  52. Ruohong Zhang, Yau Shian Wang, Yiming Yang, Donghan Yu, Tom Vu, and Likun Lei. 2023. Long-tailed extreme multi-label text classification by the retrieval of generated pseudo label descriptions. find
  53. EACL 2023 - 17th Conference of the European Chapter of the *Figure 1: Few-shot prompt example of assessing emotion fear.* *Figure 2: Review prompt example of assesing emotion fear.* *Figure 3: Revision prompt example of assesing emotion fear.* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/utama-2026-cross-lingual-emotion-recognition-balinese/