Elamur Research⌕
☰
home / papers / 2025 / Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding

Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding

Paper · 2025 · Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele et al.

Summary

LLM evaluation has paid little attention to multi-label emotion detection, where one text can carry several emotions at once, and even less to languages other than English. The authors address this with EthioEmo, a newly built benchmark spanning four languages, namely Tigrinya (tir), Amharic (amh), Somali (som) and Afan Oromo (orm), with annotations covering Ekman's six basic emotions as well as a neutral category. They gathered texts from X/Twitter, Facebook, YouTube comments and BBC news headlines, then filtered them with emotion lexicons. Native speakers annotated each text, with at least three annotators per item. Using EthioEmo and the English SemEval 2018 Task 1 data, they compare three approaches: fine-tuned encoder-only models, encoder-decoder models, and decoder-only LLMs prompted in zero- and few-shot settings. They also test prompt variants and English-translated test sets. Fine-tuned Afri-centric encoders whose pre-training included the target languages beat the prompted LLMs. Among generative models, the open Cohere-aya-101 ranked just behind the commercial GPT-4o-mini. The task is still far from solved even for English, and low-resource languages trail considerably. The released data, lexicons and guidelines offer a baseline for culturally grounded emotion research.

Key points

  • The emotion lexicons combine three sources: NRC EmoLex translated via Google Translate and checked by native speakers, nearest neighbours from Word2Vec/FastText embeddings, and manually added keywords. The final lists have 293 (amh), 275 (orm), 283 (som) and 280 (tir) query terms.
  • Texts sampled for annotation number about 6,000 (amh), 5,800 (orm), 5,800 (som) and 6,200 (tir). Each language includes 500 BBC headlines, with the rest drawn from X, Facebook and YouTube.
  • Annotation used a customized POTATO tool, and a label counted as gold when at least two annotators agreed. Agreement was measured with the multi-label agreement (MLA) method of Li et al. (2023).
  • AfroXLMR-76L was the strongest encoder-only model. Overall, fine-tuning smaller encoders still beats few-shot prompting of LLMs.
  • Common errors fall into four groups:
    • predicting a single label when several apply, or the reverse;
    • relying on emotion keywords or emojis rather than overall context;
    • producing no label at all, mainly with encoder-decoder models;
    • over-predicting labels, as with GPT-4o-mini.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany148 works · 18,287 citations🎓Universität HamburgHamburg · Germany79 works · 5,422 citations🎓Instituto Politécnico NacionalMexico City · Mexico18 works · 4,307 citations🎓Bahir Dar UniversityBahir Dar · Ethiopia11 works · 261 citations🎓Wollo UniversityDessie · Ethiopia2 works · 64 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSentiment & emotion · 972 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philip Slusallek, Olga Kolesnikova and Seid Muhie Yimam. 2025. Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{belay2025evaluating,
  title = {{Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding}},
  author = {Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philip Slusallek, Olga Kolesnikova and Seid Muhie Yimam},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.237/}
}

References

77 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, and En-Shiun Lee. 2024a. SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects. In Proceedings of the 18th Conference of the European Chapter of…
  2. David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba O. Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Chukwuneke, Happy Buzaaba, Blessing Sibanda, Godson Kalipe, Jonathan Mukiibi, Salomon Kabongo, Foutse Yuehgoh, Mmasibid… — IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
  3. Rishabh Agarwal, Avi Singh, Lei M. Zhang, Bernd Bohnet, Luis Rosias, Stephanie Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, John D. Co-Reyes, Eric Chu, Feryal Behbahani, Aleksandra Faust, and Hugo Larochelle. 2024. Many-shot in-context learning. arXiv preprint arXiv:2404.11018.
  4. Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. FLAIR: An easy-to-use framework for state-of-the-art NLP. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages…
  5. Iqra Ameer, Noman Ashraf, Grigori Sidorov, and Helena Gómez Adorno. 2020. Multi-label emotion classification using content-based features in twitter. Computación y Sistemas, 24(3):1159–1164. find
  6. Iqra Ameer, Necva Bölücü, Hua Xu, and Ali Al Bataineh. 2023. Findings of WASSA 2023 shared task: Multi-label and multi-class emotion classification on code-mixed text messages. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages… find
  7. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, and others. 2023. PaLM 2 technical report. Preprint, arXiv:2305.10403.
  8. Geetha A.V., Mala T., Priyanka D., and Uma E. 2024. Multimodal emotion recognition with deep learning: Advancements, challenges, and future directions. Information Fusion, 105:102218. find
  9. Abinew Ali Ayele, Seid Muhie Yimam, Tadesse Destaw Belay, Tesfa Asfaw, and Chris Biemann. 2023. Exploring Amharic hate speech data collection and classification approaches. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 49–59, Varna, Bulg…
  10. Tadesse Destaw Belay, Abinew Ali Ayele, Getie Gelaye, Seid Muhie Yimam, and Chris Biemann. 2021. Impacts of homophone normalization on semantic models for Amharic. In 2021 International Conference on Information and Communication Technology for Development for Africa (ICT4DA), pages 101–106, Bahir D… find
  11. Tadesse Destaw Belay, Seid Muhie Yimam, Abinew Ayele, and Chris Biemann. 2022. Question answering classification for Amharic social media community based questions. In Proceedings of the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages, pages 137– 145, Marseill…
  12. Federico Bianchi, Debora Nozza, and Dirk Hovy. 2022. XLM-EMO: Multilingual emotion prediction in social media text. In Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, pages 195–203, Dublin, Ireland. find
  13. Gioele Cageggi, Emanuele Di Rosa, and Asia Uboldi. 2023. App2check at emit: Large language models for multilabel emotion classification. In Proceedings of the Eighth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, Parma, Italy. find
  14. Lihu Chen and Gaël Varoquaux. 2024. What is the role of small models in the LLM era: A survey. Preprint, arXiv:2409.06857.
  15. Alexandra Ciobotaru, Mihai Vlad Constantinescu, Liviu P. Dinu, and Stefan Dumitrescu. 2022. RED v2: Enhancing RED dataset for multi-label emotion detection. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1392– 1399, Marseille, France. find
  16. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. find
  17. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  18. Didar Dadebayev, Wei Wei Goh, and Ee Xion Tan. 2022. Eeg-based emotion recognition: Review of commercial eeg devices and machine learning techniques. Journal of King Saud University - Computer and Information Sciences, 34(7):4385–4401. find
  19. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040–4054, Online. find
  20. Jiawen Deng and Fuji Ren. 2020. Multi-label emotion detection via emotion-specified feature extraction and emotion correlation learning. IEEE Transactions on Affective Computing, 14(1):475–486. find
  21. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  22. Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Emezue. 2022. AfroLM: A self-active learning-based multilingual pretrained language model for 23 African languages. In Proceedings of The Third Worksh… find
  23. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  24. David M. Eberhard, Gary F. Simons, and Charles D. Fennig. 2024. Ethnologue: Languages of the World. Twenty-third edition. Dallas, Texas: SIL International. http://www.ethnologue.com/. [Accessed 01-06-2024]. ↗
  25. Aleksandra Edwards and Jose Camacho-Collados. 2024. Language models for text classification: Is in-context learning enough? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10058–10072, Torino, Italia. find
  26. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion, 6(3-4):169–200.
  27. Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, and Mikel Artetxe. 2023. Do multilingual language models think better in english? arXiv preprint arXiv:2308.01223.
  28. Mauajama Firdaus, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2020. MEISD: A multimodal multi-label emotion, intensity and sentiment dialogue dataset for emotion recognition and sentiment analysis in conversations. In Proceedings of the 28th International Conference on Computational Lingu… find
  29. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  30. Fitsum Gaim, Wonsuk Yang, and Jong C. Park. 2022. GeezSwitch: Language identification in typologically related low-resourced East African languages. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6578–6584, Marseille, France.
  31. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… find
  32. Team Gemma, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and others. 2024. Gemma: Open models based on gemini research and technology. Preprint, arXiv:2403.08295. find
  33. Daiva Goštautaitė and Leonidas Sakalauskas. 2022. Multi-label classification and explanation methods for students’ learning style prediction and interpretation. Applied Sciences, 12(11). find
  34. Chenyang Huang, Amine Trabelsi, Xuebin Qin, Naw-shad Farruque, Lili Mou, and Osmar Zaïane. 2021. Seq2Emo: A sequence to multi-label emotion classification model. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn… find
  35. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… find
  36. Klaus Krippendorff. 2011. Computing Krippendorff’s alpha-reliability. find
  37. Sheetal Kusal, Shruti Patil, Jyoti Choudrie, Ketan Kotecha, Deepali Vora, and Ilias Pappas. 2022. A review on text-based emotion detection–techniques, applications, datasets, and future directions. arXiv preprint arXiv:2205.03235. — [arXiv:2205.03235](https://arxiv.org/abs/2205.03235) find
  38. Sheetal Kusal, Shruti Patil, Jyoti Choudrie, Ketan Kotecha, Deepali Vora, and Ilias Pappas. 2023. A systematic review of applications of natural language processing and future challenges with special emphasis in text-based emotion detection. Artificial Intelligence Review, 56(12):15129–15215. find
  39. Sanaa Laabar and Wajdi Zaghouani. 2024. Multi-dimensional insights: Annotated dataset of stance, sentiment, and emotion in Facebook comments on Tunisia’s July 25 measures. In Proceedings of the Second Workshop on Natural Language Processing for Political Sciences @ LREC-COLING 2024, pages 22–32, Tor… find
  40. Sheng Li, Rong Yan, Qing Wang, Juru Zeng, Xun Zhu, Yueke Liu, and Henghua Li. 2023. Annotation quality measurement in multi-label annotations. In International Conference on Natural Language Processing and Chinese Computing, pages 30–42, Cham, Switzerland. find
  41. Simon Liegl and Marco R Furtner. 2024. Emotional leader communication in the digital age: An experimental investigation on the role of emoji. Computers in Human Behavior, 154:108148. find
  42. Xuan Liu, Tianyi Shi, Guohui Zhou, Mingzhe Liu, Zhengtong Yin, Lirong Yin, and Wenfeng Zheng. 2023. Emotion classification for short texts: an improved multi-label method. Humanities and Social Sciences Communications, 10(1):1–9. find
  43. Zhiwei Liu, Kailai Yang, Tianlin Zhang, Qianqian Xie, and Sophia Ananiadou. 2024. Emollms: A series of emotional large language models and annotation tools for comprehensive affective analysis. Preprint, arXiv:2401.08508. — [arXiv:2401.08508](https://arxiv.org/abs/2401.08508) find
  44. Marian Marchal, Merel Scholman, Frances Yung, and Vera Demberg. 2022. Establishing annotation quality in multi-label annotations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3659–3668, Gyeongju, Republic of Korea. find
  45. Abdullah Al Maruf, Fahima Khanam, Md. Mahmudul Haque, Zakaria Masud Jiyad, M. F. Mridha, and Ze-yar Aung. 2024. Challenges and opportunities of text-based emotion detection: A survey. IEEE Access, 12:18416–18450. find
  46. Meta. 2024. Introducing Meta Llama 3: The most capable openly available LLM to date — ai.meta.com. https://ai.meta.com/blog/ meta-llama-3/. [Accessed 01-06-2024]. ↗
  47. Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval-2018 task 1: Affect in tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 1–17, New Orleans, LA, USA.
  48. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29(3):436–465. find
  49. Shamsuddeen Muhammad, Idris Abdulmumin, Abinew Ayele, Nedjma Ousidhoum, David Adelani, Seid Yimam, Ibrahim Ahmad, Meriem Beloucif, Saif Mohammad, Sebastian Ruder, Oumaima Hourrane, Alipio Jorge, Pavel Brazdil, Felermino Ali, Davis David, Salomey Osei, Bello Shehu-Bello, Falalu Lawan, Tajuddeen Gwada… — AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
  50. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021. Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic.
  51. OpenAI. 2024. GPT-4o mini: advancing cost-efficient intelligence. https://openai.com/. [Accessed Augest-07-2024]. ↗
  52. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and others. 2024. GPT-4 technical report. Preprint, arXiv:2303.08774.
  53. Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, and David Jurgens. 2022. POTATO: The portable text annotation tool. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 327–33…
  54. Justus J. Randolph. 2005. Free-marginal multirater kappa (multirater k[free]): An alternative to fleiss’ fixed-marginal multirater kappa. find
  55. Sahand Sabour, Siyang Liu, Zheyuan Zhang, June M. Liu, Jinfeng Zhou, Alvionna S. Sunaryo, Juanzi Li, Tatia M. C. Lee, Rada Mihalcea, and Minlie Huang. 2024. Emobench: Evaluating the emotional intelligence of large language models. find
  56. Kashfia Sailunaz, Manmeet Dhaliwal, Jon Rokne, and Reda Alhajj. 2018. Emotion detection from text and speech: a survey. Social Network Analysis and Mining, 8(1):28. find
  57. Octavio Sánchez-Velázquez and Gerardo Sierra. 2016. Let’s agree to disagree: Measuring agreement between annotators for opinion mining task. Research in Computing Science, 110:9–19. find
  58. Phakhawat Sarakit, Thanaruk Theeramunkong, Choochart Haruechaiyasak, and Manabu Okumura. 2015. Classifying emotion in thai youtube comments. In 2015 6th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES), pages 1–5. find
  59. Gopendra Vikram Singh, Priyanshu Priya, Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. 2022. EmoInHindi: A multi-label emotion and intensity annotated dataset in Hindi for emotion recognition in dialogues. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages… find
  60. Nicolas Stefanovitch and Jakub Piskorski. 2023. Holistic inter-annotator agreement and corpus coherence estimation in a large-scale multilingual annotation campaign. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 71–86, Singapore. find
  61. Carlo Strapparava and Rada Mihalcea. 2007. SemEval-2007 task 14: Affective text. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 70–74, Prague, Czech Republic.
  62. Jiuding Sun, Chantal Shaib, and Byron C Wallace. 2023. Evaluating the zero-shot robustness of instruction-tuned language models. arXiv preprint arXiv:2306.11270. — [arXiv:2306.11270](https://arxiv.org/abs/2306.11270) find
  63. Jie Tao and Xing Fang. 2020. Toward multi-label sentiment analysis: a transfer learning based approach. Journal of Big Data, 7(1):1–26. find
  64. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, and others. 2022. No language left behind: Scaling human-centered machine translation. Preprint, arXiv:2207.04672. find
  65. Abrhalei Tela, Abraham Woubie, and Ville Hauta-maki. 2020. Transferring monolingual model to low-resource language: the case of tigrinya. arXiv preprint arXiv:2006.07698.
  66. Atnafu Lambebo Tonja, Israel Abebe Azime, Tadesse Destaw Belay, Mesay Gemeda Yigezu, Moges Ahmed Mehamed, Abinew Ali Ayele, Ebrahim Chekol Jibril, Michael Melese Woldeyohan-nis, Olga Kolesnikova, Philipp Slusallek, and others. 2024. EthioLLM: Multilingual large language models for Ethiopian language… find
  67. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  68. Fanfan Wang, Heqing Ma, Rui Xia, Jianfei Yu, and Erik Cambria. 2024a. SemEval-2024 task 3: Multimodal emotion cause analysis in conversations. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 2039–2050, Mexico City, Mexico. find
  69. Kaipeng Wang, Zhi Jing, Yongye Su, and Yikun Han. 2024b. Large language models on fine-grained emotion detection dataset with data augmentation and transfer learning. Preprint, arXiv:2403.06108. — [arXiv:2403.06108](https://arxiv.org/abs/2403.06108) find
  70. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and others. 2024c. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574.
  71. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
  72. Seid Muhie Yimam, Hizkiel Mitiku Alemayehu, Abinew Ayele, and Chris Biemann. 2020. Exploring Amharic sentiment analysis from social media texts: Building annotation tools and classification models. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1048– 1060, Ba…
  73. Seid Muhie Yimam, Abinew Ali Ayele, Gopalakrishnan Venkatesh, Ibrahim Gashaw, and Chris Biemann. 2021. Introducing various semantic models for amharic: Experimentation and evaluation with multiple tasks and datasets. Future Internet, 13(11).
  74. Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba O Alabi, Xiaoyu Shen, Dietrich Klakow, and Marius Mosbach. 2024a. The impact of demonstrations on multilingual in-context learning: A multidimensional analysis. arXiv preprint arXiv:2402.12976. find
  75. Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Pan, and Lidong Bing. 2024b. Sentiment analysis in the era of large language models: A reality check. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3881–3906, Mexico City, Mexico.
  76. Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2023. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. find
  77. Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D’souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Muennighoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. Aya model: An instruction fine…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/belay-2025-evaluating-capabilities-large-language-models/