Elamur Research⌕
☰
home / papers / 2025 / NTA at SemEval-2025 Task 11: Enhanced Multilingual Textual Multi-label Emotion Detection via Integrated Augmentation Learning

NTA at SemEval-2025 Task 11: Enhanced Multilingual Textual Multi-label Emotion Detection via Integrated Augmentation Learning

Paper · 2025 · Nguyen Pham Hoang Le, An Nguyen Tran Khuong, Tram Nguyen Thi Ngoc et al.

Summary

The NTA team entered Track A of SemEval-2025 Task 11, which asks systems to decide which of six perceived emotions (joy, sadness, fear, anger, surprise, disgust; English omits disgust) a short text expresses, across 28 languages. Many of these languages, such as Tigrinya and Emakhuwa, have very little labelled data. The team tackled this scarcity mainly through data augmentation:

  • random word swaps and deletions;
  • oversampling by joining two samples and merging their labels;
  • inserting emotion words from the NRC Emotion Lexicon;
  • for English only, WordNet synonym replacement filtered by NRC emotion checks.

Light preprocessing came first: lowercasing, whitespace cleanup, and English contraction and slang expansion. The classifiers were ensembles of fine-tuned transformers: DeBERTa-v3, XLNet and RoBERTa for English, and mBERT, mXLM-R and AfroXLMR-large for other languages. Their outputs were combined by hard or soft voting, and the sigmoid decision threshold was tuned on validation data.

Augmentation lifted overall macro F1 from 0.4965 to 0.5233. The system placed 17th overall and won Tigrinya. However, it fell below the baseline on several high-resource languages, which suggests augmentation mostly helps where data is thin.

Key points

  • Augmentation improved macro F1 in 21 of 28 languages, raising the overall score from 0.4965 to 0.5233 (+0.0267). On the private test the system scored 0.5233 against a baseline of 0.5093.
  • The system took first place in Tigrinya (0.5905 vs. baseline 0.4628). It also gained strongly on Oromo (0.5048 vs. 0.1263), Yoruba and Emakhuwa.
  • It underperformed the baseline on languages such as ary, deu, ptbr, ptmz, ron and kin. Ukrainian came out roughly level (0.5335 vs. 0.5345).
  • Hard voting was used for English and African languages, where at least three models were available; all other languages used soft voting.
  • The threshold was searched over 0.40–0.60 by minimizing BCE on an 80:20 validation split (seed 42). Training used HF Trainer, lr 2e-5, AdamW with cosine schedule, up to 20 epochs, and early stopping with patience 3.

Interesting ideas

  • I1. Two labelled samples are joined into one text whose label is the union of both label sets, producing cheap multi-label examples from single-label ones.
  • I2. Synonym swaps are accepted only if an emotion lexicon confirms the replacement keeps the label, so augmentation stays label-preserving rather than blind.
  • I3. Random word swaps and deletions served as noise to teach order-insensitivity. Run in reverse, the same perturbations reveal whether a model actually uses structure.
  • I4. The voting rule depended on how many component models existed: majority vote needs three or more, otherwise probabilities are averaged.
  • I5. The decision threshold was treated as a tuned hyperparameter chosen by validation loss instead of fixed at 0.5.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Concatenate two short texts and check that the resulting world holds exactly the union of entities and events from each part. This gives a free compositionality test for the world builder.
  • I2 (tree): Substitute lemmas with WordNet or VerbNet-class siblings of the same UPOS to multiply treebank sentences while keeping the gold UD tree unchanged. A frame-consistency check should reject any swap that alters argument structure.
  • I3 (learn): Use scrambled and word-dropped versions of test texts as negative controls. If answers survive word shuffling unchanged, that flags hidden bag-of-words behaviour.
  • I4 (learn): When building silver data from witness parsers, choose strict majority agreement for arcs only if at least three witnesses are present. Otherwise fall back to averaged arc scores.
  • I5 (frames): Calibrate cut-offs for irony or speech-act detectors on held-out data, rather than hard-coding them, so each decision has a documented, explainable threshold.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksSemitic languages · 3,066 worksMultilingual & cross-lingual NLP · 882 works

Cite

Cite as Nguyen Pham Hoang Le, An Nguyen Tran Khuong, Tram Nguyen Thi Ngoc and Thin Dang Van. 2025. NTA at SemEval-2025 Task 11: Enhanced Multilingual Textual Multi-label Emotion Detection via Integrated Augmentation Learning. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{le2025nta,
  title = {{NTA at SemEval-2025 Task 11: Enhanced Multilingual Textual Multi-label Emotion Detection via Integrated Augmentation Learning}},
  author = {Nguyen Pham Hoang Le, An Nguyen Tran Khuong, Tram Nguyen Thi Ngoc and Thin Dang Van},
  booktitle = {Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)},
  year = {2025},
  url = {https://aclanthology.org/2025.semeval-1.237/}
}

References

13 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexander Panchenko, Charles Henrique Porto Ferreira, Vitaly Protasov, Samuel Rutunda, Manish Shrivastava, Aura Cristina Udrea, Lilian Diana Awuor Wanzare, Sophie Wu, Florian Valentin Wunderlich, Hanif Muhammad Zhafran, Tianhui Zhang, Yi Zhou, and Saif M. Mohammad. 2025a. Brighter: Bridging the gap in human-annotated textual emotion recognition datasets for 28 languages. Preprint, arXiv:2502.11926. ↗
  2. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022a. Adapting pre-trained language models to african languages via multilingual adaptive fine-tuning. Preprint, arXiv:2204.06487. ↗
  3. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022b. Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. ↗
  4. Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philip Slusallek, Olga Kolesnikova, and Seid Muhie Yimam. 2025. Evaluating the capabilities of large language models for multi-label emotion understanding. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3523–3540, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  6. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. Preprint, arXiv:2006.03654. ↗
  7. George A. Miller. 1994. WordNet: A lexical database for English. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
  8. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word-emotion association lexicon. Preprint, arXiv:1308.6297. ↗
  9. Shamsuddeen Hassan Muhammad, Nedjma Ousidhoum, Idris Abdulmumin, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine de Kock, Nirmal Surange, Daniela Teodorescu, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino D. M. A. Ali, Ilseyar Alimova, Vladimir Araujo, Nikolay Babakov, Naomi Baes, Ana-Maria Bucur, Andiswa Bukula, Guanqun Cao, Rodrigo Tufino Cardenas, Rendi Chevi, Chiamaka Ijeoma Chukwuneke, Alexandra Ciobotaru, Daryna Dementieva, Murja Sani Gadanya, Robert Geislinger, Bela Gipp, Oumaima Hourrane, Oana Ignat, Falalu Ibrahim Lawan, Rooweither Mabuya, Rahmad Mahendra, Vukosi Marivate, Andrew Piper, Shamsuddeen Hassan Muhammad, Nedjma Ousidhoum, Idris Abdulmumin, Seid Muhie Yimam, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine De Kock, Tadesse Destaw Belay, Ibrahim Said Ahmad, Nirmal Surange, Daniela Teodorescu, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino Ali, Vladimir Araujo, Abinew Ali Ayele, Oana Ignat, Alexander Panchenko, Yi Zhou, and Saif M. Mohammad. 2025b. SemEval task 11: Bridging the gap in text-based emotion detection. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), Vienna, Austria. Association for Computational Linguistics. find
  10. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics. ↗
  11. Jason Wei and Kai Zou. 2019. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6382–6388, Hong Kong, China. Association for Computational Linguistics. ↗
  12. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  13. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2020. Xlnet: Generalized autoregressive pretraining for language understanding. Preprint, arXiv:1906.08237. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/le-2025-nta-semeval-2025-task-11/