Elamur Research⌕
☰
home / papers / 2026 / CITD@UIT at SemEval-2026 Task 2: Temporal Mixture-of-Experts for Longitudinal Valence and Arousal Prediction from Ecological Essays

CITD@UIT at SemEval-2026 Task 2: Temporal Mixture-of-Experts for Longitudinal Valence and Arousal Prediction from Ecological Essays

Paper · 2026 · Son The Phuong, My Thuy-Tra Ngo, Tri Minh Dao et al.

Summary

The team from the University of Information Technology (VNU-HCM) built a system for SemEval-2026 Task 2, which models how a person's emotions change over time in their ecological essays. Subtask 1 asks for continuous Valence and Arousal scores for each text in a user's sequence. Subtask 2 asks for forecasts of short-term state change (2A) and long-term dispositional change (2B).

The backbone is cardiffnlp/twitter-roberta-base-sentiment-latest. Posts are ordered by user and time, and consecutive posts are joined with a sliding window so the encoder sees recent history. For 2B, the text is split into past and future groups, and the mean past V&A values are fused into the representation late. The regression heads are either funnel-shaped MLPs or Mixture-of-Experts heads with sparse or soft gating. Features come from attention pooling or mean pooling. Training uses a joint CCC-based loss for both dimensions, which matches the task's evaluation metric.

The system placed 9th in Subtask 1 and 5th in Subtask 2A. It did not generalize on 2B. The results suggest that expert routing helps short-horizon emotion forecasting, while long-range dispositional modeling remains difficult.

Key points

  • The backbone, cardiffnlp/twitter-roberta-base-sentiment-latest, was pretrained on 124M tweets from 2018–2021. Each MLP head narrows from 768 to 256 to 128 to 1 dimensions, with separate branches for Valence and Arousal.
  • The MoE head has 4 experts. The sparse variant uses Top-k gating, and the soft variant uses dense softmax gating over all experts. The loss is the sum of (1 − CCC) for Valence and Arousal.
  • In Subtask 2A, Sparse MoE with Attention Pooling and a window of k = 8 ranked 5th of 15 teams. Its test r_avg was 0.631, slightly above its validation score of 0.623.
  • In Subtask 2B, Sparse MoE with Mean Pooling fell from 0.327 r_avg on validation to −0.114 on the blind test. The authors attribute this to three factors:
    • imbalance between the 768-dimensional text embedding and the 2-dimensional past-emotion features;
    • expert collapse;
    • overfitting given the small number of users.
  • Arousal was harder to predict than Valence. Code is available at github.com/PTSown0222/SemEval-2026-Task-2.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vietnam National University Ho Chi Minh CityHo Chi Minh City · Vietnam21 works · 14 citations🎓University of Information TechnologyHo Chi Minh City · Vietnam13 works · 7 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksSocial Psychology · 330 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 1,035 works

Cite

Cite as Son The Phuong, My Thuy-Tra Ngo, Tri Minh Dao and Duc-Vu Nguyen. 2026. CITD@UIT at SemEval-2026 Task 2: Temporal Mixture-of-Experts for Longitudinal Valence and Arousal Prediction from Ecological Essays. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.25

@inproceedings{phuong2026citd,
  title = {{CITD@UIT at SemEval-2026 Task 2: Temporal Mixture-of-Experts for Longitudinal Valence and Arousal Prediction from Ecological Essays}},
  author = {Son The Phuong, My Thuy-Tra Ngo, Tri Minh Dao and Duc-Vu Nguyen},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.25},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.25}
}

References

10 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bagus Tris Atmaja and Masato Akagi. 2020. Combined learning of speech intensity and emotional valence-arousal. IEEE Access, 8:192801–192813. find
  2. Jose Camacho-collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa Anke, Fangyu Liu, Eugenio Martínez Cámara, and 1 others. 2022. TweetNLP: Cutting-edge natural language processing for social media. In Proceedings of the 2022 Confer… find
  3. Damai Dai, Chengqi Deng, Chenggang Zhao, R.X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. 2024. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language mode… find
  4. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23:1–40. find
  5. Christo Hristov, Ion Martinis, Yalcin Tur, Kevina Wang, and Miko Rimer. 2025. Text and valence-arousal: A two-dimensional foundational approach for mental health prediction. Technical report, Stanford University. Technical Report, CS 277 / BIODS 271. find
  6. Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive mixtures of local experts. Neural computation, 3(1):79–87. find
  7. Arun Kashyap. 2024. Mental health chatbot using roberta and gemini. https://github.com/kashyaparun25/Mental-Health-Chatbot-using-RoBERTa-and-Gemini. GitHub repository. ↗
  8. Gonçalo Azevedo Mendes and Bruno Martins. 2023. Quantifying valence and arousal in text with multilingual pre-trained transformers. arXiv preprint arXiv:2302.14021. — [arXiv:2302.14021](https://arxiv.org/abs/2302.14021) find
  9. Liyu Meng, Yuchen Liu, Xiaolong Liu, Zhaopei Huang, Wenqiang Jiang, Tenggan Zhang, Chuanhe Liu, and Qin Jin. 2022. Valence and arousal estimation based on multimodal temporal-aware features for videos in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (… find
  10. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 14…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/phuong-2026-citd-uit-semeval-2026-task/