Elamur Research⌕
☰
home / papers / 2026 / ES4MLL at SemEval-2026 Task 2: Set Attention Aggregation and Recurrent Temporal Modeling for Longitudinal Affect Prediction

ES4MLL at SemEval-2026 Task 2: Set Attention Aggregation and Recurrent Temporal Modeling for Longitudinal Affect Prediction

Paper · 2026 · Andrea Lolli, Chiara Lunazzi, Riccardo Coppola et al.

Summary

The paper presents the ES4MLL system for SemEval-2026 Task 2. The task asks systems to predict self-reported valence and arousal from users' essays and feeling words written over time. The authors address Subtask 1 (longitudinal affect assessment) and Subtask 2a (forecasting short-term affect change). Their model is hierarchical:

  • bert-base-uncased encodes each text;
  • Set Transformer blocks (ISAB, then PMA) compress tokens into a fixed document vector;
  • an LSTM models the sequence of documents per user.

For forecasting, past valence and arousal scores are fused with text features. The prediction head is also conditioned on the last known state, so the model effectively learns the change rather than the absolute value. Because the dataset is small, the encoder is adapted with DoRA or BitFit. Training uses a weighted MSE+CCC loss that puts most weight on arousal, which is the harder dimension.

In post-hoc test evaluation, the Subtask 2a system reached an average correlation of 0.587, a level that falls in the upper part of the leaderboard. Arousal remained the main weakness. The study shows that pooling choices should depend on the task: attention pooling helps assessment, while mean pooling suits forecasting.

Key points

  • Data: the training set has 2,765 samples. Users contribute from 2 to over 200 texts, with a median of 14 documents per user. Inputs are truncated to 128 tokens.
  • Configuration: ISAB uses 32 inducing points and PMA uses 8 seeds. The LSTM has 1 layer with hidden size 256. DoRA uses rank 16 and α = 32. The loss uses λ = 0.15 (favouring CCC) and ω_v = 0.2, which gives 80% of the weight to arousal.
  • Subtask 1 (Sim. 11, DoRA + BitFit):
    • Validation r_comp was 0.6802.
    • The leaderboard comparison table reports a test r_comp of 0.533, against 0.611 for the winner (UKP_Psycontrol). The conclusion gives 0.5420 for the same test setting, which is an inconsistency in the paper.
    • Arousal scored only 0.411.
  • Subtask 2a (Sim. 9, Bi-LSTM + DoRA): test average was 0.587. The valence score of 0.663 would rank second, behind YNU (0.692). These scores come from evaluation on gold labels released after the deadline, not from an official submission.
  • Ablation findings and limitations:
    • DoRA outperformed both a frozen encoder and BitFit.
    • A unidirectional LSTM worked best together with ISAB in Subtask 1, while a Bi-LSTM without attention worked best for forecasting.
    • The authors note limitations: the model assumes evenly spaced observations, and larger encoders (DeBERTa-v3-base, ELECTRA-base) did not fit in free Colab GPU memory.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Politecnico di TorinoTurin · Italy11 works · 18 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSocial Psychology · 306 works

Cite

Cite as Andrea Lolli, Chiara Lunazzi, Riccardo Coppola and Flavio Giobergia. 2026. ES4MLL at SemEval-2026 Task 2: Set Attention Aggregation and Recurrent Temporal Modeling for Longitudinal Affect Prediction. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.102

@inproceedings{lolli2026es4mll,
  title = {{ES4MLL at SemEval-2026 Task 2: Set Attention Aggregation and Recurrent Temporal Modeling for Longitudinal Affect Prediction}},
  author = {Andrea Lolli, Chiara Lunazzi, Riccardo Coppola and Flavio Giobergia},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.102},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.102}
}

References

11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Elad Ben-Zaken, Shauli Ravfogel, and Yoav Goldberg. 2022. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. Preprint, arXiv:2106.10199.
  2. Lukas Christ, Shahin Amiriparian, Manuel Milling, Ilhan Aslan, and Björn W. Schuller. 2022. Automatic emotion modelling in written stories. Preprint, arXiv:2212.11382. — [arXiv:2212.11382](https://arxiv.org/abs/2212.11382) find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  4. Flavio Giobergia, Alkis Koudounas, and Elena Baralis. 2024. Large language models-aided literature reviews: A study on few-shot relevance classification. In 2024 IEEE 18th International Conference on Application of Information and Communication Technologies (AICT), pages 1–5. IEEE. find
  5. Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. 2019. Set transformer: A framework for attention-based permutation-invariant neural networks. In Proceedings of the 36th International Conference on Machine Learning, pages 3744–3753. find
  6. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. Preprint, arXiv:2402.09353. find
  7. Gonçalo Azevedo Mendes and Bruno Martins. 2023. Quantifying valence and arousal in text with multilingual pre-trained transformers. Preprint, arXiv:2302.14021. — [arXiv:2302.14021](https://arxiv.org/abs/2302.14021) find
  8. Nikki A. Puccetti, William J. Villano, Jonathan P. Fadok, and Aaron S. Heller. 2022. Temporal dynamics of affect in the brain: Evidence from human imaging and animal models. Neuroscience & Biobehavioral Reviews, 133. find
  9. James Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology, 39:1161–1178. find
  10. Nikita Soni, H. Andrew Schwartz, Ryan L. Boyd, Phi Long Bui, Syeda Mahwish, August Håkan Nilsson, Adithya V Ganesan, Lyle Ungar, Niranjan Balasubramanian, and Saif M. Mohammad. 2026. SemEval-2026 task 2: Predicting variation in emotional valence and arousal over time from ecological essays. In Proce…
  11. Yu Zhu, Hao Li, Yikang Liao, Beidou Wang, Ziyu Guan, Haifeng Liu, and Deng Cai. 2017. What to do next: Modeling user behaviors by time-lstm. In IJCAI, volume 17, pages 3602–3608. Melbourne, VIC. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lolli-2026-es4mll-semeval-2026-task-2/