Elamur Research⌕
☰
home / papers / 2026 / AI4PC-Howard University at SemEval-2026 Task 2: Fine-Tuning DistilBERT, DeBERTa and ModernBERT for Valence–Arousal Prediction and Change Estimation

AI4PC-Howard University at SemEval-2026 Task 2: Fine-Tuning DistilBERT, DeBERTa and ModernBERT for Valence–Arousal Prediction and Change Estimation

Paper · 2026 · Araj Shah, Utsav Shah and Saurav Aryal

Summary

This paper combines the SemEval-2026 Task 2 submissions of two Howard University teams, AI4PC-Howard University (Team 1) and BisonAI4PC (Team 2). The task is predicting valence–arousal (VA) from a longitudinal collection of English essays, each produced by an American employee in the service sector. The authors split the problem into three views: per-essay VA state (Subtask 1), short-horizon VA change (Subtask 2A), and long-horizon disposition change (Subtask 2B).

Both teams trained only on the official data. They used user-disjoint splits so that writer-specific cues could not leak between training and evaluation, and they kept the pipelines small and deterministic. The systems were:

  • Subtask 1: fine-tuned DistilBERT or ModernBERT regressors.
  • Subtask 2A: a GRU over ModernBERT embedding windows, blended with a linear previous-delta baseline.
  • Subtask 2B: pooled DeBERTa-v3 history embeddings with trajectory summary features and an MLP.

On essay-level VA, the systems clearly beat the organizers' baseline. For both change-prediction subtasks, they scored below the simple linear(prev) baseline. The paper is useful mainly as a reproducible, leakage-aware reference, and its results suggest that naive temporal baselines are hard to beat for longitudinal affect.

Key points

  • Subtask 1 (best: Team 2, DistilBERT): r_comp of 0.665 for valence and 0.468 for arousal (average 0.567), versus 0.428 for the linear(BERT) baseline. Team 1's ModernBERT reached 0.631/0.462. Team 1 rescaled arousal from [0, 2] to [−2, 2] during training.
  • Subtask 2A (best: Team 1, ModernBERT + GRU + blend): r of 0.597 for valence and 0.413 for arousal (average 0.505). This trails the linear(prev) baseline at 0.643. Team 1's GRU used the last k = 32 embeddings plus time-gap features. Team 2's per-anchor DistilBERT + history MLP averaged only 0.232.
  • Subtask 2B (best: Team 1, pooled DeBERTa-v3-base + MLP): r of 0.046 for valence and 0.348 for arousal (average 0.197), versus 0.509 for the baseline. Team 2's DistilBERT biLSTM had negative correlations (average −0.112).
  • Training and preprocessing: all models were trained with AdamW, with selection on fixed, seeded unseen-user splits. Numeric features were standardized using training-split statistics only. Maximum input length was 512 tokens for the DistilBERT systems and 256 for the embedding pipelines.
  • Authors' caveat: encoder choice and aggregation strategy changed together across systems, so the comparisons cannot separate the effect of each backbone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Howard UniversityWashington · United States2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSocial Psychology · 306 works

Cite

Cite as Araj Shah, Utsav Shah and Saurav Aryal. 2026. AI4PC-Howard University at SemEval-2026 Task 2: Fine-Tuning DistilBERT, DeBERTa and ModernBERT for Valence–Arousal Prediction and Change Estimation. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.89

@inproceedings{shah2026ai4pc,
  title = {{AI4PC-Howard University at SemEval-2026 Task 2: Fine-Tuning DistilBERT, DeBERTa and ModernBERT for Valence–Arousal Prediction and Change Estimation}},
  author = {Araj Shah, Utsav Shah and Saurav Aryal},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.89},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.89}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saurav K Aryal, Howard Prioleau, and Surakshya Aryal. 2023a. Sentiment analysis across multiple african languages: A current benchmark. arXiv preprint arXiv:2310.14120. — [arXiv:2310.14120](https://arxiv.org/abs/2310.14120) find
  2. Saurav K Aryal, Howard Prioleau, Surakshya Aryal, and Gloria Washington. 2023b. Baselining performance for multilingual codeswitching sentiment classification. Journal of Computing Sciences in Colleges, 39(3):337–346. find
  3. Saurav K Aryal, Howard Prioleau, and Legand Burge. 2022. Acoustic-linguistic features for modeling neurological task score in alzheimer’s. In Pacific Symposium on Biocomputing 2023: Kohala Coast, Hawaii, USA, 3–7 January 2023, pages 335–346. find
  4. Saurav K Aryal, Ujjawal Shah, Howard Prioleau, and Legand Burge. 2023c. Ensembling and modeling approaches for enhancing alzheimer’s disease scoring and severity assessment. In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), pages 1364–1370. IEEE. find
  5. Saurav Keshari Aryal and Gaurav Adhikari. 2023. Evaluating impact of emoticons and pre-processing on sentiment classification of translated african tweets. find
  6. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. Preprint, arXiv:1607.06450.
  7. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua…
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  9. Desta Haileselassie Hagos, Saurav Keshari Aryal, Patrick Ymele-Leki, and Legand L Burge. 2025. Ai-driven multimodal colorimetric analytics for biomedical and behavioral health diagnostics. Computational and structural biotechnology journal, 27:2219–2232. find
  10. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021a. DeBERTaV3: Improving DeBERTa using electra-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  11. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021b. DeBERTa: Decoding-enhanced BERT with disentangled attention.
  12. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (GELUs). Preprint, arXiv:1606.08415. find
  13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  14. Peter J. Huber. 1964. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1):73–101. find
  15. Jazette Johnson, Lucretia Williams, Jaye Nias, Saurav K Aryal, and Gloria Washington. 2025. Centering black voices: Lessons learned and reflections from a large-scale aave data collection at a historically black university. In Proceedings of the Extended Abstracts of the CHI Conference on Human Fact… find
  16. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. Preprint, arXiv:1412.6980.
  17. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). find
  18. Mikel K Ngueajio, Saurav Aryal, Marcellin Atemkeng, Gloria Washington, and Danda Rawat. 2025. Decoding fake news and hate speech: A survey of explainable ai techniques. ACM Computing Surveys, 57(7):1–37. find
  19. James A. Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology, 39(6):1161–1178. find
  20. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. Preprint, arXiv:1910.01108. find
  21. Mike Schuster and Kuldip K. Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. find
  22. Nikita Soni, H. Andrew Schwartz, Ryan L. Boyd, Phi Long Bui, Syeda Mahwish, August Håkan Nilsson, Adithya V Ganesan, Lyle Ungar, Niranjan Balasubramanian, and Saif M. Mohammad. 2026. SemEval-2026 task 2: Predicting variation in emotional valence and arousal over time from ecological essays. In Proce…
  23. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS).
  24. Benjamin Warner, Alex Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, and 1 others. 2024. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. Preprint, arXiv:2412.13663.
  25. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clément Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural L…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shah-2026-ai4pc-howard-university-semeval-2026/