Elamur Research⌕
☰
home / papers / 2026 / Beyond the Black Box: Ethical and Theoretical Grounding in Affective Computing

Beyond the Black Box: Ethical and Theoretical Grounding in Affective Computing

Paper · 2026 · Wajdi Zaghouani

Summary

This position paper contends that affective computing has advanced faster than its conceptual foundations. Emotion recognition is spreading into healthcare, education, hiring, policing and marketing, yet these systems learn surface correlations between signals (faces, voice, physiology) and categorical labels without drawing on any psychological account of emotion. The author identifies four linked problems: missing theory, demographic unfairness, inadequate protection of emotional privacy, and the risks of machines simulating empathy, especially toward vulnerable users. The paper's claimed contribution is combining these four into one framework and arguing that the theory deficit is the root of the other three. It draws on appraisal, constructionist, sociodynamic and evolutionary perspectives, along with dimensional models and cultural display rules. On that basis it proposes a "theory-first" design: continuous multidimensional outputs, context-aware multimodal modelling, and annotation that preserves disagreement. It also calls for intersectional fairness audits and for treating affective data as a specially protected category. This matters for anyone building emotion-annotated corpora or deploying affect classifiers across cultures and languages.

Key points

  • Cited evidence shows facial emotion recognition can exceed 80% accuracy, yet racial performance gaps reach 27.2 percentage points or more (Fan et al., 2023). Further disparities are reported by age, gender and language.
  • The theoretical grounding draws on the Emotion Review special issue (vol. 6, no. 4, 2014): Barrett's Conceptual Act Theory, Moors on appraisal, Mesquita and Boiger's sociodynamic model, and Tracy's evolutionary view.
  • Cultural findings are highlighted: East Asian perceivers may group happy, sad and angry expressions into fewer dimensions, and collectivist display norms produce more muted expressions.
  • Annotation is framed through four "Qualities of Indeterminacy" (subjectivity, uncertainty, ambiguity, vagueness). The paper recommends collecting multiple labels per instance and treating annotator disagreement as signal rather than noise.
  • The author links this to his own Arabic resources, including hate speech annotation toll, a YouTube trauma corpus, and depression and stress datasets from social media.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Northwestern University in QatarDoha · Qatar39 works · 148 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksSocial Sciences · 1,954 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 worksSafety Research · 626 worksHuman-Computer Interaction · 160 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksBias, fairness & safety · 587 works

Cite

Cite as Wajdi Zaghouani. 2026. Beyond the Black Box: Ethical and Theoretical Grounding in Affective Computing. Proceedings of Computational Affective Science (CAS) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/29tzm79c59a7

@inproceedings{zaghouani2026beyond,
  title = {{Beyond the Black Box: Ethical and Theoretical Grounding in Affective Computing}},
  author = {Wajdi Zaghouani},
  booktitle = {Proceedings of Computational Affective Science (CAS) @ LREC 2026},
  year = {2026},
  doi = {10.63317/29tzm79c59a7},
  url = {https://doi.org/10.63317/29tzm79c59a7}
}

References

34 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. This paper raises but does not resolve several ethical questions that warrant sustained attention. The case scenarios presented in Section 4.4 are constructed from documented patterns rather than specific incidents, and we have been careful not to attribute hypothetical harms to specific companies o… find
  2. Maitha M. Al Emadi and Wajdi Zaghouani. 2024. Emotional toll and coping strategies: Navigating the effects of annotating hate speech data. In Proceedings of the Workshop on Legal and Ethical Issues in Human Language Technologies @ LREC-COLING 2024, pages 66–72. find
  3. Dinko Bacic and Curt A. Gilstrap. 2024. Predicting video virality and viewer engagement: A biometric data and machine learning approach. Behaviour & Information Technology, 43(12):2854– 2880. find
  4. D. Barker, M. K. R. Tippireddy, A. Farhan, and B. Ahmed. 2025. Ethical considerations in emotion recognition research. Psychology International. find
  5. Lisa Feldman Barrett. 2014. The conceptual act theory: A précis. Emotion Review, 6(4):292–297. find
  6. Mohamed Benouis, Elisabeth Andre, and Yekta Said Can. 2024. Balancing between privacy and utility for affect recognition using multitask learning in differential privacy-added federated learning settings. JMIR Mental Health, 11:e60003. find
  7. Chaona Chen, Daniel S. Messinger, Cheng Chen, Hongmei Yan, Yaocong Duan, Robin A. A. Ince, Oliver G. B. Garrod, Philippe G. Schyns, and Rachael E. Jack. 2024. Cultural facial expressions dynamically convey emotion category and intensity information. Current Biology, 34(1):213– 223.e5. find
  8. Woan-Shiuan Chien, Shreya G. Upadhyay, and Chi-Chun Lee. 2024. Balancing speaker-rater fairness for gender-neutral speech emotion recognition. In ICASSP 2024, pages 11861–11865. find
  9. Julia F. Christensen, Klaus Frieler, Megan Vartanian, Shiva Khorsandi, Fereshteh Farahi, Seyed Hojjat Nazafi Yazdi, and Vincent Walsh. 2025. An enculturation-induced joy bias for emotion recognition in full-body-movement. Scientific Reports, 15:37163. find
  10. Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM), pages 128–137. find
  11. Pawel Drozdowski, Christian Rathgeb, Antitza Dantcheva, Naser Damer, and Christoph Busch. 2020. Demographic bias in biometrics: A survey on an emerging challenge. IEEE Transactions on Technology and Society, 1(2):89–103. find
  12. Nicola Fabiano. 2025. Affective computing and emotional data: Challenges and implications in privacy regulations, the AI act, and ethics in large language models. arXiv preprint arXiv:2509.20153. — [arXiv:2509.20153](https://arxiv.org/abs/2509.20153) find
  13. Andrew Fan, Xuhai Xiao, and Peter Washington. 2023. Addressing racial bias in facial emotion recognition. arXiv preprint arXiv:2308.04674. — [arXiv:2308.04674](https://arxiv.org/abs/2308.04674) find
  14. Hatice Gunes. 2023. Fairness for affective and well-being computing. In Proceedings of the 1st International Workshop on Multimodal and Responsible Affective Computing (MRAC ’23), pages 1–2. Association for Computing Machinery. find
  15. Ramanpreet Kaur, Kanwal Preet Singh Attwal, and Harmandeep Singh. 2025. A systematic review on facial emotion recognition system datasets. International Journal on Science and Technology (IJSAT), 16(2). Paper ID: 6406. find
  16. Eugenia Kim, De’Aira G. Bryant, Deepak Srikanth, and Ayanna M. Howard. 2021. Age bias in emotion detection: An analysis of facial emotion recognition performance on young, middle-aged, and older adults. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’21), pages 638–6… find
  17. Mei Li, Jiankun Peng, Yinkun Zhu, and Wei Cheng. 2025. Emotional recognition in affective tutoring system: A systematic review. In 2025 International Conference on Digital Education and Learning (ICDEL), pages 344–348. find
  18. Yi-Cheng Lin, Huang-Cheng Chou, and Hung-yi Lee. 2025. Mitigating subgroup disparities in multi-label speech emotion recognition: A pseudo-labeling and unsupervised learning approach. find
  19. Batja Mesquita and Michael Boiger. 2014. Emotions in context: A sociodynamic model of emotions. Emotion Review, 6(4):298–302. find
  20. Amro Mohamed and Wajdi Zaghouani. 2024. Expression of depression among Arab Twitter users using Arabic corpus analysis. In Procedia Computer Science, volume 244, pages 76–85. find
  21. Agnes Moors. 2014. Flavors of appraisal theories of emotion. Emotion Review, 6(4):303–307. find
  22. Mustaqeem and Soonil Kwon. 2020. A CNN-assisted enhanced audio signal processing for speech emotion recognition. Sensors, 20(1):183. find
  23. Tomoya Nakai, Laura Rachman, Pablo Arias Sarah, Kazuo Okanoya, and Jean-Julien Aucouturier. 2023. Algorithmic voice transformations reveal the phonological basis of language-familiarity effects in cross-cultural emotion judgments. PLOS ONE, 18(5):e0285028. find
  24. Toshiki Saito, Kosuke Motoki, and Yuji Takano. 2023. Cultural differences in recognizing emotions of masked faces. Emotion, 23(6):1648– 1657. find
  25. Celine Shurafa and Wajdi Zaghouani. 2024. Sentiment analysis and emotion annotation of a large-scale Arabic YouTube trauma corpus. In 2024 11th International Conference on Behavioural and Social Computing (BESC), pages 1–7. IEEE. find
  26. Xin Tang, Yuying Gong, Yuchen Xiao, Jiang Xiong, and Lei Bao. 2024. Facial expression recognition for probing students emotional engagement in science learning. Journal of Science Education and Technology. find
  27. Jessica L. Tracy. 2014. An evolutionary approach to understanding distinct emotions. Emotion Review, 6(4):308–312. find
  28. Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou, and Hung-yi Lee. 2025. CO-VADA: A confidence-oriented voice augmentation debiasing approach for fair speech emotion recognition. find
  29. Ioannis Tsangko, Andreas Triantafyllopoulos, Adria Mallol-Ragolta, and Björn W. Schuller. 2025. Reading smiles: Proxy bias in foundation models for facial emotion recognition. IEEE Access. Early Access. find
  30. Yeamin Yuk, Eiko Matsuda, and Kaori Ando. 2025. Self-construal and nonverbal emotional expressivity: A cross-cultural comparison between Korea and Japan. Psychology and Behavioral Sciences, 14(6). find
  31. Wajdi Zaghouani. 2018. A large-scale social media corpus for the detection of youth depression (project note). Procedia Computer Science, 142:347–351. find
  32. Wajdi Zaghouani, Engy S. Shlkamy, and Mabrouka Bessghaier. 2026. From posts to pressure: An Arabic dataset about stress and mental-health monitoring. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (Ab-jadNLP 2026).
  33. Jianhua Zhang, Zhong Yin, Peng Chen, and Stefano Nichele. 2020. Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review. Information Fusion, 59:103–126. find
  34. Lena Pruessner and Asli Altan-Atalay. 2025. Cultural context shapes the selection and adaptiveness of interpersonal emotion regulation strategies. Emotion, 25(2):526–540. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2026-beyond-black-box-ethical-theoretical/