Elamur Research⌕
☰
home / papers / 2026 / Automatic scoring of ENEM essays: an empirical study of linguistic and contextual representations

Automatic scoring of ENEM essays: an empirical study of linguistic and contextual representations

Original (Portuguese): Avaliação Automática de Redações do Enem: Um Estudo Empírico sobre Representações Linguísticas e Contextuais

Paper · 2026 · Gabriel Gonçalves de Matos and Valéria D. Feltrim

Summary

This study tackles automated essay scoring (AES) for Brazilian Portuguese essays from the Enem exam. Each essay is graded on five competencies with ordered, discrete scores. The authors ran a controlled comparison of text representations under a single fixed protocol, using the public Enem-AES benchmark. Each competency was framed as ordinal prediction with the CORAL framework, keeping the neural architecture constant so that only the input representation changed. The candidates were:

  • a TF-IDF baseline,
  • general-purpose linguistic metrics from NILC-Metrix,
  • task-oriented handcrafted features,
  • contextual embeddings,
  • concatenations of these.

Hybrid models reached the best average agreement with human scores, though results varied by competency and representation type. The authors also examined essays where human raters disagreed and found that annotation variability clearly limits achievable performance. The work matters because it gives systematic evidence that combining explicit linguistic indicators with contextual embeddings is a promising direction for Enem scoring. It also highlights that rater noise caps what any model can reach.

Key points

  • Data: Enem-AES has 1,165 essays scored per competency on {0, 40, 80, 120, 160, 200}, with skewed score distributions and three ratings per essay. The authors used the official 65/17/18 train/validation/test split.
  • Rater agreement: trained annotators agreed with each other at QWK 0.4–0.8. Their agreement with the online platform's scores was weaker, at QWK 0.2–0.6. A non-divergent test subset (rater gap under 80 points) was used for extra analysis.
  • Feature sets:
    • TF-IDF reduced to 1,000 dimensions,
    • 200 NILC-Metrix metrics covering lexical, syntactic, semantic and discourse levels,
    • 27 task-oriented features from Neves (2025).
  • Hybrid design: hybrids combine representations by simple vector concatenation. No formal statistical significance tests were run, so the comparisons are descriptive.
  • Code: code and hyperparameters are released at github.com/GGMmattos/AES-ENEM-HybridModel.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Gabriel Gonçalves de Matos and Valéria D. Feltrim. 2026. Avaliação Automática de Redações do Enem: Um Estudo Empírico sobre Representações Linguísticas e Contextuais. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{matos2026avaliacao,
  title = {{Avaliação Automática de Redações do Enem: Um Estudo Empírico sobre Representações Linguísticas e Contextuais}},
  author = {Gabriel Gonçalves {de Matos} and Valéria D. Feltrim},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.48/}
}

References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evelin Amorim e Adriano Veloso. 2017. A multi-aspect analysis of automatic essay scoring for Brazilian Portuguese. Em Proceedings of the Student Research Workshop at the 15th Conference of the European Chapter of the Association for Computational Linguistics, páginas 94–102, Valencia, Spain. Associa… find
  2. Yigal Attali e Jill Burstein. 2006. Automated essay scoring with e-rater [r] v. 2. Journal of Technology, Learning, and Assessment, 4(3). find
  3. André Barbosa, Igor Cataneo Silveira, e Denis Deratani Mauá. 2025. An empirical analysis of large language models for automated cross-prompt essay trait scoring in brazilian portuguese. Journal of the Brazilian Computer Society, 31(1):857–870. find
  4. Wenzhi Cao, Vahid Mirjalili, e Sebastian Raschka. 2020. Rank consistent ordinal regression for neural networks with application to age estimation. Pattern Recognition Letters, 140:325–331. find
  5. Aluizio Haendchen Filho, Fernando Concatto, Hércules Antonio do Prado, e Edilson Ferneda. 2021. Comparing feature engineering and deep learning methods for automated essay scoring of brazilian national high school examination. Em Proceedings of the 23rd International Conference on Enterprise Informa… find
  6. Erick Fonseca, Ivo Medeiros, Dayse Kamikawachi, e Alessandro Bokan. 2018. Automatically grading brazilian student essays. Em International Conference on Computational Processing of the Portuguese Language, páginas 170–179. Springer. find
  7. Sidney Evaldo Leal, Magali Sanches Duran, Carolina Evaristo Scarton, Nathan Siegle Hartmann, e Sandra Maria Aluísio. 2023. Nilc-metrix: assessing the complexity of written and spoken language in brazilian portuguese. Lang Resources & Evaluation, 58:73—-110. find
  8. Feng Li, Xuefeng Xi, Zhiming Cui, Dongyang Li, e Wanting Zeng. 2023. Automatic essay scoring method based on multi-scale features. Applied Sciences, 13(11):6775. find
  9. Jeziel C. Marinho, Rafael T. Anchiêta, e Raimundo S. Moura. 2022a. Essay-br: a brazilian corpus to automatic essay scoring task. Journal of Information and Data Management, 13(1). find
  10. Jeziel C Marinho, Fábio Cordeiro, Rafael T Anchiêta, e Raimundo S Moura. 2022b. Automated essay scoring: An approach based on enem competencies. Em Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), páginas 49–60. SBC. find
  11. Murilo Luis Calvo Neves. 2025. Extração e avaliação de indicadores lingísticos no contexto de correção automatizada de redações aplicadas ao exame nacional do ensino médio (enem). Trabalho de Conclusão de Curso (Graduação em Ciência da Computação), Universidade Estadual de Maringá, Maringá, PR, Bras… find
  12. Igor Cataneo Silveira, André Barbosa, Daniel Silva Lopes da Costa, e Denis Deratani Mauá. 2025. Investigating universal adversarial attacks against transformers-based automatic essay scoring systems. Em Intelligent Systems, páginas 169–183, Cham. Springer Nature Switzerland. find
  13. Igor Cataneo Silveira, André Barbosa, e Denis Deratani Mauá. 2024. A new benchmark for automatic essay scoring in portuguese. Em Proceedings of the 16th International Conference on Computational Processing of Portuguese-Vol. 1, páginas 228–237. find
  14. Fábio Souza, Rodrigo Nogueira, e Roberto Lotufo. 2020. Bertimbau: Pretrained bert models for brazilian portuguese. Em Intelligent Systems, páginas 403–417, Cham. Springer International Publishing. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/matos-2026-avaliacao-automatica-de-redacoes-do/