Elamur Research⌕
☰
home / papers / 2026 / Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays

Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays

Paper · 2026 · Igor Cataneo Silveira and Denis Deratani Mauá

Summary

Automated scoring of ENEM argumentative essays has mostly been treated as plain score prediction. That approach ignores the official grading rubrics and gives students little insight into why they received a grade. The authors draw on the official ENEM Graders Guideline, which breaks each of the five traits into subcriteria, and propose two neuro-symbolic scoring methods.

In the first, GPT4o writes a rubric-grounded justification of an essay's score. A BERTimbau-base model then predicts the trait score from that explanation rather than from the essay itself. The second method encodes the rubrics as logical rules over subcriteria, whose truth values are predicted by neural subnetworks trained with the Semantic Loss.

To support and evaluate this, two professional graders annotated a new set of 63 mock-ENEM essays for subcriteria. Both methods score about as well as purely statistical state-of-the-art models. They also offer finer-grained feedback, which is relevant for formative assessment and transparent high-stakes grading.

Key points

  • New dataset. 63 essays from AES-ENEM Source B were labeled by two expert graders against 71 subcriteria grouped into 20 dimensions. It is released on Hugging Face (igorcs/enem-subcriteria-dataset).
  • Rubric structure. Trait C1 has 9 subcriteria in two dimensions: syntax (5) and mistakes (4). Exactly one subcriterion holds per dimension, and the score follows logically from them.
  • Inter-grader agreement (QWK) is lower on the subcriteria dataset than on AES-ENEM. For example, C4 drops from .45 to .18 and C2 from .54 to .30.
  • Explanation-based scoring. Training on GPT4o explanations performs comparably to training on essay text. The authors take this as evidence that the explanations are reliable.
  • Transfer and limitations. For C1, subnetworks trained only on the small dataset transferred zero-shot to AES-ENEM with .58 QWK, and further distant supervision improved results. The authors note limitations: limited data coverage (especially for low-scoring essays), use of GPT4o as the only LLM, and a model that was not fully optimized.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 works

Cite

Cite as Igor Cataneo Silveira and Denis Deratani Mauá. 2026. Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{silveira2026neuro,
  title = {{Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays}},
  author = {Igor Cataneo Silveira and Denis Deratani Mauá},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.78/}
}

References

26 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evelin Amorim and Adriano Veloso. 2017. A multi-aspect analysis of automatic essay scoring for Brazilian Portuguese. In Proceedings of the Student Research Workshop at the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 94–102. Association for Computat… find
  2. Rafael Anchiêta, Anthony Luz, Shara Lopes, and Raimundo Moura. 2025. A zero-shot prompting approach for automated feedback generation on enem Renato Lui Geh, Jonas Gonçalves, Igor C. Silveira, Denis D. Mauá, and Fabio G. Cozman. 2024. dPASP: A Probabilistic Logic Programming Environment For Neurosym… find
  3. Renato Lui Geh, Jonas Gonçalves, Igor Cataneo Silveira, Denis Deratani Mauá, and Fabio Gagliardi Cozman. 2023. dpasp: A comprehensive differentiable probabilistic answer set programming environment for neurosymbolic learning and reasoning. Preprint, arXiv:2308.02944. — [arXiv:2308.02944](https://arxiv.org/abs/2308.02944) find
  4. Jiani Huang, Ziyang Li, Binghong Chen, Karan Samel, Mayur Naik, Le Song, and Xujie Si. 2021. Scallop: From probabilistic deductive databases to scalable differentiable reasoning. In Advances in Neural Information Processing Systems, volume 34, page 25134–25145. Curran Associates, Inc. find
  5. Shiyu Ji, Farnoosh Hashemi, Joice Chen, Juanwen Pan, Weicheng Ma, Hefan Zhang, Sophia Pan, Ming Cheng, Shubham Mohole, Saeed Hassanpour, Soroush Vosoughi, and Michael Macy. 2025. A generalizable rhetorical strategy annotation model using LLM-based debate simulation and labelling. In Findings of the… find
  6. Sidney Evaldo Leal, Magali Sanches Duran, Carolina Evaristo Scarton, Nathan Siegle Hartmann, and Sandra Maria Aluísio. 2024. NILC-Metrix: Assessing the Complexity of Written and Spoken Language in Brazilian Portuguese. Language Resources and Evaluation, 58(1):73–110. find
  7. Ziyang Li, Jiani Huang, and Mayur Naik. 2023. Scallop: A language for neurosymbolic programming. Proc. ACM Program. Lang., 7(PLDI). find
  8. Jiawei Liu, Yang Xu, and Yaguang Zhu. 2019. Automated essay scoring based on two-stage learning. Preprint, arXiv:1901.07744. — [arXiv:1901.07744](https://arxiv.org/abs/1901.07744) find
  9. Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2018. Deepproblog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
  10. Robin Manhaeve, Sebastijan Dumančić, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2021. Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 298:103504. find
  11. Jeziel Marinho, Rafael Anchiêta, and Raimundo Moura. 2021. Essay-br: a brazilian corpus of essays. In Anais do III Dataset Showcase Workshop, pages 53– 64. Sociedade Brasileira de Computação. find
  12. Jeziel Marinho, Rafael Anchiêta, and Raimundo Moura. 2022a. Essay-br: a brazilian corpus to automatic essay scoring task. Journal of Information and Data Management, 13(1):65–76. find
  13. Jeziel Marinho, Fábio Cordeiro, Rafael Anchiêta, and Raimundo Moura. 2022b. Automated essay scoring: An approach based on enem competencies. In Anais do XIX Encontro Nacional de Inteligência Artificial e Computacional, pages 49–60. find
  14. Rafael Ferreira Mello, Hilário Oliveira, Moésio Wenceslau, Hyan Batista, Thiago Cordeiro, Ig Ibert Bittencourt, and Seiji Isotanif. 2024. PROPOR’24 Competition on Automatic Essay Scoring of Portuguese Narrative Essays. In Proceedings of the 16th International Conference on Computational Processing o… find
  15. Hilário Oliveira, Rafael Ferreira Mello, Péricles Miranda, Hyan Batista, Moésio Wenceslau Silva da Filho, Thiago Cordeiro, Ig Ibert Bittencourt, and Seiji Isotani. 2025. A benchmark dataset of narrative student essays with multi-competency grades for automatic essay scoring in brazilian portuguese.… find
  16. Ellis B. Page. 1966. The imminence of... grading essays by computer. The Phi Delta Kappan, pages 238–243. find
  17. Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2024a. Automatic text readability assessment in European Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 97–107, Santiago de Compostela, Galicia/Spain. Association for Computat… find
  18. Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2024b. Exploring the Automated Scoring of Narrative Essays in Brazilian Portuguese using Transformer Models. In Proceedings of the 16th International Conference on Computational Processing of Portuguese-Vol. 2, pages 14–17. find
  19. Victor Hugo Nascimento Rocha, Igor Cataneo Silveira, Paulo Pirozelli, Denis Deratani Mauá, and Fabio Gagliardi Cozman. 2023. Assessing good, bad and ugly arguments generated by chatgpt: a new dataset, its methodology and associated tasks. In Progress in Artificial Intelligence: 22nd EPIA Conference… find
  20. Mark D Shermis and Jill Burstein. 2013. Handbook of automated essay evaluation. NY: Routledge. find
  21. Igor Cataneo Silveira, André Barbosa, Daniel Silva Lopes da Costa, and Denis Deratani Mauá. 2025a. Investigating Universal Adversarial Attacks Against Transformers-Based Automatic Essay Scoring Systems. In Intelligent Systems, pages 169–183, Cham. Springer Nature Switzerland. find
  22. Igor Cataneo Silveira, André Barbosa, and Denis Deratani Mauá. 2024. A new benchmark for automatic essay scoring in Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 228–237. find
  23. Igor Cataneo Silveira, Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2025b. Aprendizado por transferência para correçao automática de redaçao. Linguamática, 17(2):99–116. find
  24. Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020. BERTimbau: pretrained BERT models for Brazilian Portuguese. In 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23 (to appear). find
  25. Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. 2018. A semantic loss function for deep learning with symbolic knowledge. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5502–5511. PMLR. find
  26. Zhun Yang, Adam Ishay, and Joohyung Lee. 2020. Neurasp: Embracing neural networks into answer set programming. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 1755–1762. International Joint Conferences on Artificial Intelligence Organizat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/silveira-2026-neuro-symbolic-approaches-rubric-based/