Elamur Research⌕
☰
home / papers / 2025 / Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension

Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension

Paper · 2025 · Ping Li

Summary

The paper targets three weaknesses of AI tools that score or support English reading comprehension: opaque decisions, unequal accuracy across learner groups, and unreliable behaviour in real classrooms. The author proposes one Transformer-based pipeline that combines three parts. The first renders multi-head attention as sharpened heatmaps. The second scores each token's influence by multiplying attention by its gradient. The third is an adversarial debiasing objective that adds a KL consistency term and a demographic-parity regularizer to the comprehension loss.

The system is tested on a labelled set of 18,077 reading exercises with answers, against tuned BERT and RoBERTa baselines. The paper says it beats both on accuracy and macro-F1, sometimes coming close to human-level results. It also reports better agreement between model highlights and teacher-marked rationales. A six-week deployment with high school teachers found higher trust and willingness to use the AI feedback. However, the paper gives few exact figures, so its claims are hard to verify. Its value is mainly as a template for joining explanation, fairness and user evaluation in one educational system.

Key points

  • The dataset has 18,077 English reading comprehension items with reference answers, split into train, dev and held-out test folds. Inputs are capped at 512 subword tokens.
  • The baselines are fine-tuned and hyperparameter-tuned BERT and RoBERTa. Results come from three independent runs with a fixed seed.
  • The explanation layer has two steps:
    • attention heatmaps, row-normalized and sharpened with an exponent γ;
    • token attribution as gradient × attention, summed over layers and heads, then z-scored within each input.
  • The debiasing method uses FGSM-style sign-gradient perturbations of input representations. The total loss combines three terms: comprehension loss, β·KL(clean ‖ perturbed) and α·fairness loss.
  • Interpretability is scored with an "attention alignment index". This is the average Jaccard overlap between model-highlighted tokens and teacher-annotated rationale tokens. A six-week study with teachers used Likert-scale trust and usability measures.

Interesting ideas

  • I1. Measure explanation quality as Jaccard overlap between the evidence the model highlights and evidence that human experts mark on their own, so explanations become a number you can track.
  • I2. Add a KL consistency term that penalizes the model when small input perturbations change its output distribution, so it learns to stay stable rather than fit noise.
  • I3. Z-normalize attribution scores within each input so that token importance can be compared across sentences of different lengths and difficulty.
  • I4. Report accuracy per subgroup before and after an intervention, not just one overall score, to reveal hidden gaps.
  • I5. Test the system with its real end users over several weeks and measure trust and understanding, not only accuracy.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (answer): For fable and school-test QA, record which tree nodes or world facts each answer relied on, and score their overlap with a small hand-marked rationale set.
  • I2 (learn): When training gparse on silver data, add a consistency loss between clean and lightly perturbed sentences (typos, casing, synonyms) to make arc and label predictions more robust.
  • I3 (tree): Normalizing per-token confidence or attribution in the transformer parser would help flag unusually uncertain attachments for repair rules or teacher review, whatever the sentence length.
  • I4 (learn): Break evaluation down by genre, treebank source or dialect conversion path, so that a gain on one slice cannot hide a regression on another.
  • I5 (answer): Show teachers or learners Mova's step-by-step reasoning on school tests and collect Likert ratings, to check whether its explanations are actually understood.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Shandong Management University.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksQuestion answering · 441 works

Cite

Cite as Ping Li. 2025. Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension. arXiv:2604.23615 · Neural Computing and Applications.35(2025)18603-18622.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2604.23615 · arXiv 2604.23615

@misc{li2025applications,
  title = {{Applications of the Transformer Architecture in AI-Assisted English Reading Comprehension}},
  author = {Ping Li},
  year = {2025},
  doi = {10.48550/arxiv.2604.23615},
  eprint = {2604.23615},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2604.23615}
}

References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kumar, S., & Solanki, A. (2023). An abstractive text summarization technique using transformer model with self-attention mechanism. Neural Computing and Applications, 35(25), 18603-18622. ↗
  2. Kaliyar, R. K. (2020, January). A multi-layer bidirectional transformer encoder for pre-trained word embedding: A survey of bert. In 2020 10th International conference on cloud computing, data science & engineering (confluence) (pp. 336-340). IEEE. find
  3. Li, H. (2025). Automatic evaluation and enhancement of reading strategies in English reading comprehension based on the BERT model. Journal of Computational Methods in Sciences and Engineering, 25(1), 794-807. ↗
  4. Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022). A systematic review of explainable artificial intelligence in terms of different application domains and tasks. Applied Sciences, 12(3), 1353. ↗
  5. Wang, Z. J., Turko, R., & Chau, D. H. (2021, August). Dodrio: Exploring transformer models with interactive visualization. In Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: system demonstrations (pp. 132-141). find
  6. Meng, C., Trinh, L., Xu, N., Enouen, J., & Liu, Y. (2022). Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset. Scientific reports, 12(1), 7166. ↗
  7. Lee, M., Kim, Y., Mok, W. L., & Lee, S. (2025, July). CURRICULUM DEBIASING: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 9524-9540). find
  8. Wang, Z. J., Turko, R., Shaikh, O., Park, H., Das, N., Hohman, F., ... & Chau, D. H. P. (2020). CNN explainer: learning convolutional neural networks with interactive visualization. IEEE Transactions on Visualization and Computer Graphics, 27(2), 1396-1406. ↗
  9. Kinga Wojcik. (2025). Automated Essay Scoring via NLP: System Architectures, Feature Engineering, and Evaluation Metrics. Data Engineering and Applications, 3(1), 1–19. https://doi.org/10.64972/dea.2025.v3i1.32 ↗
  10. Galassi, A., Lippi, M., & Torroni, P. (2020). Attention in natural language processing. IEEE transactions on neural networks and learning systems, 32(10), 4291-4308. ↗
  11. Wang, T., Wang, X., Qin, Y., Packer, B., Li, K., Chen, J., ... & Chi, E. (2020, November). Cat-gen: Improving robustness in nlp models via controlled adversarial text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 5141-5146). find
  12. Johnson, M. S., & McCaffrey, D. F. (2023). Evaluating fairness of automated scoring in educational measurement. In Advancing natural language processing in educational assessment (pp. 142-164). Routledge. ↗
  13. Mathew, D. E., Ebem, D. U., Ikegwu, A. C., Ukeoma, P. E., & Dibiaezue, N. F. (2025). Recent emerging techniques in explainable artificial intelligence to enhance the interpretable and understanding of AI models for human. Neural Processing Letters, 57(1), 16. ↗
  14. Diaz-Garcia, J. A., & Lopez, J. A. D. (2025). A survey on cutting-edge relation extraction techniques based on language models. Artificial Intelligence Review, 58(9), 287. ↗
  15. Sukaria, M. I. (2025). Evaluating the Credibility of AI-Based Authentic Assessment in Early Numeracy Education: A Systematic Review. Information Technology Education Journal, 519-526. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2026-applications-transformer-architecture-ai-assisted/