Elamur Research⌕
☰
home / papers / 2026 / Tracing Relational Knowledge Recall in Large Language Models

Tracing Relational Knowledge Recall in Large Language Models

Paper · 2026 · Nicholas Popovič and Michael Färber

Summary

The paper asks where inside an LLM a relation type becomes linearly readable at the moment the model is about to generate the second entity of a statement, and what makes some relations easier to decode than others. Working with few-shot relation classification on English FewRel data, the authors train linear probes on several candidate features taken from attention and MLP sublayers. The strongest and most consistent probe inputs are per-head attention contributions to the residual stream, which outperform probes trained on the complete attention-layer and MLP representations. To explain probe decisions, they propose HeadScore and TokenScore, which split each prediction across attention heads and source tokens. Probe precision drops for relations with a broad output range and high entity connectedness (both estimated from Wikidata), and when the probe's signal is spread thinly over many heads. That last factor depends only on the probe, so it may work as a data-free diagnostic. Few errors look like lexical shortcuts. The work offers tools for tracing relational recall back to specific heads and tokens.

Key points

  • Tested on four instruction-tuned models: LLaMA-3.2 1B, LLaMA-3.2 3B, LLaMA-3.1 8B and Qwen3 4B.
  • Relations come from FewRel. Wikidata properties approximate relation specificity (output range) and entity connectedness.
  • HeadScore and TokenScore attribute linear-probe predictions to individual attention heads and to individual source tokens.
  • Lexical-cue analysis shows weak alignment between attributions and lexical cues:
    • Spearman ρ between TokenScore and lexical profiles is 0.095–0.115.
    • Positive attribution mass on lexically supportive tokens is 0.475–0.491.
    • Only 5.3–7.7% of errors are strongly lexically aligned.
  • Limitations: the study covers classification only, with no NOTA or negative cases, English only, and models up to 8B parameters. Code is publicly released.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Technische Universität DresdenDresden · Germany13 works · 7,336 citations🎓Center for Scalable Data Analytics and Artificial IntelligenceDresden · Germany10 works · 77 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksInterpretability · 130 works

Cite

Cite as Nicholas Popovič and Michael Färber. 2026. Tracing Relational Knowledge Recall in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2160

@inproceedings{popovic2026tracing,
  title = {{Tracing Relational Knowledge Recall in Large Language Models}},
  author = {Nicholas Popovič and Michael Färber},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2160},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2160}
}

References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Workshop Track, Toulon, France. find
  2. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2895– 2905, Florence, Italy. Association for Com…
  3. Bilal Chughtai, Alan Cooney, and Neel Nanda. 2024. Summing up the facts: Additive mechanisms behind factual recall in llms. Preprint, arXiv:2402.07321. — [arXiv:2402.07321](https://arxiv.org/abs/2402.07321) find
  4. Xavier Suau Cuadros, Luca Zappella, and Nicholas Apostoloff. 2022. Self-conditioning pre-trained language models. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 4455–4473. PMLR. find
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, and Angela Fan et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  6. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. Preprint, a… find
  7. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, Singapore. Association for Computational… find
  8. Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018. FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4803–480… find
  9. Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2668– 2677. PMLR. find
  10. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA.
  11. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. PMET: Precise Model Editing in a Transformer. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):18564– 18572. find
  12. Yihong Liu, Runsheng Chen, Lea Hirlimann, Ahmad Dawar Hakimi, Mingyang Wang, Amir Hossein Kargaran, Sascha Rothe, François Yvon, and Hinrich Schuetze. 2025. On relation-specific neurons in large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processin… find
  13. Ang Lv, Yuhan Chen, Kaiyi Zhang, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, and Rui Yan. 2024. Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models. arXiv preprint. ArXiv:2403.19521 [cs]. — [arXiv:2403.19521](https://arxiv.org/abs/2403.19521) find
  14. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, volume 35, pages 17359–17372. Curran Associates, Inc. find
  15. Victor Morand, Nadi Tomeh, Josiane Mothe, and Benjamin Piwowarski. 2025. ToMMeR – Efficient Entity Mention Detection from Large Language Models. arXiv preprint. ArXiv:2510.19410 [cs].
  16. W. James Murdoch, Peter J. Liu, and Bin Yu. 2018. Beyond word importance: Contextual decomposition to extract interactions from LSTMs. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018). find
  17. Roee Hendel, Mor Geva, and Amir Globerson. 2023. In-context learning creates task vectors. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9318–9333, Singapore. Association for Computational Linguistics.
  18. Nicholas Popovic and Michael Färber. 2024. Embedded named entity recognition using probing classifiers. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17830–17850, Miami, Florida, USA. Association for Computational Linguistics. find
  19. Evan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, and David Bau. 2024. Linearity of relation decoding in transformer language models. In The twelfth international conference on learning representations. find
  20. Masaki Sakata, Benjamin Heinzerling, Sho Yokoi, Takumi Ito, and Kentaro Inui. 2025. On entity identification in language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16717–16741, Vienna, Austria. Association for Computational Linguistics. find
  21. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory sayres. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Mukund Sundararajan, Ankur Taly, and Qiqi Y… find
  22. Eric Todd, Millicent Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. 2024. Function vectors in large language models. In The Twelfth International Conference on Learning Representations. find
  23. Yiqun Wang, Chaoqun Wan, Sile Hu, Yonggang Zhang, Xiang Tian, Yaowu Chen, Xu Shen, and Jieping Ye. 2025. Tracing and dissecting how LLMs recall factual knowledge for real world questions. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper… find
  24. Zijian Wang, Britney Whyte, and Chang Xu. 2024. Locating and extracting relational concepts in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4818– 4832, Bangkok, Thailand. Association for Computational Linguistics. find
  25. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/popovic-2026-tracing-relational-knowledge-recall-large/