Elamur Research⌕
☰
home / papers / 2026 / Large Language Models for Citation Function Classification

Large Language Models for Citation Function Classification

Paper · 2026 · Daniel Vodička, Pavel Kral, Christophe Cerisara et al.

Summary

Citation counts treat every reference alike, but citations serve very different purposes. One may build on a method, another may dispute a finding. This paper examines how well current large language models can automatically label citation function. The authors benchmark four open LLMs (Mistral 7B, Orca 2-7B, LLaMA 3.1-8B, Falcon 7B) and the encoder SciBERT under zero-shot prompting, few-shot prompting and fine-tuning. Fine-tuning is done only on the established ACL-ARC benchmark. The paper also releases AC³ (ArXiv Citation Classification Corpus), a new resource built from arXiv papers. Its seven-label scheme separates neutral references from evaluative ones, such as praise, criticism or conflicting results, which older schemes lump into a single comparison class. AC³ comes in four versions with differently sized citation contexts, so the effect of context length can be measured. Because AC³ is small, it is used only for prompting experiments. The fine-tuned Falcon 7B sets a new state of the art on ACL-ARC. The work fills a gap noted in a recent survey, which observed that no systematic comparison of LLMs for this task existed.

Key points

  • Fine-tuned Falcon 7B reaches 73.3% macro F1 on ACL-ARC, outperforming earlier approaches such as SciCite-era models, CitePrompt and dynamic context extraction.
  • ACL-ARC has 1,941 citations labelled with six classes: Background, Uses, Compare_contrast, Motivation, Extension and Future. Background alone covers about 52% of the data.
  • AC³ uses seven labels: Criticizing, Compliment, Comparison, Use, Substantiating, Contradiction and Basis. The scheme draws on Liu et al. (2022) and Abu-Jbara et al. (2013).
  • AC³ is provided in four context-extraction variants. The authors report that the best context window depends on both the model and the citation type.
  • The corpus is freely available for research on GitHub (DanielRafaello/ArXiv-Citation-Classification-Corpus). Class imbalance, with rare labels like Compliment and Contradiction, remains a key difficulty.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksInformation Systems · 523 worksDecision Sciences · 85 worksStatistics, Probability and Uncertainty · 32 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1,343 works

Cite

Cite as Daniel Vodička, Pavel Kral, Christophe Cerisara and Jakub Šmíd. 2026. Large Language Models for Citation Function Classification. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4sb25z5kxz3q

@inproceedings{vodicka2026large,
  title = {{Large Language Models for Citation Function Classification}},
  author = {Daniel Vodička, Pavel Kral, Christophe Cerisara and Jakub Šmíd},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4sb25z5kxz3q},
  url = {https://doi.org/10.63317/4sb25z5kxz3q}
}

References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amjad Abu-Jbara, Jefferson Ezra, and Dragomir Radev. 2013. Purpose and polarity of citation: Towards NLP-based bibliometrics. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 596–606, Atlanta, Geo… find
  2. Naif Radi Aljohani, Ayman Fayoumi, and Saeed-Ul Hassan. 2023. A novel focal-loss and class-weight-aware convolutional neural network for the classification of in-text citations. Journal of Information Science, 49(1):79–92. find
  3. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The falcon series of open language mo…
  4. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: A pretrained language model for scientific text.
  5. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural scaffolds for citation intent classification in scientific publications.
  6. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. find
  7. Ying Ding, Guo Zhang, Tamy Chambers, Min Song, Xiaolong Wang, and Chengxiang Zhai. 2014. Content-based citation analysis: The next generation of citation analysis. Journal of the Association for Information Science and Technology, 65(9):1820–1833. find
  8. Cailing Dong and Ulrich Schäfer. 2011. Ensemble-style self-training on citation classification. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 623– 631, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find
  9. Abhimanyu Dubey et al. 2024. The llama 3 herd of models. find
  10. Eugene Garfield et al. 1965. Can citation indexing be automated. In Statistical association methods for mechanized documentation, symposium proceedings, volume 269, pages 189–192. Citeseer. find
  11. Saeed-Ul Hassan, Naif R Aljohani, Nimra Idrees, Raheem Sarwar, Raheel Nawaz, Eugenio Martínez-Cámara, Sebastián Ventura, and Francisco Herrera. 2020. Predicting literature’s early impact with sentiment analysis in twitter. Knowledge-Based Systems, 192:105383. find
  12. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  13. Xiaorui Jiang and Jingqiang Chen. 2023. Contextualised segment-wise citation function classification. Scientometrics, 128(9):5117–5158. find
  14. David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky. 2016. Citation classification for behavioral analysis of a scientific field. find
  15. David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky. 2018. Measuring the evolution of a scientific field through citation frames. Transactions of the Association for Computational Linguistics, 6:391–406.
  16. Suchetha N. Kunnath, David Pride, and Petr Knoth. 2023. Prompting strategies for citation classification. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM ’23, page 1127–1137, New York, NY, USA. Association for Computing Machinery. find
  17. Avishek Lahiri, Debarshi Kumar Sanyal, and Imon Mukherjee. 2024. Citeprompt: Using prompts to identify citation intent in scientific papers. In Proceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries, JCDL ’23, page 51–55. IEEE Press. find
  18. Jiaying Liu, Feng Xia, Xu Feng, Jing Ren, and Huan Liu. 2022. Deep graph learning for anomalous citation detection. find
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. find
  20. Dongqing Lyu, Xuanmin Ruan, Juan Xie, and Ying Cheng. 2021. The classification of citing motivations: a meta-synthesis. Scientometrics, 126. find
  21. Himanshu Maheshwari, Bhavyajeet Singh, and Vasudeva Varma. 2021. SciBERT sentence representation for citation context classification. In Proceedings of the Second Workshop on Scholarly Document Processing, pages 130–133, Online. Association for Computational Linguistics. find
  22. Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, and Ahmed Awadallah. 2023. Orca 2: Teaching small language models how to reason.
  23. Suchetha Nambanoor Kunnath, David Pride, and Petr Knoth. 2022a. Dynamic context extraction for citation classification. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Pr… find
  24. Suchetha Nambanoor Kunnath, Valentin Stauber, Ronin Wu, David Pride, Viktor Botev, and Petr Knoth. 2022b. ACT2: A multi-disciplinary semi-structured dataset for importance and purpose classification of citations. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 33… find
  25. David Pride and Petr Knoth. 2020. An authoritative approach to citation classification. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, JCDL ’20, page 337–340, New York, NY, USA. Association for Computing Machinery. find
  26. Muhammad Roman, Abdul Shahid, Shafiullah Khan, Anis Koubaa, and Lisu Yu. 2021. Citation intent classification using word embedding. IEEE Access, 9:9982–9995. find
  27. Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. Automatic classification of citation function. In Proceedings of the 2006 conference on empirical methods in natural language processing, pages 103–110.
  28. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe…
  29. Yang Zhang, Yufei Wang, Quan Z Sheng, Adnan Mahmood, Wei Emma Zhang, and Rongying Zhao. 2021. Tdm-cfc: Towards document-level multi-label citation function classification. In Web Information Systems Engineering–WISE 2021: 22nd International Conference on Web Information Systems Engineering, WISE 202… find
  30. Yang Zhang, Yufei Wang, Quan Z. Sheng, Lina Yao, Haihua Chen, Kai Wang, Adnan Mahmood, Wei Emma Zhang, Munazza Zaib, Subhash Sagar, and Rongying Zhao. 2025. Deep learning meets bibliometrics: A survey of citation function classification. Journal of Informetrics, 19(1):101608. find
  31. He Zhao, Zhunchen Luo, Chong Feng, Anqing Zheng, and Xiaopeng Liu. 2019. A context-based framework for modeling the role and function of on-line resource citations in scientific literature. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vodicka-2026-large-language-models-citation-function/