Elamur Research⌕
☰
home / papers / 2025 / Explaining Relationships Among Research Papers

Explaining Relationships Among Research Papers

Paper · 2025 · Xiangci Li and Jessica Ouyang

Summary

Researchers struggle to keep up with the volume of new publications. The authors argue that useful literature overviews need more than isolated citation sentences. They also need expository and transition sentences that explain how the cited works connect. No dataset of short, reader-tailored reviews exists, so the authors use related work sections, which can be extracted automatically from articles, as a substitute. Their method first prompts LLMs (gpt-3.5-turbo-0301 and gpt-4-0314) to produce human-readable features. These include citation intents, relationships between the target and cited papers and among co-cited papers, and faceted summaries of each cited work. The features are then combined into a single prompt that writes a paragraph containing several citations plus connecting sentences. Generation is guided by a brief human-written plan of main ideas. Domain experts, mostly authors of the target papers, rated the outputs. Their preferences correlated strongly with integrative writing: abstract, high-level citations woven into a coherent narrative. The work points toward tools that summarize personalized paper feeds, though human planning still proves essential.

Key points

  • Features such as intents, pairwise and group relationships, and faceted summaries are extracted by prompting LLMs, then composed into one generation prompt that produces multiple citations in one pass.
  • A planning-based setup supplies human-written main-idea sentences. The fully automated variant performed very poorly, and outputs still fall short of human-written reviews in organization and style.
  • In a comparison reproducing the paper's own Section 2, GPT-4-powered Bing Chat produced generic, poorly organized text with misattributed claims.
  • The prompts were also tried on other models: gpt-3.5-turbo-0613, gpt-4-0613 and Claude-v2 gave similar results, text-bison-32k gave weaker style, and LLaMA-2 70B Chat produced irrelevant text.
  • The evaluation set is concentrated in computer science, especially NLP, plus one geology paper. Limitations include unavailable cited PDFs, input-length chunking, and inconsistent citation markers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AmazonSeattle · United States112 works · 40,565 citations🎓The University of Texas at DallasRichardson · United States61 works · 5,614 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHistory and Philosophy of Science · 53 works

Cite

Cite as Xiangci Li and Jessica Ouyang. 2025. Explaining Relationships Among Research Papers. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{li2025explaining,
  title = {{Explaining Relationships Among Research Papers}},
  author = {Xiangci Li and Jessica Ouyang},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.73/}
}

References

68 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed AbuRa’ed, Horacio Saggion, and Luis Chiruzzo. 2020. A multi-level annotated corpus of scientific papers for scientific document summarization and cross-document relation discovery. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6672–6679, Marseille, France. E… find
  2. Ahmed AbuRa’ed, Horacio Saggion, Alexander Shvets, and Àlex Bravo. 2020. Automatic related work section generation: experiments in scientific document abstracting. Scientometrics, 125:3159–3185. find
  3. Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
  4. Yassine Benajiba, Jin Sun, Yong Zhang, Longquan Jiang, Zhiliang Weng, and Or Biran. 2019. Siamese networks for semantic pattern similarity. In 2019 IEEE 13th international conference on semantic computing (ICSC), pages 191–194. IEEE. find
  5. Ziqiang Cao, Furu Wei, Li Dong, Sujian Li, and Ming Zhou. 2015. Ranking with recursive neural networks and its application to multi-document summarization. In Proceedings of the AAAI conference on artificial intelligence, volume 29. find
  6. Jingqiang Chen and Hai Zhuge. 2019. Automatic generation of related work through summarizing citations. Concurrency and Computation: Practice and Experience, 31(3):e4261. find
  7. Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Rui Yan, Xin Gao, and Xiangliang Zhang. 2022. Target-aware abstractive related work generation with contrastive learning. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 373– 38… find
  8. Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Xiangliang Zhang, Dongyan Zhao, and Rui Yan. 2021. Capturing relations between scientific papers: An abstractive model for related work section generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and… find
  9. Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural scaffolds for citation intent classification in scientific publications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  10. Zekun Deng, Zixin Zeng, Weiye Gu, Jiawen Ji, and Bolin Hua. 2021. Automatic related work section generation by sentence extraction and reordering. In AII@ iConference, pages 101–110. find
  11. Kaize Ding, Jianling Wang, James Caverlee, and Huan Liu. 2022. Meta propagation networks for graph few-shot semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 6524–6531. find
  12. Cailing Dong and Ulrich Schäfer. 2011. Ensemble-style self-training on citation classification. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 623–631, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find
  13. Pedro Faustini, Zhiyu Chen, Besnik Fetahu, Oleg Rokhlenko, and Shervin Malmasi. 2023. Answering unanswered questions through semantic reformulations in spoken QA. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), pages 729– 743, To… find
  14. Eugene Garfield et al. 1965. Can citation indexing be automated. In Statistical association methods for mechanized documentation, symposium proceedings, volume 269, pages 189–192. Washington. find
  15. Yubin Ge, Ly Dinh, Xiaofeng Liu, Jinsong Su, Ziyao Lu, Ante Wang, and Jana Diesner. 2021. BACO: A background knowledge- and content-based framework for citing sentence generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Jo… find
  16. Sarik Ghazarian, Zixi Liu, Akash S M, Ralph Weischedel, Aram Galstyan, and Nanyun Peng. 2021. Plot-guided adversarial example construction for evaluating open-domain story generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics… find
  17. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pa…
  18. Nianlong Gu and Richard H. R. Hahnloser. 2023. Controllable citation sentence generation with language models. find
  19. Tairan He, Yuge Zhang, Kan Ren, Minghuan Liu, Che Wang, Weinan Zhang, Yuqing Yang, and Dongsheng Li. 2022. Reinforcement learning with automated auxiliary loss search. Advances in neural information processing systems, 35:1820–1834. find
  20. Cong Duy Vu Hoang and Min-Yen Kan. 2010. Towards automated related work summarization. In Coling 2010: Posters, pages 427–435, Beijing, China. find
  21. Coling 2010 Organizing Committee. find
  22. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. Unnatural instructions: Tuning language models with (almost) no human labor. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14409–14428, Toronto, Canada. Associat…
  23. Yibo Hu, MohammadSaleh Hosseini, Erick Skorupa Parolin, Javier Osorio, Latifur Khan, Patrick Brandt, and Vito D’Orazio. 2022. ConfliBERT: A pre-trained language model for political conflict and violence. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Compu… find
  24. Yue Hu and Xiaojun Wan. 2014. Automatic generation of related work sections in scientific papers: An optimization approach. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1624–1633, Doha, Qatar. Association for Computational Linguistics. find
  25. Kung-Hsiang Huang, ChengXiang Zhai, and Heng Ji. 2022a. CONCRETE: Improving cross-lingual fact-checking with cross-lingual retrieval. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1024–1035, Gyeongju, Republic of Korea. International Committee on Computation… find
  26. Zili Huang, Shinji Watanabe, Shu-wen Yang, Paola García, and Sanjeev Khudanpur. 2022b. Investigating self-supervised learning for speech enhancement and separation. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6837–6841. IEEE. find
  27. Kokil Jaidka, Muthu Kumar Chandrasekaran, Sajal Rustagi, and Min-Yen Kan. 2018. Insights from cl-scisumm 2016: the faceted scientific document summarization shared task. International Journal on Digital Libraries, 19(2):163–171. find
  28. Kokil Jaidka, Michihiro Yasunaga, Muthu Kumar Chandrasekaran, Dragomir Radev, and Min-Yen Kan. 2019. The cl-scisumm shared task 2018: Results and key insights. arXiv preprint arXiv:1909.00764. — [arXiv:1909.00764](https://arxiv.org/abs/1909.00764) find
  29. David Jurgens, Srijan Kumar, Raine Hoover, Dan Mc-Farland, and Dan Jurafsky. 2018. Measuring the evolution of a scientific field through citation frames. Transactions of the Association for Computational Linguistics, 6:391–406.
  30. Christopher SG Khoo, Jin-Cheon Na, and Kokil Jaidka. 2011. Analysis of the macro-level discourse structure of literature reviews. Online Information Review. find
  31. Román A Lara-Cueva, Diego S Benítez, Enrique V Carrera, Mario Ruiz, and José Luis Rojo-Álvarez. 2016. Automatic recognition of long period events from volcano tectonic earthquakes at cotopaxi volcano. IEEE Transactions on Geoscience and Remote Sensing, 54(9):5247–5257. find
  32. Anne Lauscher, Brandon Ko, Bailey Kuehl, Sophie Johnson, Arman Cohan, David Jurgens, and Kyle Lo. 2022. MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting. In Proceedings of the 2022 Conference of the North American Chapter of the Association for… find
  33. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguist…
  34. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR.
  35. Shengjie Li and Vincent Ng. 2022. End-to-end neural discourse deixis resolution in dialogue. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11322–11334, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  36. Xiangci Li, Yi-Hui Lee, and Jessica Ouyang. 2024. Cited text spans for citation text generation. In Proceedings of the Fourth Workshop on Scholarly Document Processing. find
  37. Xiangci Li, Biswadip Mandal, and Jessica Ouyang. 2022. CORWA: A citation-oriented related work annotation dataset. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5426–5440, Seattle, United State… find
  38. Xiangci Li and Jessica Ouyang. 2022. Automatic related work generation: A meta study. arXiv preprint arXiv:2201.01880. — [arXiv:2201.01880](https://arxiv.org/abs/2201.01880) find
  39. Xiangci Li and Jessica Ouyang. 2024. Related work and citation text generation: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13846–13864, Miami, Florida, USA. Association for Computational Linguistics. find
  40. Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. S2ORC: The semantic scholar open research corpus. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969–4983, Online. Association for Computational Linguistics.
  41. Yangxiao Lu, Ninad Khargonkar, Zesheng Xu, Charles Averill, Kamalesh Palanisamy, Kaiyu Hang, Yunhui Guo, Nicholas Ruozzi, and Yu Xiang. Self-supervised unseen object instance segmentation via long-term robot interaction. find
  42. Josh Magnus Ludan, Yixuan Meng, Tai Nguyen, Saurabh Shah, Qing Lyu, Marianna Apidianaki, and Chris Callison-Burch. 2023. Explanation-based fine-tuning makes models more robust to spurious cues. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find
  43. Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, and Noah A. Smith. 2021. Explaining relationships between scientific documents. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lang… find
  44. Rui Meng, Khushboo Thaker, Lei Zhang, Yue Dong, Xingdi Yuan, Tong Wang, and Daqing He. 2021. Bringing structure into summaries: a faceted summarization dataset for long scientific documents. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Inter… find
  45. OpenAI. 2023. Gpt-4 technical report.
  46. Jessica Ouyang and Kathy McKeown. 2019. Neural network alignment for sentential paraphrases. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4724– 4735, Florence, Italy. Association for Computational Linguistics. find
  47. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  48. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083, Vancouver, Canada. Association for Compu… find
  49. Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. 2023. What the DAAM: Interpreting stable diffusion using cross attention. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… find
  50. Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. Automatic classification of citation function. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 103–110, Sydney, Australia. Association for Computational Linguistics.
  51. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  52. Nilesh Tripuraneni, Chi Jin, and Michael Jordan. 2021. Provable meta-learning of linear representations. In International Conference on Machine Learning, pages 10434–10443. PMLR. find
  53. Suppawong Tuarob, Sung Woo Kang, Poom Wet-tayakorn, Chanatip Pornprasit, Tanakitti Sachati, Saeed-Ul Hassan, and Peter Haddawy. 2019. Automatic classification of algorithm citation functions in scientific literature. IEEE Transactions on Knowledge and Data Engineering, 32(10):1881–1896. find
  54. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  55. Ningna Wang, Bin Wang, Wenping Wang, and Xiaohu Guo. 2022. Computing medial axis transform with feature preservation via restricted power diagram. ACM Transactions on Graphics (TOG), 41(6):1–18. find
  56. Pancheng Wang, Shasha Li, Haifang Zhou, Jintao Tang, and Ting Wang. 2019. Toc-rwg: Explore the combination of topic model and citation information for automatic related work generation. IEEE Access, 8:13043–13055. find
  57. Yongzhen Wang, Xiaozhong Liu, and Zheng Gao. 2018. Neural related work summarization with a joint context-driven attention mechanism. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1776–1786, Brussels, Belgium. Association for Computational Linguisti… find
  58. Josh Wiedemeier, Elliot Tarbet, Max Zheng, Sangsoo Ko, Jessica Ouyang, Sang Kil Cha, and Kangkook Jee. 2024. Pylingual: Toward perfect decompilation of evolving high-level languages. In 2025 IEEE Symposium on Security and Privacy (SP), pages 52–52. IEEE Computer Society. find
  59. He Zhao, Zhunchen Luo, Chong Feng, Anqing Zheng, and Xiaopeng Liu. 2019. A context-based framework for modeling the role and function of on-line resource citations in scientific literature. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internat… find
  60. Abudukelimu Wuerkaixi, Kunda Yan, You Zhang, Zhiyao Duan, and Changshui Zhang. 2022. Dyvise: Dynamic vision-guided speaker embedding for audio-visual speaker diarization. In 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), pages 1–6. IEEE. find
  61. Xinyu Xing, Xiaosheng Fan, and Xiaojun Wan. 2020. Automatic generation of citation texts in scholarly papers: A pilot study. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6181–6190, Online. Association for Computational Linguistics. find
  62. Mu Yang, Andros Tjandra, Chunxi Liu, David Zhang, Duc Le, and Ozlem Kalinli. 2023a. Learning asr pathways: A sparse multilingual asr model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  63. Xianjun Yang, Yujie Lu, and Linda Petzold. 2023b. Few-shot document-level event argument extraction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8029–8046, Toronto, Canada. Association for Computational Linguistics. find
  64. Xiaocong Yang, James Y. Huang, Wenxuan Zhou, and Muhao Chen. 2023c. Parameter-efficient tuning with special token adaptation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 865–872, Dubrovnik, Croatia. Association for Computation… find
  65. Michihiro Yasunaga, Jungo Kasai, Rui Zhang, Alexander R Fabbri, Irene Li, Dan Friedman, and Dragomir R Radev. 2019. Scisummnet: A large annotated corpus and content-impact models for scientific paper summarization with citation networks. In Proceedings of the AAAI conference on artificial intelligen… find
  66. Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. Prompting large language model for machine translation: A case study. In International Conference on Machine Learning, pages 41092–41110. PMLR.
  67. Wenzheng Zhang and Karl Stratos. 2021. Understanding hard negatives in noise contrastive estimation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1090–1101, Online. Association for Computation… find
  68. Prompt Title: {{title}} Abstract: {{abstract}} Introduction: {{introduction}} Conclusion: {{conclusion}} What are the objective, method, findings, contributions and keywords of the paper above? Answer in the format of Objective: XXX. Method: XXX. Findings: XXX. Contribution: XXX. Keywords: A; B; C.… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-explaining-relationships-among-research-papers/