Elamur Research⌕
☰
home / papers / 2024 / Taxonomy Tree Generation from Citation Graph

Taxonomy Tree Generation from Citation Graph

Paper · 2024 · Yuntong Hu, Zhuofeng Li, Zheng Zhang et al.

Summary

Building a topic hierarchy for a scientific field usually takes expert effort. It is slow, can be biased, and may miss important papers that receive few citations. Existing taxonomy learning methods from NLP work on plain text and ignore how documents link to each other. The authors propose HiGTL (also called CGT in parts of the paper), which builds a taxonomy tree from a citation graph and is guided by a topic the user specifies.

HiGTL first clusters papers level by level, using both paper text and citation links, and merges lower-level topics into "supernodes." A pre-trained LLM then names each cluster, building the labels upward so they stay consistent across levels. The clustering and naming modules are pre-trained and then fine-tuned together.

The evaluation dataset has 518 citation graphs, each taken from a human-written computer science survey. HiGTL's hierarchical clustering clearly beats K-means and LLM-based clustering. HiReview, an extension that uses the taxonomy to draft literature reviews, outperforms AutoSurvey and RAG-based LLMs. The work points toward automated survey writing and mapping of research fields.

Key points

  • Dataset: 518 citation graphs, each linked to a high-quality human-written literature review in computer science.
  • Clustering quality (average over two levels): HiGTL's hierarchical clustering scores 0.6761, compared with 0.3962 for K-means and 0.4407 for GPT-4o-based clustering.
  • In literature review generation, HiReview reaches Structure 0.9484, Relevance 0.9428 and Coverage 0.9163. AutoSurvey scores 0.9122, 0.9093 and 0.8646 on the same metrics.
  • In the ablation, removing the taxonomy module lowers Structure to 0.8790 and Coverage to 0.8612. Removing retrieval gives the largest drop, with Coverage falling to 0.6705.
  • A case study on continual learning shows that the generated taxonomies are coherent but coarser than expert ones. For example, they lack legal and medical subcategories, so they are best used as drafts for experts to refine.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Emory UniversityAtlanta · United States36 works · 39,779 citations🎓Shanghai UniversityShanghai · China4 works · 124 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksDecision Sciences · 209 worksPhysics and Astronomy · 44 worksStatistical and Nonlinear Physics · 40 worksStatistics, Probability and Uncertainty · 61 works

Cite

Cite as Yuntong Hu, Zhuofeng Li, Zheng Zhang, Chen Ling, Raasikh Kanjiani, Boxin Zhao and Liang Zhao. 2024. Taxonomy Tree Generation from Citation Graph. arXiv:2410.03761.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2410.03761 · arXiv 2410.03761

@misc{hu2024taxonomy,
  title = {{Taxonomy Tree Generation from Citation Graph}},
  author = {Yuntong Hu, Zhuofeng Li, Zheng Zhang, Chen Ling, Raasikh Kanjiani, Boxin Zhao and Liang Zhao},
  year = {2024},
  doi = {10.48550/arxiv.2410.03761},
  eprint = {2410.03761},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.03761}
}

References

51 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Thomas Bonald, Bertrand Charpentier, Alexis Galland, and Alexandre Hollocou. 2018. Hierarchical Graph Clustering using Node Pair Sampling. In MLG 2018-14th International Workshop on Mining and Learning with Graphs. Continual Knowledge Editing Continual Alignment External Knowledge Retrieval-Based Li… find
  2. Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, Overcoming Catastrophic Forgetting and Yang Yang. 2023. Graphllm: Boosting graph reasoning ability of large lan-Lifelong Learning Strategies guage model. arXiv preprint arXiv:2310.05845 (2023). — [arXiv:2310.05845](https://arxiv.org/abs/2310.05845) find
  3. Jeroen De Knijff, Flavius Frasincar, and Frederik Hogenboom. 2013. Domain Continual Learning in Specific Domains taxonomy learning from text: The subsumption method versus hierarchical Continual Relation Engineering Extraction 83 (2013), 54–69. clustering. Data & Knowledge find
  4. Jörg Diederich and Wolf-Tilo Balke. 2007. The semantic growbag algorithm: Auto-Continual Semantic Segmentation matically deriving categorization systems. In Research and Advanced Technology Continual Object Detection for Digital Libraries: 11th European Conference, ECDL 2007, Budapest, Hungary, Sept… find
  5. Emmanuelle-Anna Dietz, Damir in Vandic, and Flavius Frasincar. 2012. Taxolearn: A semantic approach domain Learning taxonomy learning. In 2012 IEEE/WIC/ACM Applications of to Continual International Conferences on Web Intelligence and Intelligent Agent Technology, Large Language Model Editing and Tu… find
  6. Ning Ding, Yujia Continual in Task-Oriented Systems Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-Continual Detection efficient fine-tuning of Event large-scale pre-trained language models. Nature Machine Intelligence 5, 3 (2023), 220–235. Continual Text Classification find
  7. Christof Doll, Tanja Hartmann, and Dorothea Wagner. 2011. Fully-dynamic Learning using for Aspect Sentiment Classification Tasks and hierarchical Continual graph clustering cut trees. In Workshop on Algorithms *Figure 5: Taxonomy Tree Generated by HiReview.* Data Structures. Springer, 338–349. find
  8. Adel Elmahdy, Junhyung Ahn, Changho Suh, and Soheil Mohajer. 2020. Matrix completion with hierarchical graph side information. Advances in neural information processing systems 33 (2020), 9061–9074. find
  9. Gary William Flake, Robert E Tarjan, and Kostas Tsioutsiouliklis. 2004. Graph clustering and minimum cut trees. Internet Mathematics 1, 4 (2004), 385–408. find
  10. Zeyu Han, Chao Gao, Jinyang Liu, Sai Qian Zhang, et al. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 (2024).
  11. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In COLING 1992 volume 2: The 14th international conference on computational linguistics.
  12. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning. PMLR, 2790–2799. find
  13. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021).
  14. Subin Huang, Xiangfeng Luo, Jing Huang, Yike Guo, and Shengwei Gu. 2019. An unsupervised approach for learning a Chinese IS-A taxonomy from an unstructured corpus. Knowledge-Based Systems 182 (2019), 104861. find
  15. Yong-Bin Kang, Pari Delir Haghigh, and Frada Burstein. 2015. Taxofinder: A graph-based approach for taxonomy learning. IEEE Transactions on Knowledge and Data Engineering 28, 2 (2015), 524–536. find
  16. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  17. Zornitsa Kozareva and Eduard Hovy. 2010. A semi-supervised method to learn and construct taxonomies using the web. In Proceedings of the 2010 conference on empirical methods in natural language processing. 1110–1118. find
  18. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 3045–3059.
  19. Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 4582–4597. find
  20. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys 55, 9 (2023), 1–35.
  21. Xueqing Liu, Yangqiu Song, Shixia Liu, and Haixun Wang. 2012. Automatic taxonomy construction from keywords. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 1433–1441. find
  22. Zheyuan Liu, Xiaoxin He, Yijun Tian, and Nitesh V Chawla. 2024. Can we soft prompt LLMs for graph learning tasks?. In Companion Proceedings of the ACM on Web Conference 2024. 481–484. find
  23. Félix Martel and Amal Zouaq. 2021. Taxonomy extraction using knowledge graph embeddings and hierarchical clustering. In Proceedings of the 36th Annual ACM Symposium on Applied Computing. 836–844. find
  24. Patrick Pantel and Marco Pennacchiotti. 2006. Espresso: Leveraging generic patterns for automatically harvesting semantic relations. In Proceedings of the 21st international conference on computational linguistics and 44th annual meeting of the Association for Computational Linguistics. 113–120.
  25. Patrick Pantel and Deepak Ravichandran. 2004. Automatically labeling semantic classes. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004. 321–328.
  26. Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. 2024. Let your graph do the talking: Encoding structured data for llms. arXiv preprint arXiv:2402.05862 (2024). — [arXiv:2402.05862](https://arxiv.org/abs/2402.05862) find
  27. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247 (2020).
  28. Simone Paolo Ponzetto and Roberto Navigli. 2010. Knowledge-rich word sense disambiguation rivaling supervised systems. In Proceedings of the 48th annual meeting of the association for computational linguistics. 1522–1531. find
  29. Simone Paolo Ponzetto and Michael Strube. 2011. Taxonomy induction based on a collaboratively built knowledge repository. Artificial Intelligence 175, 9-10 (2011), 1737–1756. find
  30. Jing Qiu, Yaqi Si, and Zhihong Tian. 2018. Automatic Taxonomy Construction for Eye Colors Data without Using Context Information. In 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). IEEE, 837–841. find
  31. N Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084 (2019).
  32. Ana B Rios-Alvarado, Ivan Lopez-Arevalo, and Victor J Sosa-Sosa. 2013. Learning concept hierarchies from textual resources for ontologies construction. Expert Systems with Applications 40, 15 (2013), 5907–5915. find
  33. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389. find
  34. Nico Schlitter, Tanja Falkowski, and Jörg Lässig. 2014. DenGraph-HO: a density-based hierarchical graph clustering algorithm. Expert Systems 31, 5 (2014), 469– 479. find
  35. Jingbo Shang, Xinyang Zhang, Liyuan Liu, Sha Li, and Jiawei Han. 2020. Nettaxo: Automated topic taxonomy construction from text-rich network. In Proceedings of the web conference 2020. 1908–1919. find
  36. Jiaming Shen, Zeqiu Wu, Dongming Lei, Chao Zhang, Xiang Ren, Michelle T Vanni, Brian M Sadler, and Jiawei Han. 2018. Hiexpan: Task-guided taxonomy construction by hierarchical tree expansion. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2180– 2… find
  37. Zhihong Shen, Hao Ma, and Kuansan Wang. 2018. A Web-scale system for scientific knowledge exploration. In Proceedings of ACL 2018, System Demonstrations. 87–92. find
  38. Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. 2020. Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509 (2020). — [arXiv:2009.03509](https://arxiv.org/abs/2009.03509) find
  39. Prachi Singh and Sriram Ganapathy. 2025. End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization. IEEE Transactions on Audio, Speech and Language Processing (2025). find
  40. Rion Snow, Dan Jurafsky, and Andrew Y Ng. 2006. Semantic taxonomy induction from heterogenous evidence. In Proceedings of the 21st international conference on computational linguistics and 44th annual meeting of the association for computational linguistics. 801–808. find
  41. Fabian M Suchanek, Georgiana Ifrim, and Gerhard Weikum. 2006. Combining linguistic and statistical analysis to extract relations from web documents. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 712–717. find
  42. Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web. 697–706. find
  43. László Szilágyi, Sándor Miklós Szilágyi, and Béat Hirsbrunner. 2014. A fast and memory-efficient hierarchical graph clustering algorithm. In International Conference on Neural Information Processing. Springer, 247–254. find
  44. Paola Velardi, Stefano Faralli, and Roberto Navigli. 2013. Ontolearn reloaded: A graph-based algorithm for taxonomy induction. Computational Linguistics 39, 3 (2013), 665–707. find
  45. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  46. Wei Wang, Payam Mamaani Barnaghi, and Andrzej Bargiela. 2009. Probabilistic topic models for learning terminological ontologies. IEEE Transactions on knowledge and Data engineering 22, 7 (2009), 1028–1040. find
  47. Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, et al. 2024. AutoSurvey: Large Language Models Can Automatically Write Surveys. arXiv preprint arXiv:2406.10252 (2024). find
  48. Jianping Zeng and Hongfeng Yu. 2015. Parallel modularity-based community detection on large-scale graphs. In 2015 IEEE International Conference on Cluster Computing. IEEE, 1–10. find
  49. Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, and Jiawei Han. 2018. Taxogen: Unsupervised topic taxonomy construction by adaptive term embedding and clustering. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data… find
  50. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations. find
  51. Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. 2024. Towards Lifelong Learning of Large Language Models: A Survey. arXiv preprint arXiv:2406.06391 (2024).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hu-2024-taxonomy-tree-generation-citation-graph/