Elamur Research⌕
☰
home / papers / 2023 / Understanding Finetuning for Factual Knowledge Extraction from Language Models

Understanding Finetuning for Factual Knowledge Extraction from Language Models

Paper · 2023 · Mehran Kazemi, Sid Mittal and Deepak Ramachandran

Summary

Finetuning pretrained LMs on known facts is seen as a promising way to extract factual knowledge for knowledge graph (KG) construction, but its side effects are poorly understood. The authors study finetuned masked LMs on cloze-style relational queries, using the manual prompts of Petroni et al. (LAMA). They identify Frequency Shock: after finetuning, the model is skewed by how often each answer entity appeared in the finetuning data. Entities that are normally rare but show up in training get over-predicted, and common entities that are underrepresented get under-predicted. A related effect, Range Shift, pushes predictions toward entities seen as training answers. To isolate these effects, they build three datasets whose train–test answer statistics diverge to different degrees. Past a certain mismatch, these harms can cancel out the gains from task learning. Two remedies are tested: combining the finetuned model with a zero- or few-shot model, and finetuning jointly on masked language modeling. Used together, they improve Hit@1 by 12.4% relative to vanilla finetuning. This gives a practical recipe for building KGs from LMs.

Key points

  • The analysis splits finetuning into three effects: Task Learning (beneficial), Frequency Shock and Range Shift (potentially harmful). It also argues that some errors previously blamed on forgetting are better explained by Frequency Shock.
  • The three datasets are LowMismatch, MediumMismatch and HighMismatch:
    • Entity coverage is 83.8 / 95.8 / 0.0.
    • The Pearson correlation of answer frequencies between train and test is 0.68 / 0.30 / −0.02.
  • Mixture finetuning combines the knowledge task with masked language modeling at an α:β batch ratio, for example 1:15. It helped on all three datasets, and most on HighMismatch.
  • Macro-average Hit@1 across the datasets:
    • Zero-shot (ZS): 30.0
    • Few-shot (FS): 38.9
    • Vanilla finetuning (FT): 37.8
    • FT+FS: 42.2
    • Mixture-finetuned 1:15 + FS: 42.5 (best)
  • Recommendations depend on budget:
    • Without cost constraints, use a mixture-finetuned model combined with FS.
    • To avoid the cost of mixture training, use FT+FS.
    • If only one model can be called at inference, use the mixture-finetuned model alone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Mehran Kazemi, Sid Mittal and Deepak Ramachandran. 2023. Understanding Finetuning for Factual Knowledge Extraction from Language Models. arXiv:2301.11293.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.11293 · arXiv 2301.11293

@misc{kazemi2023understanding,
  title = {{Understanding Finetuning for Factual Knowledge Extraction from Language Models}},
  author = {Mehran Kazemi, Sid Mittal and Deepak Ramachandran},
  year = {2023},
  doi = {10.48550/arxiv.2301.11293},
  eprint = {2301.11293},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.11293}
}

References

49 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031, 2022. find
  2. Emily Allaway, Jena D Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, and Yejin Choi. Penguins don’t fly: Reasoning about generics through instantiations and exceptions. arXiv preprint arXiv:2205.11658, 2022. find
  3. Sumithra Bhakthavatsalam, Chloe Anastasiades, and Peter Clark. Genericskb: A knowledge base of generic statements. arXiv preprint arXiv:2005.00660, 2020. find
  4. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.
  5. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. Comet: Commonsense transformers for automatic knowledge graph construction. arXiv preprint arXiv:1906.05317, 2019.
  6. Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, and Jin Xu. Knowledgeable or educated guess? revisiting language models as knowledge bases. arXiv preprint arXiv:2106.09231, 2021.
  7. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. Toward an architecture for never-ending language learning. In Twenty-Fourth AAAI conference on artificial intelligence, 2010. find
  8. Jeff Da, Ronan Le Bras, Ximing Lu, Yejin Choi, and Antoine Bosselut. Analyzing commonsense emergence in few-shot knowledge models. arXiv preprint arXiv:2101.00297, 2021. — [arXiv:2101.00297](https://arxiv.org/abs/2101.00297) find
  9. Joe Davison, Joshua Feldman, and Alexander M Rush. Commonsense knowledge mining from pretrained models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1173–1178, 2…
  10. Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining,… find
  11. Leandra Fichtel, Jan-Christoph Kalo, and Wolf-Tilo Balke. Prompt tuning or fine-tuning-investigating relational knowledge in pre-trained language models. In 3rd Conference on Automated Knowledge Base Construction, 2021. find
  12. Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. Mrqa 2019 shared task: Evaluating generalization in reading comprehension. arXiv preprint arXiv:1910.09753, 2019.
  13. Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker, and Pascal Poupart. Diachronic embedding for temporal knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3988–3995, 2020. find
  14. Tejas Gokhale, Pratyay Banerjee, Chitta Baral, and Yezhou Yang. Mutant: A training paradigm for out-of-distribution generalization in visual question answering. arXiv preprint arXiv:2009.08566, 2020. find
  15. Shibo Hao, Bowen Tan, Kaiwen Tang, Hengzhe Zhang, Eric P Xing, and Zhiting Hu. Bertnet: Harvesting knowledge graphs from pretrained language models. arXiv preprint arXiv:2206.14268, 2022. — [arXiv:2206.14268](https://arxiv.org/abs/2206.14268) find
  16. Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James Glass, and Fuchun Peng. Analyzing the forgetting problem in pretrain-finetuning of open-domain dialogue response models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main… find
  17. Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423–438, 2020.
  18. Seyed Mehran Kazemi and David Poole. Simple embedding for link prediction in knowledge graphs. Advances in neural information processing systems, 31, 2018. find
  19. Corentin Kervadec, Grigory Antipov, Moez Baccouche, and Christian Wolf. Roses are red, violets are blue... but should vqa expect them to? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2776–2785, 2021. find
  20. Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. Tensor decompositions for temporal knowledge base completion. arXiv preprint arXiv:2004.04926, 2020. find
  21. Patrick Lewis, Pontus Stenetorp, and Sebastian Riedel. Question and answer test-train overlap in open-domain question answering datasets. arXiv preprint arXiv:2008.02637, 2020. find
  22. Xiang Li, Aynaz Taheri, Lifu Tu, and Kevin Gimpel. Commonsense knowledge base completion. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1445–1455, 2016.
  23. Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. Birds have four legs?! numersense: Probing numerical commonsense knowledge of pre-trained language models. arXiv preprint arXiv:2005.00683, 2020a.
  24. Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang Ren, and William W Cohen. Differentiable open-ended commonsense reasoning. arXiv preprint arXiv:2010.14439, 2020b. find
  25. Linqing Liu, Patrick Lewis, Sebastian Riedel, and Pontus Stenetorp. Challenges in generalization in open domain question answering. arXiv preprint arXiv:2109.01156, 2021.
  26. George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995.
  27. Nona Naderi, Julien Knafou, Jenny Copara, Patrick Ruch, and Douglas Teodoro. Ensemble of deep masked language models for effective named entity recognition in health and life science corpora. Frontiers in research metrics and analytics, 6, 2021. find
  28. Benjamin Newman, Prafulla Kumar Choubey, and Nazneen Rajani. P-adapters: Robustly extracting factual information from language models with diverse prompts. arXiv preprint arXiv:2110.07280, 2021. — [arXiv:2110.07280](https://arxiv.org/abs/2110.07280) find
  29. Christopher Ormerod. Short-answer scoring with ensembles of pretrained language models. arXiv preprint arXiv:2202.11558, 2022. — [arXiv:2202.11558](https://arxiv.org/abs/2202.11558) find
  30. Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019.
  31. Raj Ratn Pranesh, Ambesh Shekhar, and Smita Pallavi. Quesbelm: A bert based ensemble language model for natural questions. In 2020 5th International Conference on Computing, Communication and Security (ICCCS), pp. 1–5. find
  32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019.
  33. Tara Safavi and Danai Koutra. Relational world knowledge representation in contextual language models: A review. arXiv preprint arXiv:2104.05837, 2021. find
  34. Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. Atomic: An atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3027–3035, 2019… find
  35. Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021. — [arXiv:2108.13624](https://arxiv.org/abs/2108.13624) find
  36. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020.
  37. Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI conference on artificial intelligence, 2017. find
  38. Mujeen Sung, Jinhyuk Lee, Sean Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. Can language models be biomedical knowledge bases? arXiv preprint arXiv:2109.07154, 2021.
  39. Alon Talmor and Jonathan Berant. Multiqa: An empirical investigation of generalization and transfer in reading comprehension. arXiv preprint arXiv:1905.13453, 2019.
  40. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In International conference on machine learning, pp. 2071–2080. PMLR, 2016.
  41. Denny Vrandečić and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85, 2014.
  42. Jonas Wallat, Jaspreet Singh, and Avishek Anand. Bertnesia: Investigating the capture and forgetting of knowledge in bert. arXiv preprint arXiv:2106.02902, 2021. find
  43. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022.
  44. Peter West, Chandra Bhagavatula, Jack Hessel, Jena D Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. arXiv preprint arXiv:2110.07178, 2021.
  45. Vikas Yadav, Steven Bethard, and Mihai Surdeanu. If you want to go far go together: Unsupervised joint candidate evidence retrieval for multi-hop question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T… find
  46. Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. Geomlama: Geo-diverse commonsense probing on multilingual pre-trained language models. arXiv preprint arXiv:2205.12247, 2022. find
  47. Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, and Dan Roth. Do language embeddings capture scales? arXiv preprint arXiv:2010.05345, 2020. find
  48. Zexuan Zhong, Dan Friedman, and Danqi Chen. Factual probing is [mask]: Learning vs. learning to recall. arXiv preprint arXiv:2104.05240, 2021.
  49. Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang. Evaluating commonsense in pre-trained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 9733–9740, 2020. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kazemi-2023-understanding-finetuning-factual-knowledge-extraction/