Elamur Research⌕
☰
home / papers / 2022 / Calibrating Factual Knowledge in Pretrained Language Models

Calibrating Factual Knowledge in Pretrained Language Models

Paper · 2022 · Qingxiu Dong, Damai Dai, Yifan Song et al.

Summary

Pretrained language models (PLMs) are often treated as implicit knowledge bases, but many of the facts they hold are wrong. This work asks how to repair such errors without retraining from scratch. The authors first introduce Contrastive Knowledge Assessment (CKA). For each fact, CKA compares the probability the model gives the true object under the correct relation prompt with the average probability under several manually written prompts that use contradictory relations. A low ratio marks the fact as false. They argue this avoids two weaknesses of rank-based LAMA probing: facts with multiple valid answers, and frequency bias.

The detected errors are then fixed with CaliNet. CaliNet is a small FFN-style module of extra key-value "calibration memory slots" whose output is added to an existing FFN layer. The original weights stay frozen. It is trained on paraphrased statements of the correct facts built from PARAREL templates.

Experiments on knowledge probing show efficient correction. In closed-book QA, the calibrated knowledge carries over after fine-tuning. The approach is task-agnostic, which makes it relevant for keeping models factually reliable.

Key points

  • CKA indicates that close to half of facts randomly sampled from T-REx are stored incorrectly in PLMs. A CKA score below 1 is the typical threshold for flagging a false fact.
  • Positive prompts come from LAMA templates. Negative prompts are handcrafted to have opposite meaning while still expecting the same entity type.
  • CaliNet's intermediate size is much smaller than the original FFN's, so it adds very few parameters.
  • Calibration data uses PARAREL (38 relations), filling each false triplet into more than five paraphrase templates.
  • On 50 human-written test facts, CaliNet cut the False Rate by 35.61%. Analysis shows it boosts both the correct token and its synonyms.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China100 works · 35,618 citations🎓University of California, Santa BarbaraSanta Barbara · United States35 works · 9,287 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China44 works · 10,162 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1,343 works

Cite

Cite as Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui and Lei Li. 2022. Calibrating Factual Knowledge in Pretrained Language Models. Findings of the Association for Computational Linguistics: EMNLP 2022.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-emnlp.438

@inproceedings{dong2022calibrating,
  title = {{Calibrating Factual Knowledge in Pretrained Language Models}},
  author = {Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui and Lei Li},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.438},
  url = {https://doi.org/10.18653/v1/2022.findings-emnlp.438}
}

References

28 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of EMNLP.
  2. Zied Bouraoui, José Camacho-Collados, and Steven Schockaert. 2020. Inducing relational knowledge from bert. In Proceedings of AAAI. find
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,…
  4. Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021a. find
  5. Knowledgeable or educated guess? revisiting language models as knowledge bases. In Proceedings of ACL.
  6. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021b. Editing factual knowledge in language models. In Proceedings of EMNLP. find
  7. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of ACL. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL.
  9. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. TACL, 9. find
  10. Hady ElSahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon S. Hare, Frédérique Laforest, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of LREC.
  11. Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. CoRR, abs/2203.14680.
  12. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of EMNLP. find
  13. Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of ACL. find
  14. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? TACL.
  15. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of EMNLP-IJCNLP.
  16. Nina Pörner, Ulli Waltinger, and Hinrich Schütze. 2019. Bert is not a knowledge base (yet): Factual knowledge vs. name-based reasoning in unsupervised qa. CoRR, abs/1911.03681. find
  17. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR.
  18. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020a. How much knowledge can you pack into the parameters of a language model? In Proceedings of EMNLP. find
  19. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020b. How much knowledge can you pack into the parameters of a language model? In Proceedings of EMNLP. find
  20. Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. 2021. K-adapter: Infusing knowledge into pre-trained models with adapters. In Findings of ACL-IJCNLP, pages 1405–1418.
  21. Yunzhi Yao, Shaohan Huang, Ningyu Zhang, Li Dong, Furu Wei, and Huajun Chen. 2022. Kformer: Knowledge injection in transformer feed-forward layers. CoRR, abs/2201.05742. — [arXiv:2201.05742](https://arxiv.org/abs/2201.05742) find
  22. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. Ernie: Enhanced language representation with informative entities. In Proceedings of ACL. find
  23. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. CoRR, abs/2012.00363. find
  24. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of ACL.
  25. Nayeon Lee, Belinda Z. Li, Sinong Wang, Wen-tau Yih, Hao Ma, and Madian Khabsa. 2020. Language models as fact checkers? In Proceedings of FEVER. find
  26. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual knowledge in GPT. CoRR, abs/2202.05262. find
  27. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022. Fast model editing at scale. In Proceedings of ICLR. find
  28. Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. In Proceedings of EMNLP-IJCNLP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dong-2022-calibrating-factual-knowledge-pretrained-language/