Elamur Research⌕
☰
home / papers / 2023 / Self-QA: Unsupervised Knowledge Guided Language Model Alignment

Self-QA: Unsupervised Knowledge Guided Language Model Alignment

Paper · 2023 · Xuanyu Zhang and Qing Yang

Summary

Instruction tuning usually depends on question–answer pairs written by people. Such data is costly to produce, and it is hard to keep it accurate, varied and focused on a domain. Bootstrapping methods like Self-Instruct reduce that cost, but they still begin from hand-made seed prompts and offer little control over topic or factual correctness.

Zhang and Yang (Du Xiaoman Financial) propose SELF-QA, which replaces human seeds with raw, unlabeled domain knowledge. The pipeline has three steps:

  1. A language model reads a passage and writes standalone questions about it.
  2. Acting as a reading-comprehension system, the model answers each question from that same passage.
  3. Rule-based filters remove pairs that break the format or still refer back to the source.

Tables and knowledge graphs are converted into plain sentences first. After fine-tuning BLOOM-7B on the resulting data, the model answered domain questions that ChatGPT got wrong. The evidence is illustrative rather than benchmarked. Still, the recipe is a cheap way to build domain-specific instruction data from existing corpora.

Key points

  • Stages: knowledge-guided instruction generation (the prompt asks for ten diverse questions without pronouns like "this"), machine reading comprehension modelled as P(A|K, Q), and heuristic filtering and pruning.
  • Structured data, such as company tables or knowledge-graph facts about Yoshua Bengio, is turned into text using slot-filling templates or by joining each value with its attribute name.
  • Comparison with other self-alignment methods: Self-Instruct uses 176 human seeds, Self-Align 195 seeds, and Self-Chat 111,502 supervised dialogues. Only SELF-QA claims domain customization and a correctness guarantee.
  • Experiment: BLOOM-7B was tuned on the generated data. On the question "Where is DXM?", it correctly answered Haidian District, Beijing, while ChatGPT said Hangzhou. No quantitative metrics are reported.
  • Merging question generation and answering into a single model call reduces the number of calls. The trade-offs are possible length overruns and less detailed answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksQuestion answering · 441 works

Cite

Cite as Xuanyu Zhang and Qing Yang. 2023. Self-QA: Unsupervised Knowledge Guided Language Model Alignment. arXiv:2305.11952.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.11952 · arXiv 2305.11952

@misc{zhang2023self,
  title = {{Self-QA: Unsupervised Knowledge Guided Language Model Alignment}},
  author = {Xuanyu Zhang and Qing Yang},
  year = {2023},
  doi = {10.48550/arxiv.2305.11952},
  eprint = {2305.11952},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.11952}
}

References

12 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Laurice M Joseph, Sheila Alber-Morgan, Jennifer Cullen, and Christina Rouse. 2016. The effects of self-questioning on reading comprehension: A literature review. Reading & Writing Quarterly, 32(2):152–173. find
  2. OpenAI. 2023. Gpt-4 technical report.
  3. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train…
  4. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277.
  5. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. find
  6. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. find
  7. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2023. Principle-driven self-alignment of language models from scratch with minimal human supervision. arXiv preprint arXiv:2305.03047.
  8. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560. find
  9. Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196.
  10. Xuanyu Zhang. 2019. MCˆ2: Multi-perspective convolutional cube for conversational machine reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6185–6190, Florence, Italy. Association for Computational Linguistics. find
  11. Xuanyu Zhang and Zhichun Wang. 2020. Rception: Wide and deep interaction networks for machine reading comprehension (student abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 34(10):13987–13988. find
  12. Xuanyu Zhang, Qing Yang, and Dongliang Xu. 2022. TranS: Transition-based knowledge graph embedding with synthetic relation representation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1202–1208, Abu Dhabi, United Arab Emirates. Association for Computational Linguis… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2023-self-qa-unsupervised-knowledge-guided/