Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks (2023 · 71 citations)
Small language models are cheaper to run than API-only LLMs and better for privacy. However, they perform poorly when answers depend on large amounts of factual or domain knowledge, because their few parameters cannot store it all. Standard reasoning distillation trains a small model to reproduce rationales written…
Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation (2023 · 31 citations)
Pre-trained language models write fluent dialogue replies but often state wrong facts. Earlier KG-grounded systems feed the model every triplet linked to entities in the conversation, although most are irrelevant to the current turn. This wastes computation and invites errors. The authors present SURGE (SUbgraph…
Distilling LLM Agent into Small Models with Retrieval and Code Tools (2025 · 11 citations)
Small language models (sLMs) trained to imitate a large teacher's chain-of-thought (CoT) traces often fail on questions that need rare facts or exact arithmetic.

Co-authors here

4 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

Papers of our library cite their works 76 times (of the 149,439 papers whose bibliographies we read); 11 citing papers with a page here →.

Updated 30 October 2026