A Survey on Efficient Inference for Large Language Models (2024 · 113 citations)
Running large language models is expensive. They need a lot of computation, memory traffic and storage, which raises latency, lowers throughput and makes deployment difficult on edge devices and costly in the cloud. This survey, by authors from Tsinghua University, Shanghai Jiaotong University and Infinigence-AI…
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation (2023 · 72 citations)
Answer generation in LLMs is slow largely because tokens are produced strictly one after another. This work tackles that bottleneck without modifying models, serving systems or hardware. Skeleton-of-Thought (SoT) is a two-stage prompting scheme inspired by how people outline before writing. First, the LLM drafts a…
SCVQ: Sparse-Compensated Vector Quantization for Large Language Models (2026)
Vector quantization (VQ) compresses LLM weights well at very low bit-widths, but it is slow in practice. Large codebooks and frequent table lookups consume memory bandwidth, and implicit-codebook alternatives depend on costly transforms such as Hadamard rotations. The authors propose SCVQ, a post-training…

Co-authors here

9 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

Papers of our library cite their works 149 times (of the 149,439 papers whose bibliographies we read); 33 citing papers with a page here →.

Updated 2 November 2026