Xiaodong Han
Xiaodong Han has 3 papers in our library, in 2022–2024, most of them on Language models & pretraining, Efficient models & compression; often with Weixuan Sun, Yiran Zhong, Zhen Qin.
OpenAlex · ORCID · h-index 11 · 37 works · 433 citations OpenAlex, 2026-10-14
📄 3 works in our library · 🔗 Cited 177 times in our library · ⭐ Paper Stars ranking: #5770
Affiliations
Fields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 3 worksComputer Science · 3 worksArtificial Intelligence · 3 worksComputer Vision and Pattern Recognition · 3 worksLanguage and Linguistics · 3 works
🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 2 worksEfficient models & compression · 1 work
Works by year
2020s
Papers
- PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference (2024 · 91 citations)
Serving LLMs at scale is limited by GPU memory, and the KV cache often takes far more memory than the model weights. - The Devil in Linear Transformer (2022 · 67 citations)
Kernel-based linear transformers reduce attention cost from quadratic to linear in sequence length, but they usually lose accuracy compared with the standard softmax transformer. - TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer (2023 · 33 citations)
Linear attention promises to remove the quadratic cost of softmax attention. In practice, earlier linear-attention models have trailed Transformers in language-modeling quality and rarely ran faster on real hardware. The authors close this gap with TransNormerLLM, a redesigned version of their TransNormer…
Co-authors here
- Weixuan Sun ×2
- Yiran Zhong ×2
- Zhen Qin ×2
- Dong Li
- Dongxu Li
- Hai Zhao
- Lingpeng Kong
- Nick M. Barnes
- Shilin Zhang
- Weigao Sun
- Xiao Luo
- Xuyang Shen
Venues
- Conference on Empirical Methods in Natural Language Processing (EMNLP): EMNLP 2022
- Findings of the Association for Computational Linguistics: Findings of the Association for Computational Linguistics 2024
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/xiaodong-han/