Pengle Zhang
Pengle Zhang has 2 papers in our library, in 2024, most of them on Language models & pretraining; often with Chaojun Xiao, Guangxuan Xiao, Haofeng Huang.
OpenAlex · h-index 7 · 29 works · 188 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 Cited 185 times in our library · ⭐ Paper Stars ranking: #6469
Affiliations
Fields, topics and concepts
📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 2 worksComputer Science · 2 worksArtificial Intelligence · 2 worksComputer Vision and Pattern Recognition · 2 worksLanguage and Linguistics · 2 worksHardware and Architecture · 1 work
🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1 work
Works by year
2020s
Papers
- InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory (2024 · 126 citations)
LLMs pre-trained on windows of only a few thousand tokens break down on much longer inputs. Unseen positions push them out of distribution, and large amounts of irrelevant context distract attention. The usual fix, continued pre-training on long sequences, is expensive and can hurt short-context skills. InfLLM avoids… - SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration (2024 · 74 citations)
At long sequence lengths, attention's quadratic cost makes it the dominant source of latency in transformers.
Co-authors here
Venues
- International Conference on Learning Representations (ICLR): ICLR 2024
- Conference on Neural Information Processing Systems (NeurIPS): NeurIPS 2024
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/pengle-zhang/