Zijia Chen
Zijia Chen has 2 papers in our library, in 2024, most of them on Language models & pretraining, Efficient models & compression; often with Ameya Sunil Mahabaleshwarkar, Shizhe Diao, Yoshi Suhara.
OpenAlex · ORCID · h-index 2 · 9 works · 16 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 Cited 89 times in our library · ⭐ Paper Stars ranking: #6918
Affiliations
Fields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 2 worksComputer Science · 2 worksArtificial Intelligence · 2 worksComputer Vision and Pattern Recognition · 2 worksLanguage and Linguistics · 2 works
🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 2 worksEfficient models & compression · 1 work
Works by year
2020s
Papers
- Hymba: A Hybrid-head Architecture for Small Language Models (2024 · 50 citations)
Small language models must balance accuracy against memory and speed. Transformers recall context well, but their compute cost and KV cache grow quickly with sequence length. State space models such as Mamba run cheaply but recall poorly. Earlier hybrids stacked the two layer types one after another, so a weak layer… - LLM Pruning and Distillation in Practice: The Minitron Approach (2024 · 44 citations)
Minitron-style compression trains only the largest model in a family and derives smaller ones by structured pruning followed by knowledge distillation.
Co-authors here
- Ameya Sunil Mahabaleshwarkar ×2
- Shizhe Diao ×2
- Yoshi Suhara ×2
- Jan Kautz
- Jiaqi Zeng
- Xin Dong
- Yingyan Celine Lin
- Yonggan Fu
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/zijia-chen/