Elamur Research⌕
☰
home / authors / Kaiyue Wen

Kaiyue Wen

Kaiyue Wen has 2 papers in our library, in 2022–2025, most of them on Language models & pretraining, Interpretability; often with Bo Zheng, Dayiheng Liu, Ivan Titov.

2 papers in our library · 2022–2025

OpenAlex · ORCID · h-index 3 · 31 works · 109 citations OpenAlex, 2026-10-14

📄 2 works in our library · 🔗 Cited 183 times in our library · 8 more in our library without a page yet · ⭐ Paper Stars ranking: #7812

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

 Affiliations

🎓Stanford UniversityStanford · United States2024–2026 · 17 paperscurrent🏢Huawei TechnologiesShenzhen · China2026 · 1 papercurrent🎓Shanghai University of Finance and EconomicsShanghai · China2026 · 1 papercurrent🎓Tsinghua UniversityBeijing · China2022–2024 · 4 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine.

 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Artificial Intelligence · 2 worksArts and Humanities · 2 worksComputer Science · 2 worksLanguage and Linguistics · 2 works

🧭 Topics 2Research subjects and languages the work is about.Interpretability · 1 workLanguage models & pretraining · 1 work

 Works by year

2020s

20251 paper20221 paper

 Papers

The most cited first (OpenAlex and our library).

  1. Finding Skill Neurons in Pre-trained Transformer-based Language Models (2022 · 85 citations)
    Pre-trained Transformers perform well across many NLP tasks, but it is unclear where task-relevant abilities sit among their parameters.
  2. Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (2025 · 21 citations)
    Mixture-of-Experts LLMs rely on a Load-balancing Loss (LBL) so that the router spreads tokens across experts.

All works

10 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

  1. On Transferability of Prompt Tuning for Natural Language Processing ↗Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · no summary yetYusheng Su et al.2022 · 107 citations
  2. Finding Skill Neurons in Pre-trained Transformer-based Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, et al.2022 · 85 citations
  3. Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Zihan Qiu, Zeyu Huang, Bo Zheng, et al.2025 · 21 citations
  4. Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective ↗arXiv (Cornell University) · no summary yetKaiyue Wen et al.2024 · 18 citations
  5. RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval ↗arXiv:2402.18510 · no summary yetKaiyue Wen, Xingyu Dang, Kaifeng Lyu2024 · 17 citations
  6. Fantastic Pretraining Optimizers and Where to Find Them ↗arXiv (Cornell University) · no summary yetKaiyue Wen et al.2025 · 11 citations
  7. Weight Ensembling Improves Reasoning in Language Models ↗arXiv:2504.10478 · COLM 2025 · no summary yetXingyu Dang et al.2025 · 10 citations
  8. Overtrained Language Models Are Harder to Fine-Tune ↗arXiv (Cornell University) · no summary yetJacob M. Springer et al.2025 · 8 citations
  9. Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization ↗arXiv (Cornell University) · no summary yetKaiyue Wen et al.2026 · 2 citations
  10. Priced Guidance: Can Language Models Generate Future Research Ideas? ↗arXiv (Cornell University) · no summary yetKaiyue Wen, Tengyu Ma, Percy Liang2026

 Co-authors here

12 of 13 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

 Cited by

Papers of our library cite their works 183 times (of the 149,439 papers whose bibliographies we read); 26 citing papers with a page here →.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/kaiyue-wen/