Wenge Rong
Wenge Rong has 2 papers in our library, in 2022–2023, most of them on Interpretability; often with Jie Zhou, Xiaofeng Zhang, Yikang Shen.
OpenAlex · h-index 6 · 17 works · 431 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 Cited 128 times in our library · ⭐ Paper Stars ranking: #7171
Affiliations
🎓Beihang UniversityBeijing ·
China2012–2025 · 14 papers🔬State Key Laboratory of Software Development EnvironmentBeijing ·
China2012–2023 · 3 papers
Fields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 2 worksComputer Science · 2 worksArtificial Intelligence · 2 worksLanguage and Linguistics · 2 worksComputer Vision and Pattern Recognition · 1 work
🧭 Topics 1Research subjects and languages the work is about.Interpretability · 1 work
Works by year
2020s
Papers
- Transformer-Patcher: One Mistake worth One Neuron (2023 · 123 citations)
Deployed language models keep producing new errors, yet most Model Editing (ME) research fixes only a single mistake at a time. - Mixture of Attention Heads: Selecting Attention Heads Per Token (2022 · 44 citations)
Sparse Mixture-of-Experts (MoE) layers in Transformers have mostly replaced the feed-forward block. Multi-head attention has received little such treatment, even though earlier analyses showed that individual heads specialize. This paper introduces Mixture of Attention Heads (MoA), in which a pool of attention…
Co-authors here
- Jie Zhou ×2
- Xiaofeng Zhang ×2
- Yikang Shen ×2
- Zeyu Huang ×2
- Zhang Xiong ×2
Venues
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/wenge-rong/