Fuzhao Xue
Fuzhao Xue has 2 papers in our library, in 2023–2024, most of them on Language models & pretraining; often with Yang You, Wangchunshu Zhou, Yao Fu.
OpenAlex · h-index 11 · 32 works · 740 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 175 citations in our library · 1 more in our library without a page yet · ⭐ Paper Star rank #4707
Affiliations
🎓National University of SingaporeSingapore ·
Singapore2021–2024 · 11 papers🎓Nanyang Technological UniversitySingapore ·
Singapore2021–2022 · 3 papers🎓Harbin Engineering UniversityHarbin ·
China2019–2020 · 2 papers
Concepts, shelves and topics
📚 Shelves 6 Arts and Humanities · 2 works Computer Science · 2 works Artificial Intelligence · 2 works Computer Vision and Pattern Recognition · 2 works Language and Linguistics · 2 works Hardware and Architecture · 1 work
🧭 Topics 1 Language models & pretraining · 1 work
Works by year
2020s
Papers
- Sequence Parallelism: Long Sequence Training from System Perspective (2023 · 128 citations)
Self-attention memory grows quadratically with input length. Standard training setups also need each GPU to hold the entire sequence, which caps the usable context. Most prior work attacks this algorithmically with cheaper attention variants. This paper takes a systems approach instead. It introduces sequence… - OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models (2024 · 61 citations)
Before this work, few open Mixture-of-Experts (MoE) language models had been trained on trillion-scale, diverse data, and little was known about how their routers actually behave.
All works
- Sequence Parallelism: Long Sequence Training from System PerspectiveShenggui Li, Fuzhao Xue, Chaitanya Baranwal, et al.Annual Meeting of the Association for… · 128 citations
- OpenMoE: An Early Effort on Open Mixture-of-Experts Language ModelsFuzhao Xue, Zian Zheng, Yao Fu, et al.arXiv:2402.01739 · 61 citations
- MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (original ↗)Jinjie Ni et al.2024 · arXiv:2406.06565 · 42 citations · no summary yet
Co-authors here
Venues
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/fuzhao-xue/