Elamur Research⌕
☰
home / authors / Xuyang Shen

Xuyang Shen

Xuyang Shen has 2 papers in our library, in 2023–2024, most of them on Language models & pretraining; often with Dong Li, Weigao Sun, Weixuan Sun.

2 papers in our library · 2023–2024

OpenAlex · h-index 5 · 27 works · 139 citations OpenAlex, 2026-10-14

📄 2 works in our library · 🔗 Cited 184 times in our library · 1 more in our library without a page yet · ⭐ Paper Stars ranking: #7459

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

 Affiliations

🎓Australian National UniversityCanberra · Australia2020–2022 · 6 papers🔬Shanghai Artificial Intelligence LaboratoryShanghai · China2024 · 2 papers🎓Imperial College LondonLondon · United Kingdom2022 · 2 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 4 more institutions on single bylines are not listed.

 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 2 worksComputer Science · 2 worksArtificial Intelligence · 2 worksLanguage and Linguistics · 2 worksComputer Vision and Pattern Recognition · 1 workHardware and Architecture · 1 work

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 2 works

 Works by year

2020s

20241 paper20231 paper

 Papers

The most cited first (OpenAlex and our library).

  1. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer (2023 · 33 citations)
    Linear attention promises to remove the quadratic cost of softmax attention. In practice, earlier linear-attention models have trailed Transformers in language-modeling quality and rarely ran faster on real hardware. The authors close this gap with TransNormerLLM, a redesigned version of their TransNormer…
  2. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models (2024 · 31 citations)
    Linear attention should cost O(n) instead of the O(n²) of softmax attention. In causal language modeling, however, existing implementations lose this advantage: the kernel trick needs a cumulative summation (cumsum) that parallelizes poorly on GPUs. Lightning Attention-1 had already reduced the memory-I/O bottleneck…

All works

3 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

  1. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormerarXiv:2307.14995Zhen Qin, Dong Li, Weigao Sun, et al.2023 · 33 citations
  2. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language ModelsarXiv:2401.04658Zhen Qin, Weigao Sun, Dong Li, et al.2024 · 31 citations
  3. Linear Attention Sequence Parallelism ↗arXiv:2404.02882 · no summary yetWeigao Sun et al.2024 · 12 citations

 Co-authors here

8 co-authors with a page, the most papers together first.

 Cited by

Papers of our library cite their works 184 times (of the 149,439 papers whose bibliographies we read); 45 citing papers with a page here →.

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/authors/xuyang-shen/