Elamur Research⌕
☰
home / authors / Junyuan Shang

Junyuan Shang

Junyuan Shang has 4 papers in our library, in 2020–2025, most of them on Language models & pretraining; often with Shuohuan Wang, Yu Sun, Siyu Ding.

4 papers in our library · 2020–2025

OpenAlex · ORCID · h-index 13 · 67 works · 1,861 citations OpenAlex, 2026-10-14

📄 4 works in our library · 🔗 Cited 237 times in our library · 2 more in our library without a page yet · ⭐ Paper Stars ranking: #4136

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

 Affiliations

🏢BaiduBeijing · China2020–2026 · 26 paperscurrent🎓South China University of TechnologyGuangzhou · China2018–2026 · 21 paperscurrent🎓Peking UniversityBeijing · China2017–2021 · 10 papers🎓Georgia Institute of TechnologyAtlanta · United States2018–2019 · 2 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 1 more institution on single bylines are not listed.

 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 4 worksComputer Science · 4 worksArtificial Intelligence · 4 worksComputer Vision and Pattern Recognition · 4 worksLanguage and Linguistics · 4 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 2 works

 Works by year

2020s

20251 paper20212 papers20201 paper

 Papers

The most cited first (OpenAlex and our library).

  1. ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021 · 332 citations)
    Most very large language models, such as GPT-3, learn only from raw text and are trained auto-regressively.
  2. ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021 · 63 citations)
    Very large language models such as GPT-3 show that scale pays off. However, most are trained on plain text only, and their outputs are hard to steer or verify against real-world facts. This work scales up the knowledge-enhanced ERNIE 3.0 framework, which combines auto-encoding and auto-regressive networks and learns…
  3. ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2020 · 27 citations)
    Standard Transformers struggle with long documents because self-attention cost grows quadratically with input length.
  4. Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking (2025 · 16 citations)
    Larger LLMs bring diminishing returns, and small models hit capacity limits. Chain-of-Thought style test-time scaling also has a weakness: it only works if the model produces the critical tokens correctly. The authors trained GPT-2 on AQuA and tracked gradient norms layer by layer. Tokens the model gets right settle…

All works

6 works of theirs in our library, the most cited first; 4 with a page here, the others link their original until we summarise them.

  1. GAMENet: Graph Augmented MEmory Networks for Recommending Medication Combination ↗Proceedings of the AAAI Conference on Artificial Intelligence · no summary yetJunyuan Shang et al.2019 · 341 citations
  2. ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationarXiv:2107.02137Yu Sun, Shuohuan Wang, Shikun Feng, et al.2021 · 332 citations
  3. Pre-training of Graph Augmented Transformers for Medication Recommendation ↗International Joint Conference on Artificial Intelligence · no summary yetJunyuan Shang et al.2019 · 313 citations
  4. ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationarXiv:2112.12731Shuohuan Wang, Yu Sun, Yang Xiang, et al.2021 · 63 citations
  5. ERNIE-Doc: A Retrospective Long-Document Modeling TransformerarXiv:2012.15688Siyu Ding, Junyuan Shang, Shuohuan Wang, et al.2020 · 27 citations
  6. Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal ThinkingAnnual Meeting of the Association for Computational Linguistics (ACL)Yilong Chen, Junyuan Shang, Zhenyu Zhang, et al.2025 · 16 citations

 Co-authors here

12 of 18 co-authors with a page, the most papers together first.

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

 Cited by

Papers of our library cite their works 237 times (of the 149,439 papers whose bibliographies we read); 39 citing papers with a page here →.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/junyuan-shang/