Nayoung Lee
Nayoung Lee has 2 papers in our library, in 2023–2024, most of them on Language models & pretraining; often with Dimitris Papailiopoulos, Kangwook Lee, Jaewoong Cho.
OpenAlex · ORCID · h-index 4 · 20 works · 89 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 Cited 81 times in our library · ⭐ Paper Stars ranking: #8453
Affiliations
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Artificial Intelligence · 2 worksArts and Humanities · 2 worksComputer Science · 2 worksLanguage and Linguistics · 2 works
🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1 work
Works by year
2020s
Papers
- Teaching Arithmetic to Small Transformers (2023 · 57 citations)
The paper asks which factors let a next-token predictor acquire arithmetic. It uses a controlled setting: small decoder-only transformers (NanoGPT and GPT-2, from 10.6M up to 124M parameters) trained from random initialization with character-level tokens. Tasks include addition, subtraction, multiplication, square… - Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks (2024 · 41 citations)
In-context learning (ICL) has mostly been studied in Transformers. This paper asks whether attention-free state-space models (SSMs), chiefly Mamba and also S4, can acquire the same ability. Large-scale ICL is hard to test cheaply, so the authors follow the Garg et al. (2022) setup: small models are trained from…
Co-authors here
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/nayoung-lee/