Maosong Sun · page 2
Papers
- Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020 · 152 citations)
Relation extraction models are usually trained once on a fixed set of relation types, but new relations keep appearing in real text. - ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021 · 139 citations)
Standard pre-training objectives such as masked language modeling treat words as the basic unit. They do not explicitly teach models how entities relate to each other across a document. Citing DocRED-style statistics, the authors note that over 40% of relational facts can only be recovered by combining several… - Self-Knowledge Guided Retrieval Augmentation for Large Language Models (2023 · 135 citations)
Retrieval augmentation is often treated as universally helpful for large language models, but the authors show that retrieved passages can sometimes distract a model from answers it would otherwise get right. - InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory (2024 · 126 citations)
LLMs pre-trained on windows of only a few thousand tokens break down on much longer inputs. Unseen positions push them out of distribution, and large amounts of irrelevant context distract attention. The usual fix, continued pre-training on long sequences, is expensive and can hurt short-context skills. InfLLM avoids… - MoEfication: Transformer Feed-forward Layers are Mixtures of Experts (2022 · 115 citations)
Feed-forward networks (FFNs) hold roughly two-thirds of a Transformer's parameters, yet how they compute has received far less study than attention. - Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages (2024 · 30 citations)
LLMs trained mostly on English handle most other languages much worse. The usual remedy is to machine-translate instruction data into the target language and fine-tune on it. The authors argue this approach has three weaknesses: it ignores what the model already does well in its strongest language, it inherits… - Sub-Character Tokenization for Chinese Pretrained Language Models (2023 · 29 citations)
Chinese pretrained language models usually tokenize text either one character at a time or by running sub-word algorithms directly on raw characters. - ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025 · 24 citations)
Open-source multimodal LLMs read charts reasonably well, but they struggle to turn a chart image into plotting code. Their output often fails to run, picks the wrong chart type, or loses details such as axis sizes. The authors argue that code is a lossless description of a chart, unlike textual summaries. - ConPET: Continual Parameter-Efficient Tuning for Large Language Models (2023 · 14 citations)
Adapting large language models to a stream of new tasks is costly when every update changes all weights. Full fine-tuning also tends to erase skills learned on earlier tasks, a problem known as catastrophic forgetting. Song et al. propose ConPET, a framework for continual task adaptation of LLMs in which training… - Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering (2025 · 11 citations)
Instruction tuning can teach LLMs to hallucinate when the training examples contain knowledge the model never acquired in pre-training. - GATEAU: Selecting Influential Samples for Long Context Alignment (2025 · 5 citations)
Teaching LLMs to follow instructions over very long inputs is hard because human annotation of long instruction-following data is costly. - A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks (2026 · 4 citations)
LLM-based agents on long, multi-step tasks often flail. Without an overall plan, they fall into aimless trial-and-error and produce hallucinated actions. Prior fixes either teach planning implicitly via SFT or RL on the executor, which needs lots of data or converges slowly, or supply explicit guidance that depends… - Long-Chain Reasoning Distillation via Adaptive Prefix Alignment (2026 · 3 citations)
Small student models often learn poorly from the long, reflective chain-of-thought traces that reasoning models such as DeepSeek-R1 produce. - Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026 · 1 citation)
Distilling long Chain-of-Thought (CoT) traces from strong reasoning models into small models often works poorly. - FaithLens: Detecting and Explaining Faithfulness Hallucination (2026 · 1 citation)
LLMs often produce claims that contradict, distort, or go beyond the context they were given. This faithfulness hallucination is a problem for RAG and summarization. Current detectors are either expensive prompted LLMs or compact classifiers that return only a binary label, give no reasoning, and generalize unevenly… - StateX: Enhancing RNN Recall via Post-training State Expansion (2026 · 1 citation)
Recurrent language models such as linear attention and state-space models process long inputs at constant cost per token.
This page in Markdown → https://md.research.elamur.ai/free/en/authors/maosong-sun/page/2/