Hua Ren Wu · page 2
Papers
- ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2020 · 27 citations)
Standard Transformers struggle with long documents because self-attention cost grows quadratically with input length. - ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2020 · 22 citations)
BERT-style masked language modeling learns mostly from fine-grained units, such as subwords in English or characters in Chinese. Earlier extensions mask whole spans, but they still predict each token of a span separately. This ignores how tokens inside an n-gram depend on one another and how different n-grams relate. - AttnPO: Attention-Guided Process Supervision for Efficient Reasoning (2026 · 18 citations)
Reasoning models trained with RLVR tend to overthink, producing long chains of thought with little benefit. - Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking (2025 · 16 citations)
Larger LLMs bring diminishing returns, and small models hit capacity limits. Chain-of-Thought style test-time scaling also has a weakness: it only works if the model produces the critical tokens correctly. The authors trained GPT-2 on AQuA and tracked gradient norms layer by layer. Tokens the model gets right settle… - DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications (2020 · 13 citations)
Neural machine reading comprehension (MRC) models often fail in real-world use. They may answer paraphrased questions inconsistently (over-sensitivity), get distracted by passage spans that share many words with the question (over-stability), or struggle outside their training domain (generalization). Earlier… - ERNIE 5.0 Technical Report (2026 · 7 citations)
Most large multimodal systems can read images, video or audio but still answer mainly in text. When generation is added, it usually comes from separate decoders attached to a language model, which can weaken core language skills. Baidu's ERNIE 5.0 addresses this with a single autoregressive model trained from scratch… - BeamLoRA: Beam-Constraint Low-Rank Adaptation (2025 · 6 citations)
Low-Rank Adaptation (LoRA) is an efficient way to fine-tune large language models, but its accuracy still has room to improve. - Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation (2026 · 2 citations)
Reinforcement learning has improved LLMs on math and short-answer QA, where outputs can be checked against a single correct target. - Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models (2026)
Under identical RL with verifiable rewards (RLVR) pipelines such as GRPO and DAPO, some model families (Qwen) improve markedly while others (Llama) gain little.
This page in Markdown → https://md.research.elamur.ai/free/en/authors/hua-ren-wu/page/2/