Elamur Research⌕
☰
home / papers / 2026 / Reviving Iterative Refinement in Diffusion-based NER with an Initializer-Restorer Approach

Reviving Iterative Refinement in Diffusion-based NER with an Initializer-Restorer Approach

Paper · 2026 · Long Hai Trieu, Phí Minh Hieu and Makoto Miwa

Summary

Diffusion-based NER, as in DiffusionNER, treats entity spans as continuous coordinates that are denoised from Gaussian noise over several reverse steps. The authors find that when training uses Exponential Moving Average (EMA), the model on ACE2004 does best after only one inference step, and extra steps add nothing or lower accuracy. This means the iterative refinement that motivates diffusion goes largely unused. To fix this, they propose a two-stage Initializer-Restorer pipeline. A conventional NER model (BERT-Tagger or GLiNER) first proposes candidate spans. A conditional diffusion Restorer then begins its reverse process from these candidates instead of from random noise. The Restorer is trained on a hybrid input that combines gold spans, Initializer predictions and pure Gaussian noise. Experiments cover ACE2004, GENIA and CleanCoNLL and compare against a re-run DiffusionNER baseline under the same conditions. The method beats this baseline on all three datasets, and its accuracy now rises as more restoration steps are added. This suggests that an informed starting point restores the practical value of multi-step diffusion for entity extraction.

Key points

  • Baseline behaviour (ACE2004 dev F1): the EMA-trained baseline peaks at γ=1 (88.51%) and falls to 87.92% by γ=10. The BERT-initialized Restorer goes the other way, rising from 85.74% at γ=1 to 88.65% at γ=10.
  • Training strategies on ACE2004: GT-Only reaches 85.5 F1, Pred-Only 87.8, and Hybrid 88.62. Mixing input types is what makes the Restorer work.
  • CleanCoNLL test results:
    • With GLiNER as initializer, the method scores 94.70 F1, compared with 93.79 for the baseline.
    • GLiNER alone scores only 71.21, and BERT alone 90.18, which the Restorer raises to 94.37.
  • ACE2004 test results: BERT initialization gives 88.62 F1, slightly above the re-run baseline's 88.28. Qualitatively, the Restorer mainly fixes span boundaries and removes false positives.
  • Setup and limitations:
    • The diffusion backbone is BERT-large, or BioBERT-large for GENIA, with k=180 candidate boxes and EMA decay of 0.9999.
    • All results come from a single random seed, and only English data was used.
    • The two-stage, multi-step inference adds latency.
    • DeBERTa-v3-large performed worse than BERT-large as a backbone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom243 works · 121,396 citations🎓Toyota Technological InstituteNagoya · Japan9 works · 3,330 citations🔬National Institute of Advanced Industrial Science and TechnologyTsukuba · Japan17 works · 2,760 citations🤝Cancer Research UKLondon · United Kingdom2 works · 42 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Named entity recognition · 200 works

Cite

Cite as Long Hai Trieu, Phí Minh Hieu and Makoto Miwa. 2026. Reviving Iterative Refinement in Diffusion-based NER with an Initializer-Restorer Approach. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-short.72

@inproceedings{trieu2026reviving,
  title = {{Reviving Iterative Refinement in Diffusion-based NER with an Initializer-Restorer Approach}},
  author = {Long Hai Trieu, Phí Minh Hieu and Makoto Miwa},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-short.72},
  url = {https://doi.org/10.18653/v1/2026.acl-short.72}
}

References

5 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbo…
  3. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. 2023. DiffuSeq: Sequence to sequence text generation with diffusion models. In The Eleventh International Conference on Learning Representations. find
  4. Mexico City, Mexico. Association for Computational Linguistics. A Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations.
  5. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Advances in neural information processing systems, volume 33, pages 6840–6851. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/trieu-2026-reviving-iterative-refinement-diffusion-based/