Elamur Research⌕
☰
home / papers / 2026 / SpanDiffusion: Flow Matching over Continuous Span Masks for Financial Causal Question Answering

SpanDiffusion: Flow Matching over Continuous Span Masks for Financial Causal Question Answering

Paper · 2026 · Georg Niess and Roman Kern

Summary

The paper addresses extractive causal question answering on financial texts for the FinCausal 2026 shared task. The task provides 4,000 bilingual English/Spanish training examples and scores outputs with an LLM-as-a-judge. Most systems predict answer boundaries token by token. The authors instead treat span extraction as generative denoising. They encode the start and end positions as two Gaussian-shaped curves over the context tokens. A separate 6-layer Transformer then learns to rebuild these curves from random noise, using cross-attention to features from a frozen DeBERTa-v3-large with LoRA adapters. Training uses rectified flow instead of DDPM, so sampling needs only 20 Euler steps.

Ablations over six diffusion configurations show that LoRA adaptation of the encoder matters most, and flow matching adds a further gain. However, a plain linear span head on the same encoder still beats the best diffusion model on Exact Match. The study is therefore a candid, partly negative result: diffusion gives inspectable step-by-step trajectories but does not yet repay its extra cost and complexity.

Key points

  • Data: FinCausal 2026 provides 2,000 English and 2,000 Spanish training samples, which the authors merged and split 90/10 for validation. The leaderboard test sets have 500 (EN) and 503 (ES) items.
  • Architecture:
    • Frozen DeBERTa-v3-large encoder (434M parameters).
    • LoRA adapters with r=16, adding 1.6M trainable parameters.
    • Denoiser with 121.8M parameters and DiT-style adaLN time conditioning.
    • Gaussian width σ=1.5 for the span masks.
  • Ablation (validation EM):
    • Soft-box target: 32.5%.
    • Dual-peak target: 42.5%.
    • Adding LoRA: 76.5% (+34 points).
    • Switching to flow matching: 82.0% (+5.5 points), with 20 instead of 50 sampling steps.
    • Longer training: 83.0%.
  • Baseline: a DeBERTa + LoRA linear span classifier reaches 85.8% EM. It uses only 2.7M trainable parameters (versus 123.4M for the diffusion model) and a single forward pass.
  • Leaderboard: the best submission, V3-Flow, scored 4.57 (EN) and 4.63 (ES) on the 1–5 judge scale, against 4.81 for the top system. The stronger baseline was not officially submitted.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Graz University of TechnologyGraz · Austria3 works · 5 citations

Also on the byline: Know Center Research GmbH.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 441 works

Cite

Cite as Georg Niess and Roman Kern. 2026. SpanDiffusion: Flow Matching over Continuous Span Masks for Financial Causal Question Answering. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4bm7v3ccgmjy

@inproceedings{niess2026spandiffusion,
  title = {{SpanDiffusion: Flow Matching over Continuous Span Masks for Financial Causal Question Answering}},
  author = {Georg Niess and Roman Kern},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/4bm7v3ccgmjy},
  url = {https://doi.org/10.63317/4bm7v3ccgmjy}
}

References

21 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  2. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. 2023. DiffuSeq: Sequence to sequence text generation with diffusion models. In Proceedings of the 11th International Conference on Learning Representations (ICLR). find
  3. Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. 2023. SSD-LM: Semi-autoregressive simplex-based diffusion language model for text generation and modular control. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 11575– 11596. find
  4. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In Proceedings of the 11th International Conference on Learning Representations (ICLR).
  5. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 6840–6851. find
  6. Jonathan Ho and Tim Salimans. 2022. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598.
  7. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shanen Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations (ICLR).
  8. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. 2022. Diffusion-LM improves controllable text generation. In Advances in Neural Information Processing Systems (NeurIPS), volume 35.
  9. Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. 2023. Flow matching for generative modeling. In Proceedings of the 11th International Conference on Learning Representations (ICLR).
  10. Xingchao Liu, Chengyue Gong, and Qiang Liu. 2023. Flow straight and fast: Learning to generate and transfer data with rectified flow. In Proceedings of the 11th International Conference on Learning Representations (ICLR).
  11. Antonio Moreno-Sandoval, Jordi Porta, Blanca Carbajo-Coronado, Yanco Torterolo, and Doaa Samy. 2025. The financial document causality detection shared task (FinCausal 2025). In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financi… find
  12. Antonio Moreno-Sandoval, Jordi Porta, Yanco Torterolo, Alexia Stanescu, Melina Chatzi, and Sofía Roseti. 2026. find
  13. The Financial Document Causality Detection Shared Task (FinCausal 2026). In Proceedings of the 7th Financial Narrative Processing Workshop (FNP 2026) at LREC 2026, Palma de Mallorca, Spain. ELRA. To appear.
  14. Antonio Moreno-Sandoval, Jordi Porta-Zamorano, Blanca Carbajo-Coronado, Doaa Samy, Dominique Mariko, and Mahmoud El-Haj. 2023. The financial document causality detection shared task (FinCausal 2023). In Proceedings of the 5th Financial Narrative Processing Workshop (FNP 2023) at the 2023 IEEE Intern… find
  15. Georg Niess, Houssam Razouk, Stasa Mandic, and Roman Kern. 2025. Addressing hallucination in causal Q&A: The efficacy of fine-tuning over prompting in LLMs. In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Proc… find
  16. William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4195–4205.
  17. Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander Rush, and Aditya Grover. 2024. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems (NeurIPS), 37.
  18. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. DiffusionNER: Boundary diffusion for named entity recognition. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3875– 3890, Toronto, Canad… find
  19. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. 8.
  20. Language Resource References Antonio Moreno-Sandoval, Yanco Amor Torterolo Orta, Maria Alexia Stanescu, and Melina Chatzi. 2026. find
  21. The Financial Document Causality Detection Shared Task (FinCausal 2026): Dataset.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/niess-2026-spandiffusion-flow-matching-over-continuous/