Elamur Research⌕
☰
home / papers / 2026 / QRAFT: QLoRA Retrieval-Augmented Fine-Tuning for Causal Span Extraction in Financial Documents

QRAFT: QLoRA Retrieval-Augmented Fine-Tuning for Causal Span Extraction in Financial Documents

Paper · 2026 · Bavya Sarda, Pulkit Chatwal and Sonal Dabral

Summary

The paper describes a system for the English subtask of FinCausal 2026. Given a paragraph from a financial report and an abstractively phrased question about a cause or an effect, the system must return the answering span copied exactly from the text. The authors argue that a compact model, properly adapted, can compete with larger general-purpose ones. They convert 2,000 annotated training examples into Qwen's ChatML conversational template and fine-tune Qwen3-4B-Instruct-2507 with QLoRA. This combines 4-bit quantization with low-rank adapters and keeps hardware needs low. At inference, TF-IDF cosine similarity selects the context sentence closest to the question. That sentence is added to the prompt as an extra hint, and greedy decoding produces a deterministic answer. The organisers scored outputs with an LLM judge on a 1–5 adequacy scale, which rewards meaning rather than exact token overlap. The system scored 4.76, ranking 4th among nine teams. This suggests that small fine-tuned models paired with cheap sparse retrieval are a practical option for causal QA in specialised domains.

Key points

  • Data: The FinCausal 2026 English data comes from 2017 UK annual reports compiled by UCREL (Lancaster University), plus excerpts from the 2018 FinT-esp corpus.
    • Compared with earlier editions, it adds over 500 new fragments with complex, multi-hop causal chains (three or more events).
    • About 10% of the questions were reworded to require deeper reasoning.
    • The split is 2,000 training instances out of 2,500 in total.
  • Model and tuning: Qwen3-4B-Instruct-2507 was trained with QLoRA, using NF4 4-bit quantization and dropout 0.05. Adapters were placed on both attention (q, k, v, o) and feed-forward (gate, up, down) projections.
  • Inference: The model receives the full context plus the single sentence that best matches the question by TF-IDF. Decoding is greedy at temperature 0, which enforces verbatim, reproducible outputs.
  • Evaluation: FinCausal 2026 replaced the earlier SAS and Exact Match metrics with LLM-as-a-judge adequacy scoring on a 1–5 scale.
  • Result: The system scored 4.76 out of 5, placing 4th of nine teams on the English leaderboard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rajiv Gandhi Institute of Petroleum TechnologyRaebareli · India3 works · 3 citations

Also on the byline: Galgotias University.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Bavya Sarda, Pulkit Chatwal and Sonal Dabral. 2026. QRAFT: QLoRA Retrieval-Augmented Fine-Tuning for Causal Span Extraction in Financial Documents. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4a9pbop2c9nc

@inproceedings{sarda2026qraft,
  title = {{QRAFT: QLoRA Retrieval-Augmented Fine-Tuning for Causal Span Extraction in Financial Documents}},
  author = {Bavya Sarda, Pulkit Chatwal and Sonal Dabral},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/4a9pbop2c9nc},
  url = {https://doi.org/10.63317/4a9pbop2c9nc}
}

References

20 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dogu Araci. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. find
  2. Eduardo Blanco, Nuria Castell, and Dan I Moldovan. 2008. Causal relation extraction. In Lrec, volume 66, page 74. find
  3. Pulkit Chatwal, Amit Agarwal, and Ankush Mittal. 2025. Enhancing causal relationship detection using prompt engineering and large language models. In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FN… find
  4. Qing Cheng, Zefan Zeng, Xingchen Hu, Yuehang Si, and Zhong Liu. 2024. A survey of event causality identification: Taxonomy, challenges, assessment, and prospects. arXiv preprint arXiv:2411.10371. — [arXiv:2411.10371](https://arxiv.org/abs/2411.10371) find
  5. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient fine-tuning of quantized llms. In Advances in Neural Information Processing Systems (NeurIPS). find
  6. Krish Didwania, Pratinav Seth, Aditya Kasliwal, and Amit Agarwal. 2024. Agrillm: harnessing transformers for framer queries. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 179–187. find
  7. Quang Do, Yee Seng Chan, and Dan Roth. 2011. Minimally supervised event causality identification. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 294–303, Edinburgh, Scotland, UK. Association for Computational Linguistics. find
  8. Roxana Girju. 2003. Automatic detection of causal relations for question answering. In Proceedings of the ACL 2003 workshop on Multilingual summarization and question answering, pages 76–83.
  9. Andrew Gordon, Zornitsa Kozareva, and Melissa Roemmele. 2012. SemEval-2012 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the s…
  10. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474.
  11. Tim Loughran and Bill McDonald. 2011. When is a liability not a liability? textual analysis, dictionaries, and 10-ks. The Journal of finance, 66(1):35–65. extraction shared task (fincausal 2022). In Proceedings of the 4th Financial Narrative Processing Workshop@ LREC2022, pages 105–107. find
  12. Antonio Moreno-Sandoval, Blanca Carbajo-Coronado, Jordi Porta Zamorano, Yanco Amor Torterolo Orta, and Doaa Samy. 2025. The financial document causality detection shared task (fincausal 2025). In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNL… find
  13. Antonio Moreno-Sandoval, Jordi Porta-Zamorano, Blanca Carbajo-Coronado, Doaa Samy, Dominique Mariko, and Mahmoud El-Haj. 2023. The financial document causality detection shared task (fincausal 2023). In 2023 IEEE International Conference on Big Data (BigData), pages 2855– 2860. find
  14. Antonio Moreno-Sandoval, Yanco Amor Torterolo Orta, Maria Alexia Stanescu, and Melina Chatzi. 2026. find
  15. The Financial Document Causality Detection Shared Task (FinCausal 2026): Dataset.
  16. Qwen Team. 2025. Qwen3 technical report. find
  17. Deepak Uniyal, Amit Agarwal, Durga Toshniwal, and Dipanjan Deb. 2021. Dense vector embedding based approach to identify prominent disseminators from twitter data amid covid-19 outbreak. IEEE Transactions on Emerging Topics in Computational Intelligence, 5(3):308–320. find
  18. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595– 46623.
  19. Dominique Mariko, Hanna Abi-Akl, Estelle Labidurie, Stephane Durfort, Hugues De Mazancourt, and Mahmoud El-Haj. 2020. The financial document causality detection shared task (fincausal 2020). In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisat… find
  20. Dominique Mariko, Hanna Abi-Akl, Kim Trottier, and Mahmoud El-Haj. 2022. The financial causality find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sarda-2026-qraft-qlora-retrieval-augmented-fine/