Elamur Research⌕
☰
home / papers / 2026 / Financial Causal QA via Instruction and Prompt Tuning of Gemma3-12B

Financial Causal QA via Instruction and Prompt Tuning of Gemma3-12B

Paper · 2026 · Avinash Trivedi and Chindukuri Mallikarjuna

Summary

The paper describes team Sarang's entry to the FinCausal 2026 shared task. Given a financial passage and a question, a system must return the cause or the effect it asks about as a span taken from the passage. Answers are scored by an LLM-as-a-judge metric (LLM Score) that rewards meaning rather than exact span overlap. The authors used the English FinCausal 2026 QA dataset and compared two families of approaches. The first was fine-tuning extractive QA encoders (RoBERTa and DeBERTa checkpoints), optionally followed by an LLM pass that checks the answer and corrects it against the context. The second was few-shot prompting of local instruction-tuned LLMs run through Ollama. Their best system did not update any weights. It tuned the instruction and demonstrations for Gemma3-12B automatically with DSPy's MIPROv2 optimizer. This reached an LLM Score of 4.54 and placed 9th. The study suggests that automated prompt optimization of a mid-sized open model can beat task-specific fine-tuning on causal financial QA.

Key points

  • Data: The English FinCausal 2026 QA set has 2,000 training and 500 test instances. Each instance contains an ID, Context, Question and Answer, and the test set has no answers.
  • Fine-tuning baselines:
    • consciousAI/question-answering-roberta-base-s scored 4.136, rising to 4.288 with the LLM answer-correction step.
    • deepset/deberta-v3-large-squad2 scored 4.292, rising to 4.302 with the same step.
  • Few-shot LLMs without tuning: gemma2:latest scored 4.424 and qwen3:latest scored 4.418, both above every fine-tuned model.
  • Prompt optimization with MIPROv2 on gemma3:12b: "light" optimization gave 4.448 and "medium" gave 4.54. The best run used auto=medium, max_bootstrapped_demos=10 and max_labeled_demos=10.
  • Note on the title: despite the mention of "instruction tuning", Gemma3-12B itself was never trained. Gains came only from optimizing the prompt and demonstrations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: SRM University, Andhra Pradesh.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 441 works

Cite

Cite as Avinash Trivedi and Chindukuri Mallikarjuna. 2026. Financial Causal QA via Instruction and Prompt Tuning of Gemma3-12B. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/24xxkstbkeqa

@inproceedings{trivedi2026financial,
  title = {{Financial Causal QA via Instruction and Prompt Tuning of Gemma3-12B}},
  author = {Avinash Trivedi and Chindukuri Mallikarjuna},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/24xxkstbkeqa},
  url = {https://doi.org/10.63317/24xxkstbkeqa}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  2. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural…
  3. Team Gemma and Google DeepMind. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. find
  4. Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. 2022. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP. arXiv preprint arXiv:2212.14024.
  5. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. Dspy: Compiling declarative language model calls into self-improving pipelines.
  6. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  7. Man Luo, Kazuma Hashimoto, Semih Yavuz, Zhiwei Liu, Chitta Baral, and Yingbo Zhou. 2022. Choose your qa model wisely: A systematic study of generative and extractive readers for question answering. In Proceedings of the 1st Workshop on Semiparametric Methods in NLP: Decoupling Logic from Knowledge,… find
  8. Dominique Mariko, Hanna Abi-Akl, Estelle Labidurie, Stephane Durfort, Hugues De Mazancourt, and Mahmoud El-Haj. 2020. The financial document causality detection shared task (fincausal 2020). In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisat… find
  9. Neurips 2020 efficientqa competition: Systems, analyses and lessons learned. In Proceedings of the NeurIPS 2020 Competition and Demonstration Track, volume 133 of PMLR, pages 86–111. find
  10. Antonio Moreno-Sandoval, Blanca Carbajo Coronado, Jordi Porta Zamorano, Yanco Amor Torterolo Orta, and Doaa Samy. 2025. The financial document causality detection shared task (fincausal 2025). In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNL… find
  11. Antonio Moreno-Sandoval, Jordi Porta, Yanco Torterolo, Alexia Stanescu, Melina Chatzi, and Sofía Roseti. 2026a. find
  12. The Financial Document Causality Detection Shared Task (FinCausal 2026). In Proceedings of the 7th Financial Narrative Processing Workshop (FNP 2026) at LREC 2026, Palma de Mallorca, Spain. ELRA.
  13. Antonio Moreno-Sandoval, Yanco Amor Torterolo Orta, Maria Alexia Stanescu, and Melina Chatzi. 2026b. find
  14. The Financial Document Causality Detection Shared Task (FinCausal 2026): Dataset.
  15. Antonio Moreno-Sandoval, Jordi Porta Zamorano, Blanca Carbajo-Coronado, Doaa Samy, Dominique Mariko, and Mahmoud El-Haj. 2023. The financial document causality detection shared task (fincausal 2023). In Proceedings of the 2023 IEEE International Conference on Big Data (Big-Data 2023), pages 2855–286… find
  16. Dominique Mariko, Hanna Abi-Akl, Kim Trottier, and Mahmoud El-Haj. 2022. The financial causality extraction shared task (fincausal 2022). In Proceedings of the 4th Financial Narrative Processing Workshop @LREC2022, pages 105– 107, Marseille, France. European Language Resources Association. find
  17. Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), pages 9308–9319. Association f…
  18. Dominique Mariko, Hanna Abi Akl, Estelle Labidurie, Stephane Durfort, Hugues de Mazan-Avinash Trivedi, Gauri Toshniwal, Sivanesan Sangeetha, and SR Balasundaram. 2025. find
  19. Sarang at fincausal 2025: Contextual qa for financial causality detection combining extractive and generative models. In Proceedings of the Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FNP), and the 1st Workshop on L… find
  20. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/trivedi-2026-financial-causal-qa-instruction-prompt/