Elamur Research⌕
☰
home / papers / 2026 / Agentic Context Strategies for Multi-Format Document Understanding: When Should Language Models Use Tools?

Agentic Context Strategies for Multi-Format Document Understanding: When Should Language Models Use Tools?

Paper · 2026 · Mansi Uniyal, Mukul Singh and Ryan Nadel

Summary

The paper asks how document content should be handed to large language models in enterprise question answering. Should the whole file go into the prompt, should only retrieved chunks be passed, or should the model explore the document with tools? The authors compare four strategy families, split into six conditions: Full Compressed, RAG Chunked (with and without tools), Full + Tools, and Outline + Tools (with or without an upfront outline). They test these on three benchmarks, each standing in for an office format: DocFinQA for Word, TAT-QA for Excel and SlideVQA for PowerPoint. Six frontier models from OpenAI and Anthropic are evaluated. The tool-equipped models work in a ReAct-style loop with search, read, preview and list functions.

Giving models active tool access raises accuracy sharply. On DocFinQA, RAG with tools reaches 46%, while static RAG reaches only 6%. These gains hold across formats and model families. The authors argue that cheap passive retrieval is inadequate for numerically demanding financial QA, even though agentic setups add considerable latency.

Key points

  • Scale of the study: 108 model×strategy×format configurations and 5,400 runs in total.
    • Models: GPT-4o, GPT-4.1, GPT-5, Claude Opus 4, Sonnet 4.5 and Haiku 4.5.
    • Tool calls were capped at 15 per question, and roughly 50 examples were used per configuration.
  • Gains from tools by format: about +40 points for Word, +35 for Excel and +28 for PowerPoint. Withholding all upfront context (even the outline) gave the same score as supplying it, 42% in both cases.
  • Tools versus pure reasoning on DocFinQA:
    • Chain-of-thought scored 28% and extended thinking scored 32%, both below tools alone at 40%.
    • Combining extended thinking with tools gave the best result, 48%.
    • Abstentions fell from 24% to 14% with tools, but the authors note that confident errors increased.
  • Model differences: Claude models led every tool-augmented condition.
    • More tool calls did not mean better answers. Haiku averaged 16.4 calls and scored 22%, while Sonnet 4.5 averaged 12.6 calls and scored 46%.
    • Accuracy formed an inverted U, peaking at 51.5% with 7–10 calls.
  • Latency cost: Claude models needed 47–51 s per question and GPT models 7–19 s, compared with 1–8 s for passive baselines. Even so, tool-based setups formed the accuracy–latency Pareto frontier.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States325 works · 142,931 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Mansi Uniyal, Mukul Singh and Ryan Nadel. 2026. Agentic Context Strategies for Multi-Format Document Understanding: When Should Language Models Use Tools?. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.133

@inproceedings{uniyal2026agentic,
  title = {{Agentic Context Strategies for Multi-Format Document Understanding: When Should Language Models Use Tools?}},
  author = {Mansi Uniyal, Mukul Singh and Ryan Nadel},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.133},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.133}
}

References

27 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anonymous. 2024a. Aragog: Agentic retrieval-augmented generation with graph reasoning. Under review. find
  2. Anonymous. 2024b. Docagent: A hierarchical agent framework for multi-document question answering. Under review. find
  3. Anthropic. 2026. Claude opus 4.6 model card. https://assets.anthropic.com/m/64823ba7485345a7/ Claude-Opus-4-5-System-Card.pdf. Technical report. ↗
  4. Akari Asai and 1 others. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection. Preprint, arXiv:arXiv:2310.xxxxx.
  5. Sebastian Borgeaud and 1 others. 2022. Improving language models by retrieving from trillions of tokens. Preprint, arXiv:arXiv:2112.04426. find
  6. Wenhu Chen and 1 others. 2023. Complex question answering with retrieval-augmented generation: Limitations and analysis. Preprint, arXiv:arXiv:2305.xxxxx. find
  7. Zhiyu Chen and 1 others. 2021. Finqa: A dataset of numerical reasoning over financial data. In EMNLP.
  8. Karl Cobbe and 1 others. 2021. Training verifiers to solve math word problems. In arXiv preprint arXiv:2110.14168.
  9. Luyu Gao and 1 others. 2023. Pal: Program-aided language models. In ICML. find
  10. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. Ruler: What’s the real context size of your long-context language models? Preprint, arXiv:2404.06654.
  11. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP.
  12. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In NeurIPS.
  13. Gary Marchionini. 1995. Information seeking in electronic environments. 9. Cambridge university press. find
  14. Reiichiro Nakano and 1 others. 2021. Webgpt: Browser-assisted question-answering with human feedback. Preprint, arXiv:arXiv:2112.09332.
  15. OpenAI. 2025. Gpt-5 technical report. https://cdn.openai.com/gpt-5-system-card.pdf. Technical report. ↗
  16. Shishir G. Patil and 1 others. 2023. Gorilla: Large language model connected with massive apis. In NeurIPS.
  17. Timo Schick and 1 others. 2024. Toolformer: Language models can teach themselves to use tools. In ICLR.
  18. Ryo Tanaka and 1 others. 2023. Slidevqa: A dataset for document-level visual question answering on presentation slides. In ACL. find
  19. Harsh Trivedi and 1 others. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In ACL.
  20. Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. 2025. Inftythink: Breaking the length limits of long-context reasoning in large language models. Preprint, arXiv:2503.06692.
  21. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In ICLR.
  22. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  23. Fengbin Zhu and 1 others. 2021a. Docfinqa: A long-document financial question answering benchmark. In ACL. find
  24. Fengbin Zhu and 1 others. 2021b. Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance. In ACL.
  25. Fengbin Zhu and 1 others. 2024. Rag fails on numerical reasoning: An empirical study. Preprint, arXiv:arXiv:2403.xxxxx. find
  26. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. In Proceedings of ACL.
  27. Xueguang Ma and 1 others. 2023. Query rewriting for retrieval-augmented large language models. In EMNLP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/uniyal-2026-agentic-context-strategies-multi-format/