Elamur Research⌕
☰
home / papers / 2026 / LLM-as-Scheduler: Agentic Workflow Dynamic Scheduling

LLM-as-Scheduler: Agentic Workflow Dynamic Scheduling

Paper · 2026 · Dawei Xiang, Kexin Chu, Wenyan Xu et al.

Summary

Multi-agent LLM pipelines, whether hand-built or found automatically by ADAS or AFlow, usually push every query through the same long chain of generation, validation, refinement and repair steps. That chain costs a lot of tokens and time. The authors ran such a workflow on MBPP, HumanEval and GSM8K. Roughly half of the problems were already solved by the first agent, about a fifth needed the full chain, and the rest failed regardless. To exploit this skew, they built LLM-as-Scheduler (LAS), a policy layer placed over an existing workflow. It uses a two-level cascade:

  • Gate: a cheap check that runs after every agent output, combining scripted signals with a small DistilBERT-based judge.
  • Scheduler: a larger LLM, invoked only for borderline cases, that decides whether to stop early, test, refine or reroute.

Averaged over benchmarks, LAS halves token use and cuts latency by more than a third, losing at most 1.4 accuracy points against a strong fixed workflow. Dynamic per-query scheduling therefore makes agentic systems much cheaper to deploy.

Key points

  • On MBPP, LAS saves 63.4% of tokens and 41.9% of latency, with an accuracy drop of no more than 0.5 pp.
  • In the MBPP ablation against the AFLOW baseline:
    • AFLOW baseline: 94.2% accuracy, about 6643 tokens, 134.3 s.
    • Gate+LAS: 93.7%, about 2430 tokens, 78.0 s.
    • Gate alone: falls to 86.7% accuracy.
  • The gate adds no tokens and about 0.15 s per call. At its best threshold it reaches 70.64% accuracy, 98.60% recall and AUC 0.737, and its judge trains in under an hour.
  • The scheduler matches oracle routing labels 65.72% of the time. Only 2.7% of its decisions are harmful, meaning it skips a needed refinement step.
  • The authors list cross-task transfer and extension to RAG and tool-use benchmarks as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates138 works · 6,590 citations🎓University of ConnecticutStorrs · United States13 works · 357 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Dawei Xiang, Kexin Chu, Wenyan Xu, Wenhui Zhang and Wei Zhang. 2026. LLM-as-Scheduler: Agentic Workflow Dynamic Scheduling. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.581

@inproceedings{xiang2026llm,
  title = {{LLM-as-Scheduler: Agentic Workflow Dynamic Scheduling}},
  author = {Dawei Xiang, Kexin Chu, Wenyan Xu, Wenhui Zhang and Wei Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.581},
  url = {https://doi.org/10.18653/v1/2026.acl-long.581}
}

References

5 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  2. Lingjiao Chen, Matei Zaharia, and James Zou. 2024. Frugalgpt: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research.
  3. We highlight the inefficiency of fixed multi-agent workflows and propose LLM-as-Scheduler (LAS), a cascade controller that adapts execution per query via a lightweight gate and an LLM scheduler. Experiments show that LAS preserves nearly all accuracy while reducing token usage by 50.5% and latency b… find
  4. Mark Chen. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  5. Jasper Dekoninck, Maximilian Baader, and Martin Vechev. 2025. A unified approach to routing and cascading for llms. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 12987–13010. PMLR. find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xiang-2026-llm-as-scheduler-agentic-workflow/