Elamur Research⌕
☰
home / papers / 2024 / Agent Workflow Memory

Agent Workflow Memory

Paper · 2024 · Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried et al.

Summary

LM-based web agents often fail at long, multi-step tasks. They usually treat each task in isolation and rely on a fixed set of examples, so they adapt poorly when tasks or websites change. Agent Workflow Memory (AWM) addresses this by letting the agent build reusable routines ("workflows") from earlier trajectories. A language model is prompted to pull shared sub-routines out of experiences, such as searching for a product, and to replace concrete values with placeholders. Each resulting workflow has a text description and a sequence of steps, and it is added to the agent's memory. AWM works in two modes. Offline, it learns workflows from annotated training examples. Online, it learns them on the fly from its own test-time predictions that an evaluator judges successful, so no supervision is needed. On Mind2Web and WebArena, AWM clearly outperforms strong baselines and generalizes across tasks, websites and domains. This matters because it shows agents can keep improving from their own experience without extra labelled data.

Key points

  • The evaluation uses Mind2Web and WebArena, which together contain over 1000 tasks from more than 200 domains, including travel, shopping and social media.
  • On WebArena, AWM achieves a 51.1% relative gain in success rate over the best published autonomous method. It also beats an approach that uses human-written workflows by 7.9%, and it needs fewer steps on solved tasks.
  • On Mind2Web cross-task, step-wise success rate rises by 24.6% in relative terms. On the cross-website and cross-domain splits, AWM leads by 8.9–14.0 absolute points, and the margin grows as the gap between training and test distributions widens.
  • Online induction compounds over time: simple workflows serve as building blocks for more complex ones. On the WebArena map split, the lead over the baseline reaches 22.5 points after only a few dozen examples.
  • About 7.3–7.4 workflows are induced, and test examples use them at a high rate (0.94 on WebArena, 0.91 on Mind2Web). Combining offline and online workflows produces results that fall between the two modes rather than adding up.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States365 works · 158,048 citations🎓Massachusetts Institute of TechnologyCambridge · United States159 works · 65,157 citations

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksInformation Systems · 523 works

Cite

Cite as Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried and Graham Neubig. 2024. Agent Workflow Memory. arXiv:2409.07429.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2409.07429 · arXiv 2409.07429

@misc{wang2024agent,
  title = {{Agent Workflow Memory}},
  author = {Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried and Graham Neubig},
  year = {2024},
  doi = {10.48550/arxiv.2409.07429},
  eprint = {2409.07429},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2409.07429}
}

References

31 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Matthew Bowers, Theo X. Olausson, Lionel Wong, Gabriel Grand, Joshua B. Tenenbaum, Kevin Ellis, and Armando Solar-Lezama. Top-down synthesis for library learning. Proc. ACM Program. Lang., 7(POPL), jan 2023. doi: 10.1145/3571234. URL https://doi.org/10.1145/ 3571234. find
  2. Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint arXiv:2305.17126, 2023. URL https://arxiv.org/pdf/ 2305.17126. ↗
  3. Michelene TH Chi, Paul J Feltovich, and Robert Glaser. Categorization and representation of physics problems by experts and novices. Cognitive science, 5(2):121–152, 1981.
  4. Michelene TH Chi, Robert Glaser, and Marshall J Farr. The nature of expertise. Psychology Press, 2014. find
  5. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/forum?id=kiYqbO3wqw. ↗
  6. Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H Laradji, Manuel Del Verme, Tom Marty, Léo Boisvert, Megh Thakkar, Quentin Cappart, David Vazquez, et al. Workarena: How capable are web agents at solving common knowledge work tasks? arXiv preprint arXiv:2403.07718, 2024. — [arXiv:2403.07718](https://arxiv.org/abs/2403.07718) find
  7. Kevin Ellis, Lionel Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lore Anaya Pozo, Luke Hewitt, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: growing generalizable, interpretable knowledge with wake–sleep bayesian program learning. Philosophical Transactions of the Royal Society A, 3… find
  8. Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. Autoguide: Automated generation and selection of state-aware guidelines for large language model agents. arXiv preprint arXiv:2403.08978, 2024. — [arXiv:2403.08978](https://arxiv.org/abs/2403.08978) find
  9. Gabriel Grand, Lionel Wong, Matthew Bowers, Theo X Olausson, Muxin Liu, Joshua B Tenenbaum, and Jacob Andreas. Lilo: Learning interpretable libraries by compressing and documenting code. arXiv preprint arXiv:2310.19791, 2023. find
  10. Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. Language models can teach themselves to program better. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=SaRj2ka1XZ3. ↗
  11. Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024. URL htt…
  12. Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9493–9500. find
  13. Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=ryTp3f-0-. ↗
  14. Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. Clin: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134, 2023. find
  15. Jiayuan Mao, Tomás Lozano-Pérez, Joshua B Tenenbaum, and Leslie Pack Kaelbling. Learning reusable manipulation strategies. In Conference on Robot Learning, pp. 1467–1483. PMLR, 2023. find
  16. Shikhar Murty, Christopher Manning, Peter Shaw, Mandar Joshi, and Kenton Lee. Bagel: Bootstrapping agents by guiding exploration with language. arXiv preprint arXiv:2403.08140, 2024. — [arXiv:2403.08140](https://arxiv.org/abs/2403.08140) find
  17. Junhyuk Oh, Satinder Singh, Honglak Lee, and Pushmeet Kohli. Zero-shot task generalization with multi-task deep reinforcement learning. In International Conference on Machine Learning, pp. 2661–2670. PMLR, 2017. find
  18. Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Autonomous evaluation and refinement of digital agents. arXiv preprint arXiv:2404.06474, 2024. find
  19. Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy P Lillicrap. An-droidinthewild: A large-scale dataset for android device control. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/forum?id=j4… ↗
  20. Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, et al. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arXiv:2405.14573, 2024. find
  21. Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of bits: An open-domain platform for web-based agents. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Resea… find
  22. Paloma Sodhi, SRK Branavan, and Ryan McDonald. Heap: Hierarchical policies for web actions using llms. arXiv preprint arXiv:2310.03720, 2023. — [arXiv:2310.03720](https://arxiv.org/abs/2310.03720) find
  23. Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, and Chao Zhang. Adaplanner: Adaptive planning from feedback with language models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=rnKgbKmelt. ↗
  24. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. URL https://openreview.net/forum?id=ehfRiF0R3a. ↗
  25. Zhiruo Wang, Zhoujun Cheng, Hao Zhu, Daniel Fried, and Graham Neubig. What are tools anyway? a survey from the language model perspective. In First Conference on Language Modeling, 2024b. URL https://openreview.net/forum?id=Xh1B90iBSR. ↗
  26. Zhiruo Wang, Graham Neubig, and Daniel Fried. TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In Forty-first International Conference on Machine Learning, 2024c. URL https://openreview.net/forum?id=DCNCwaMJjI. ↗
  27. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 20744–20757.…
  28. Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, and Jonathan Berant. Assistantbench: Can web agents solve realistic and time-consuming tasks? arXiv preprint arXiv:2407.15711, 2024. find
  29. Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arXiv preprint arXiv:2306.08647, 2023. — [arXiv:2306.08647](https://arxiv.org/abs/2306.08647) find
  30. Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. Synapse: Trajectory-as-exemplar prompting with memory for computer control. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Pc8AU1aF5e. ↗
  31. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents. In The Twelfth International Conference on Learning Representations, 202…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2024-agent-workflow-memory/