Elamur Research⌕
☰
home / papers / 2026 / MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents

MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents

Paper · 2026 · Pengxiang Zhao, Guangyi Liu, Yaozhen Liang et al.

Summary

Mobile agents driven by multimodal LLMs usually act only through the interface: they tap, swipe and type step by step. This is flexible but slow. Shortcuts such as APIs, deep-links and RPA scripts can skip whole sequences of GUI steps, but no benchmark has systematically tested agents that mix the two.

The authors built MAS-Bench, an online Android benchmark that uses snapshot resets so runs are reproducible. Every task can be solved through the GUI alone, so the benchmark measures whether an agent chooses well between GUI actions and shortcuts. It also tests whether agents can create their own reusable shortcuts from past interaction trajectories, using hand-built shortcuts as a reference upper bound.

Hybrid agents beat GUI-only ones on both success and efficiency. Shortcuts generated by agents are still clearly less robust than predefined ones. The benchmark also detects and penalizes poor shortcut selection. This gives the field a common testbed for building faster and more dependable mobile agents.

Key points

  • Benchmark contents:
    • 139 complex tasks across 11 real-world apps.
    • A knowledge base of 88 predefined shortcuts: 11 APIs, 70 deep-links and 7 RPA scripts.
    • 9 evaluation metrics.
  • Hybrid agent results:
    • MAS-GLM-4.5V reached the top success rate of 68.3%.
    • MAS-MobileAgent rose from 35.2% to 63.3% SR compared with its GUI-only version.
    • The same agent needed 38.9% fewer steps on successful tasks.
  • Effect of model capability: Gains from shortcuts depend on how capable the base model is.
    • Qwen3-VL-8B gained the most (+64.1% relative SR).
    • Qwen3-VL-4B misused shortcuts (33.5% redundancy) and dropped 66.7% in relative SR when given irrelevant shortcuts.
    • GLM-4.5V dropped only 7.7% under the same irrelevant-shortcut condition.
  • Shortcut generation (T3A executor):
    • Predefined shortcuts raised SR from 0.43 to 0.52, with a 100% shortcut success rate.
    • They cut steps by about 25% and execution time by about 16%.
    • Replay-Task shortcuts worked only 10% of the time.
    • Dynamic shortcuts reached 38% SR but showed strong efficiency potential.
  • Limitations:
    • All evaluation ran on emulators; tests on real devices are left for future work.
    • Agents still need to fall back to GUI actions when a shortcut fails.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksHuman-Computer Interaction · 141 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Pengxiang Zhao, Guangyi Liu, Yaozhen Liang, Weiqing He, Zhengxi Lu, WenHao Wang, Yuehao Huang, Yuxiang Chai, Zhaolu Kang, Yaxuan Guo, Hao Wang, Kexin Zhang, Liang Liu and Yong Liu. 2026. MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.316

@inproceedings{zhao2026mas,
  title = {{MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents}},
  author = {Pengxiang Zhao, Guangyi Liu, Yaozhen Liang, Weiqing He, Zhengxi Lu, WenHao Wang, Yuehao Huang, Yuxiang Chai, Zhaolu Kang, Yaxuan Guo, Hao Wang, Kexin Zhang, Liang Liu and Yong Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.316},
  url = {https://doi.org/10.18653/v1/2026.acl-long.316}
}

References

38 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Simone Agostinelli, Marco Lupia, Andrea Marrella, and Massimo Mecella. 2022. Reactive synthesis of software robots in rpa from user interface logs. Computers in Industry, 142:103721. find
  2. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, and 1 others. 2025a. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631.
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025b. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  4. Jingxuan Chen, Derek Yuen, Bin Xie, Yuhao Yang, Gongwei Chen, Zhihao Wu, Li Yixing, Xurui Zhou, Weiwen Liu, Shuai Wang, and 1 others. 2024. Spa-bench: A comprehensive benchmark for smartphone agent evaluation. In NeurIPS 2024 Workshop on Open-World Agents. find
  5. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401.10935.
  6. Weiyang Guo, Zesheng Shi, Liye Zhao, Jiayuan Ma, Zeen Zhu, Junxian He, Min Zhang, and Jing Li. 2026. E3-tir: Enhanced experience exploitation for tool-integrated reasoning. Preprint, arXiv:2604.09455. — [arXiv:2604.09455](https://arxiv.org/abs/2604.09455) find
  7. Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, and 1 others. 2025. Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006.
  8. Shiting Huang, Zhen Fang, Zehui Chen, Siyu Yuan, Junjie Ye, Yu Zeng, Lin Chen, Qi Mao, and Feng Zhao. 2025a. Critictool: Evaluating self-critique capabilities of large language models in tool-calling error scenarios. arXiv preprint arXiv:2506.13977.
  9. Yuehao Huang, Liang Liu, Shuangming Lei, Yukai Ma, Hao Su, Jianbiao Mei, Pengxiang Zhao, Yaqing Gu, Yong Liu, and Jiajun Lv. 2025b. Cogddn: A cognitive demand-driven navigation with decision optimization and dual-process thinking. In Proceedings of the 33rd ACM International Conference on Multimedia… find
  10. Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei. 2025a. Think only when you need with large hybrid-reasoning models. arXiv preprint arXiv:2505.14631. — [arXiv:2505.14631](https://arxiv.org/abs/2505.14631) find
  11. Wenjia Jiang, Yangyang Zhuang, Chenxi Song, Xu Yang, Joey Tianyi Zhou, and Chi Zhang. 2025b. Appagentx: Evolving gui agents as proficient smartphone users. arXiv preprint arXiv:2503.02268. — [arXiv:2503.02268](https://arxiv.org/abs/2503.02268) find
  12. Courtney Kennedy and Stephen E Everett. 2011. Use of cognitive shortcuts in landline and cell phone surveys. Public Opinion Quarterly, 75(2):336–348. find
  13. Xiaoyuan Li, Keqin Bao, Yubo Ma, Moxin Li, Wenjie Wang, Rui Men, Yichang Zhang, Fuli Feng, Dayiheng Liu, and Junyang Lin. 2025. Mtr-bench: A comprehensive benchmark for multi-turn reasoning evaluation. arXiv preprint arXiv:2505.17123.
  14. Zhicong Li, Lingjie Jiang, Yulan Hu, Xingchen Zeng, Yixia Li, Xiangwen Zhang, Guanhua Chen, Zheng Pan, Xin Li, and Yong Liu. 2026. No more stale feedback: Co-evolving critics for open-world agent learning. arXiv preprint arXiv:2601.06794.
  15. Guangyi Liu, Pengxiang Zhao, Liang Liu, Zhiming Chen, Yuxiang Chai, Shuai Ren, Hao Wang, Shibo He, and Wenchao Meng. 2025a. Learnact: Few-shot mobile gui agent with a unified demonstration benchmark. arXiv preprint arXiv:2504.13805.
  16. Guangyi Liu, Pengxiang Zhao, Liang Liu, Yaxuan Guo, Han Xiao, Weifeng Lin, Yuxiang Chai, Yue Han, Shuai Ren, Hao Wang, and 1 others. 2025b. Llm-powered gui agents in phone automation: Surveying progress and prospects. arXiv preprint arXiv:2504.19838. — [arXiv:2504.19838](https://arxiv.org/abs/2504.19838) find
  17. Zhaoyang Liu, JingJing Xie, Zichen Ding, Zehao Li, Bowen Yang, Zhenyu Wu, Xuehui Wang, Qiushi Sun, Shi Liu, Weiyun Wang, and 1 others. 2025c. Scalecua: Scaling open-source computer use agents with cross-platform data. arXiv preprint arXiv:2509.15221. — [arXiv:2509.15221](https://arxiv.org/abs/2509.15221) find
  18. Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Pengxiang Zhao, Guangyi Liu, and 1 others. 2026. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40,… find
  19. Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, and 1 others. 2025. Ui-s1: Advancing gui automation via semi-online reinforcement learning. arXiv preprint arXiv:2509.11543. — [arXiv:2509.11543](https://arxiv.org/abs/2509.11543) find
  20. OpenAI. 2024. Gpt-4 technical report. arXiv:2303.08774.
  21. Preprint, Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, and 1 others. 2024. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arXiv:2405.14573. find
  22. ByteDance Seed. 2025. seed-tars.com/1.5. find
  23. Ui-tars-1.5. https://Hao Wen, Shizuo Tian, Borislav Pavlov, Wenjie Du, Yixuan Li, Ge Chang, Shanhui Zhao, Jiacheng Liu, Yunxin Liu, Ya-Qin Zhang, and 1 others. 2024. Autodroid-v2: Boosting slm-based gui agents via code generation. arXiv preprint arXiv:2412.18116. — [arXiv:2412.18116](https://arxiv.org/abs/2412.18116) find
  24. Han Xiao, Guozhi Wang, Yuxiang Chai, Zimu Lu, Weifeng Lin, Hao He, Lue Fan, Liuyang Bian, Rui Hu, Liang Liu, and 1 others. 2025. Ui-genie: A self-improving approach for iteratively boosting mllm-based mobile gui agents. arXiv preprint arXiv:2505.21496. — [arXiv:2505.21496](https://arxiv.org/abs/2505.21496) find
  25. Han Xiao, Guozhi Wang, Hao Wang, Shilong Liu, Yuxiang Chai, Yue Pan, Yufeng Zhou, Xiaoxin Chen, Yafei Wen, and Hongsheng Li. 2026. Ui-mem: Self-evolving experience memory for online reinforcement learning in mobile gui agents. arXiv preprint arXiv:2602.05832. — [arXiv:2602.05832](https://arxiv.org/abs/2602.05832) find
  26. Mingzhe Xing, Rongkai Zhang, Hui Xue, Qi Chen, Fan Yang, and Zhen Xiao. 2024. Understanding the weakness of large language model agents within a complex android environment. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6061–6072. find
  27. Chengbing Wang, Yang Zhang, Wenjie Wang, Xiaoyan Zhao, Fuli Feng, Xiangnan He, and Tat-Seng Chua. 2025a. Think-while-generating: On-the-fly reasoning for personalized long-form generation. arXiv preprint arXiv:2512.06690. — [arXiv:2512.06690](https://arxiv.org/abs/2512.06690) find
  28. Yifan Xu, Xiao Liu, Xueqiao Sun, Siyi Cheng, Hao Yu, Hanyu Lai, Shudan Zhang, Dan Zhang, Jie Tang, and Yuxiao Dong. 2024. Androidlab: Training and systematic benchmarking of android autonomous agents. arXiv preprint arXiv:2410.24024.
  29. Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024a. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. arXiv preprint arXiv:2406.01014. — [arXiv:2406.01014](https://arxiv.org/abs/2406.01014) find
  30. Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, and 1 others. 2025. Mobile-agent-v3: Fundamental agents for gui automation. arXiv preprint arXiv:2508.15144. — [arXiv:2508.15144](https://arxiv.org/abs/2508.15144) find
  31. Luyuan Wang, Yongyu Deng, Yiwei Zha, Guodong Mao, Qinmin Wang, Tianchen Min, Wei Chen, and Shoufa Chen. 2024b. Mobileagentbench: An efficient and user-friendly benchmark for mobile llm agents. arXiv preprint arXiv:2406.08184. — [arXiv:2406.08184](https://arxiv.org/abs/2406.08184) find
  32. Wenhao Wang, Peizhi Niu, Zhao Xu, Zhaoyu Chen, Jian Du, Yaxin Du, Xianghe Pang, Keduan Huang, Yanfeng Wang, Qiang Yan, and Siheng Chen. 2025b. Mcp-flow: Facilitating llm agents to master real-world, diverse and scaling mcp tools. Preprint, arXiv:2510.24284.
  33. Wenhao Wang, Zijie Yu, Rui Ye, Jianqing Zhang, Guangyi Liu, Liang Liu, Siheng Chen, and Yanfeng Wang. 2025c. Fedmabench: Benchmarking mobile gui agents on decentralized heterogeneous user data. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26398–264… find
  34. Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, and Heng Ji. 2025d. Mobile-agent-e: Self-evolving mobile assistant for complex tasks. arXiv preprint arXiv:2501.11733. — [arXiv:2501.11733](https://arxiv.org/abs/2501.11733) find
  35. Chaoyun Zhang, Shilin He, Liqun Li, Si Qin, Yu Kang, Qingwei Lin, and Dongmei Zhang. 2025a. Api agents vs. gui agents: Divergence and convergence. arXiv preprint arXiv:2503.11069. — [arXiv:2503.11069](https://arxiv.org/abs/2503.11069) find
  36. Chaoyun Zhang, He Huang, Chiming Ni, Jian Mu, Si Qin, Shilin He, Lu Wang, Fangkai Yang, Pu Zhao, Chao Du, and 1 others. 2025b. Ufo2: The desktop agentos. arXiv preprint arXiv:2504.14603. — [arXiv:2504.14603](https://arxiv.org/abs/2504.14603) find
  37. Li Zhang, Shihe Wang, Xianqing Jia, Zhihan Zheng, Yunhe Yan, Longxi Gao, Yuanchun Li, and Mengwei Xu. 2024. Llamatouch: A faithful and scalable testbed for mobile ui automation task evaluation. arXiv preprint arXiv:2404.16054. — [arXiv:2404.16054](https://arxiv.org/abs/2404.16054) find
  38. Hanzhang Zhou, Xu Zhang, Panrong Tong, Jianan Zhang, Liangyu Chen, Quyu Kong, Chenglin Cai, Chen Liu, Yue Wang, Jingren Zhou, and 1 others. 2025. Mai-ui technical report: Real-world centric foundation gui agents. arXiv preprint arXiv:2512.22047. — [arXiv:2512.22047](https://arxiv.org/abs/2512.22047) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-mas-bench-unified-benchmark-shortcut/