Elamur Research⌕
☰
home / papers / 2026 / PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

Paper · 2026 · Yibo Lyu, Gongwei Chen, Rui Shao et al.

Summary

GUI agents usually assume that users give complete, explicit instructions. Real users often leave out details they consider obvious from past behaviour. This paper defines a new task, PersonalAlign, which treats implicit intent as hierarchical. At the first level, the agent fills in omitted preferences in vague instructions. At the second level, it anticipates routine actions from the user's current state (time, scenario) and offers suggestions without any instruction. To evaluate this, the authors build AndroidIntent, a benchmark drawn from real long-term personal Android usage logs. Preferences and routines were annotated through a two-stage pipeline that first filters candidates with quantitative scores and then has humans verify them. The authors also propose HIM-Agent, a memory framework that keeps a user memory updated incrementally and separates it into preference and routine layers. Experiments with GPT-5, Qwen3-VL and UI-TARS show that vague instructions sharply reduce execution success and that most agents produce unreliable proactive suggestions. HIM-Agent improves both abilities. The work matters because it moves GUI agents from purely reactive execution toward personalization based on a user's history.

Key points

  • AndroidIntent data: about 20k long-term records from multiple users, with 775 user-specific preferences and 215 routines annotated. Each user's records are split chronologically: 80% serve as history and 20% as executing records.
  • HIM-Agent components: a Streaming Aggregation Module, an Execution-based Preference Filter and a State-based Routine Filter. Together they produce Preference Intent Memory and Routine Intent Memory.
  • Effect of vague instructions: removing preference details lowers type accuracy by only about 3%, but SSR falls by roughly 20% and CSR by roughly 45%.
  • Main gains: HIM-Agent improves execution by 15.7% and proactive performance by 7.3%. Built on Qwen3-VL, it reaches a CSR of 42.3. Run with GPT-5.1, it balances 81.4% recall against a 49% false-alarm rate.
  • Ablation and cost: the full preference filter adds 9.1% CSR. Removing the state filter pushes false alarms to nearly 70%. The LLM-UM profiling baseline needs about 6518 extra tokens, while HIM-Agent stays efficient.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China158 works · 28,841 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksHuman-Computer Interaction · 141 works

Cite

Cite as Yibo Lyu, Gongwei Chen, Rui Shao, Weili Guan and Liqiang Nie. 2026. PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1669

@inproceedings{lyu2026personalalign,
  title = {{PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records}},
  author = {Yibo Lyu, Gongwei Chen, Rui Shao, Weili Guan and Liqiang Nie},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1669},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1669}
}

References

35 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, and 1 others. 2025. Llava-onevision-1.5: Fully open framework for democratized multimodal training. arXiv preprint arXiv:2509.23661. — [arXiv:2509.23661](https://arxiv.org/abs/2509.23661) find
  2. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. Qwen3-vl technical report. Preprint, a…
  3. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025b. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — Qwen2.5-VL Technical Report
  4. Chongyu Bao, Ruimin Dai, Yangbo Shen, Runyang Jian, Jinghan Zhang, Xiaolan Liu, and Kunpeng Liu. 2025. Galaxy: A cognition-centered framework for proactive, privacy-preserving, and self-evolving llm agents. arXiv preprint arXiv:2508.03991. — [arXiv:2508.03991](https://arxiv.org/abs/2508.03991) find
  5. Gongwei Chen, Xurui Zhou, Rui Shao, Yibo Lyu, Kaiwen Zhou, Shuai Wang, Wentao Li, Yinchuan Li, Zhongang Qi, and Liqiang Nie. 2025a. Less is more: Empowering gui agent with context-aware simplification. In Proceedings of the IEEE/CVF International Conference on Computer Vision. find
  6. Jingxuan Chen, Derek Yuen, Bin Xie, Yuhao Yang, Gongwei Chen, Zhihao Wu, Li Yixing, Xurui Zhou, Weiwen Liu, Shuai Wang, and 1 others. 2025b. Spa-bench: A comprehensive benchmark for smartphone agent evaluation. In The Thirteenth International Conference on Learning Representations. find
  7. Pengzhou Cheng, Zheng Wu, Zongru Wu, Aston Zhang, Zhuosheng Zhang, and Gongshen Liu. 2025a. Os-kairos: Adaptive interaction for mllm-powered gui agents. arXiv preprint arXiv:2503.16465. find
  8. Ziming Cheng, Zhiyuan Huang, Junting Pan, Zhaohui Hou, and Mingjie Zhan. 2025b. Navi-plus: Managing ambiguous gui navigation tasks with follow-up. arXiv preprint arXiv:2503.24180. — [arXiv:2503.24180](https://arxiv.org/abs/2503.24180) find
  9. Herbert H Clark. 1996. Using language. Cambridge university press. find
  10. Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. 2025. Memory os of ai agent. arXiv preprint arXiv:2506.06326.
  11. Wei Li, William E Bishop, Alice Li, Christopher Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. 2024a. On the effects of data scale on ui control agents. Advances in Neural Information Processing Systems, 37:92130–92154. find
  12. Wei Li, Renshan Zhang, Rui Shao, Jie He, and Liqiang Nie. 2025. Cogvla: Cognition-aligned vision-language-action model via instruction-driven routing & sparsification. In Advances in Neural Information Processing Systems. find
  13. Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. 2024b. Optimus-1: Hybrid multimodal memory empowered agents excel in long-horizon tasks. Advances in neural information processing systems, 37:49881–49913. find
  14. Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. 2025. Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory. arXiv preprint arXiv:2508.09736. — [arXiv:2508.09736](https://arxiv.org/abs/2508.09736) find
  15. Quanfeng Lu, Wenqi Shao, Zitao Liu, Lingxiao Du, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, and Ping Luo. 2025a. Guiodyssey: A comprehensive dataset for cross-app gui navigation on mobile devices. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pag… find
  16. Yaxi Lu, Shenzhi Yang, Cheng Qian, Guirong Chen, Qinyu Luo, Yesai Wu, Huadong Wang, Xin Cong, Zhong Zhang, Yankai Lin, and 1 others. 2025b. Proactive agent: Shifting llm agents from reactive responses to active assistance. In The Thirteenth International Conference on Learning Representations. find
  17. Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, and 1 others. 2024. Tell me more! towards implicit user intention understanding of language model driven agents. arXiv preprint arXiv:2402.09205. find
  18. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, and 1 others. 2025. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326.
  19. Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William E Bishop, Wei Li, Folawiyo Campbell-Ajala, and 1 others. Androidworld: A dynamic benchmarking environment for autonomous agents. In The Thirteenth International Conference on Learni… find
  20. Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. 2023. An-droidinthewild: A large-scale dataset for android device control. Advances in Neural Information Processing Systems, 36:59708–59728. find
  21. Yunxiao Shi, Wujiang Xu, Zeqi Zhang, Xing Zi, Qiang Wu, and Min Xu. 2025. Personax: A recommendation agent oriented user modeling framework for long behavior sequence. arXiv preprint arXiv:2503.02398. find
  22. Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, and 1 others. 2025a. Ui-tars-2 technical report: Advancing gui agent with multi-turn reinforcement learning. arXiv preprint arXiv:2509.02544.
  23. Lu Wang, Di Zhang, Fangkai Yang, Pu Zhao, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, and 1 others. 2025b. Lettingo: Explore user profile generation for recommendation system. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.… find
  24. Zheng Wu, Heyuan Huang, Yanjia Yang, Yuanyi Song, Xingyu Lou, Weiwen Liu, Weinan Zhang, Jun Wang, and Zhuosheng Zhang. 2025. Quick on the uptake: Eliciting implicit intents from human demonstrations for personalized mobile-use agents. arXiv preprint arXiv:2508.08645. — [arXiv:2508.08645](https://arxiv.org/abs/2508.08645) find
  25. Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and Yu Qiao. 2024. OS-ATLAS: A Foundation Action Model for Generalist GUI Agents. arXiv preprint. ArXiv:2410.23218 [cs].
  26. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, and 1 others. 2024. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37…
  27. Yuquan Xie, Zaijing Li, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, Dongmei Jiang, and Liqiang Nie. 2025. Mirage-1: Augmenting and updating gui agent with hierarchical multimodal skills. arXiv preprint arXiv:2506.10387. — [arXiv:2506.10387](https://arxiv.org/abs/2506.10387) find
  28. Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Hinrich Schütze, Volker Tresp, and Yunpu Ma. 2025. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint arXiv:2508.19828.
  29. Bufang Yang, Lilin Xu, Liekang Zeng, Kaiwei Liu, Siyang Jiang, Wenrui Lu, Hongkai Chen, Xiaofan Jiang, Guoliang Xing, and Zhenyu Yan. 2025a. Contextagent: Context-aware proactive llm agents with open-world sensory perceptions. arXiv preprint arXiv:2505.14668. — [arXiv:2505.14668](https://arxiv.org/abs/2505.14668) find
  30. Qinglong Yang, Haoming Li, Haotian Zhao, Xiaokai Yan, Jingtao Ding, Fengli Xu, and Yong Li. 2025b. Fingertip 20k: A benchmark for proactive and personalized mobile llm agents. arXiv:2507.21071. arXiv preprint Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu… — [arXiv:2507.21071](https://arxiv.org/abs/2507.21071) find
  31. Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, and 1 others. 2025. Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259. — [arXiv:2507.02259](https://arxiv.org/abs/2507.02259) find
  32. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  33. Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. 2025. Mem1: Learning to synergize memory and reasoning for efficient long-horizon agents. arXiv preprint arXiv:2506.15841. — [arXiv:2506.15841](https://arxiv.org/abs/2506.15841) find
  34. Yijie Zhu, Jie He, Rui Shao, Kaishen Yuan, Tao Tan, Xiaochen Yuan, and Zitong Yu. 2026a. Deltavla: Prior-guided vision-language-action models via world knowledge variation. arXiv preprint arXiv:2603.08361. — [arXiv:2603.08361](https://arxiv.org/abs/2603.08361) find
  35. Yijie Zhu, Rui Shao, Ziyang Liu, Jie He, Jizhihui Liu, Jiuru Wang, and Zitong Yu. 2026b. H-gar: A hierarchical interaction framework via goal-driven observation-action refinement for robotic manipulation. In Proceedings of the AAAI Conference on Artificial Intelligence. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lyu-2026-personalalign-hierarchical-implicit-intent-alignment/