Elamur Research⌕
☰
home / papers / 2026 / PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes

PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes

Paper · 2026 · Yingyu Shan, Zeming Liu, Silin Li et al.

Summary

Smart-home assistants built on LLMs usually expect each command to be fully specified. In real households, people who interact repeatedly tend to drop details they assume are already shared, so their requests get shorter and more implicit over time. The authors frame this as a new task: interpreting progressively elliptical commands. They identify two sources of difficulty. One is referential ambiguity, where several household members mean different things by the same vague phrase. The other is intention ambiguity, where a user's preferences shift as the environment changes.

To study the task, they release PEC-Home, a simulated dataset. GPT-4o generated the commands inside a virtual home, and the generation was grounded in persona data. Each dialogue moves through four ellipsis levels, and at each level more of the command's core components are removed.

The authors tested 10 LLMs, including GPT-4o, with zero-shot prompting and in-context learning. They also tried RAG over dialogue history, fine-tuning, and the tool-based assistants Sasha and SAGE. Every approach lost accuracy sharply on highly elliptical commands compared with complete ones. The benchmark therefore shows that memory and personalization in home assistants remain unsolved problems.

Key points

  • Dataset size: PEC-Home contains 1,780 dialogues built on 1,424 PersonaChat personas. Each dialogue has 4 turns that grow more elliptical, and the data is split into Multi-User Preferences and Dynamic User Preferences subsets.
  • Virtual home: It has 12 device types with over 50 executable methods. Spread across rooms, these give more than 350 room-specific methods.
  • Ellipsis levels: Lv1 through Lv4 are defined by progressively omitting the room, device, operation and operation parameters. This scheme is inspired by the Sasha command components.
  • Tool-based assistants (Qwen2.5-7B-Instruct, multi-user setting, execution accuracy):
    • SAGE fell from 51.40 at Lv1 to 4.49 at Lv4.
    • Sasha scored unevenly: 55.34 at Lv1, 23.25 at Lv2, and 43.26 at Lv4.
  • Fine-tuning and data quality:
    • Fine-tuning Qwen2.5-7B and Gemma2-9B on a 5:1:4 train/validation/test split helped on low-ellipsis commands but collapsed on high-ellipsis ones.
    • Experts manually checked 50 sample dialogues and judged 96.5% of the commands natural.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China74 works · 15,263 citations🎓Beihang UniversityBeijing · China65 works · 8,452 citations🎓Beijing Institute of TechnologyBeijing · China34 works · 3,652 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksEngineering · 133 worksElectrical and Electronic Engineering · 18 works

Cite

Cite as Yingyu Shan, Zeming Liu, Silin Li, Boao Qian, Jiashu Yao, Yuhang Guo and Haifeng Wang. 2026. PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.999

@inproceedings{shan2026pec,
  title = {{PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes}},
  author = {Yingyu Shan, Zeming Liu, Silin Li, Boao Qian, Jiashu Yao, Yuhang Guo and Haifeng Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.999},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.999}
}

References

45 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. AI@Meta. 2024. Llama 3 model card. find
  3. Susan E Brennan and Herbert H Clark. 1996. Conceptual pacts and lexical choice in conversation. Journal of experimental psychology: Learning, memory, and cognition, 22(6):1482. find
  4. John M Carroll. 1980. Naming and describing in social communication. Language and Speech, 23(4):309– 322. find
  5. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  6. Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, and Feng Zhao. 2024. T-eval: Evaluating the tool utilization capability of large language models step by step. In Proceedings of the 62nd Annual Meeting of the Association… find
  7. Herbert H Clark and Deanna Wilkes-Gibbs. 1986. Referring as a collaborative process. Cognition, 22(1):1– 39. find
  8. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, H… find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  10. Anind K Dey, Timothy Sohn, Sara Streng, and Justin Kodama. 2006. icap: Interactive prototyping of context-aware applications. In Pervasive Computing: 4th International Conference, PERVASIVE 2006, Dublin, Ireland, May 7-10, 2006. Proceedings 4, pages 254– 271. Springer. find
  11. Liangke Gui, Borui Wang, Qiuyuan Huang, Alexander Hauptmann, Yonatan Bisk, and Jianfeng Gao. 2022. KAT: A knowledge augmented transformer for vision-and-language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech… find
  12. Saurabh Gupta, Siddhant Bhambri, Karan Dhingra, Arun Balaji Buduru, and Ponnurangam Kumaraguru. 2020. Multi-objective Reinforcement Learning based approach for User-Centric Power Optimization in Smart Home Environments . In 2020 IEEE International Conference on Smart Data Services (SMDS), pages 89–9… find
  13. Robert D Hawkins, Michael C Frank, and Noah D Goodman. 2020. Characterizing the dynamics of learning in repeated reference games. Cognitive science, 44(6):e12845.
  14. Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, and Yanghua Xiao. 2024. Can large language models understand real-world complex instructions? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18188–18196. find
  15. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3.
  16. John Jaihar, Neehal Lingayat, Patel Sapan Vijaybhai, Gautam Venkatesh, and Kishor P Upla. 2020. Smart home automation using machine learning algorithms. In 2020 international conference for emerging technology (INCET), pages 1–4. IEEE. find
  17. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  18. Chadia Khraief, Faouzi Benzarti, and Hamid Amiri. 2019. Convolutional neural network based on dynamic motion and shape variations for elderly fall detection. International Journal of Machine Learning and Computing, 9(6):814–820. find
  19. Younggi Kim, Jihoon An, Minseok Lee, and Younghee Lee. 2017. An activity-embedding approach for next-activity prediction in a multi-user smart space. In 2017 IEEE International Conference on Smart Computing (SMARTCOMP), pages 1–6. IEEE. find
  20. Evan King, Haoxiang Yu, Sangsu Lee, and Christine Julien. 2024. Sasha: Creative goal-oriented reasoning in smart homes with large language models. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 8(1). find
  21. Robert M Krauss and Sidney Weinheimer. 1964. Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1:113–114. find
  22. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc…
  23. Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Jiang, Chengfei Lv, and Huajun Chen. 2024. AutoAct: Automatic agent learning from scratch for QA via self-planning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find
  24. Dmitriy Rivkin, Francois Hogan, Amal Feriani, Abhisek Konar, Adam Sigal, Xue Liu, and Gregory Dudek. 2024. Aiot smart home via autonomous llm agents. IEEE Internet of Things Journal. find
  25. Samad Sepasgozar, Reyhaneh Karimi, Leila Farahzadi, Farimah Moezzi, Sara Shirowzhan, Sanee M. Ebrahimzadeh, Felix Hui, and Lu Aye. 2020. A systematic content review of artificial intelligence and the internet of things applications in smart home. Applied Sciences, 10(9):3074. find
  26. Yingtian Shi, Xiaoyi Liu, Chun Yu, Tianao Yang, Cheng Gao, Chen Liang, and Yuanchun Shi. 2024. Bridging the gap between natural user expression with complex automation programming in smart homes. Preprint, arXiv:2408.12687. — [arXiv:2408.12687](https://arxiv.org/abs/2408.12687) find
  27. Shashi Suman, Ali Etemad, and Francois Rivest. 2022. Potential impacts of smart homes on human behavior: A reinforcement learning approach. IEEE Transactions on Artificial Intelligence, 3(4):567–580. find
  28. Niek Tax. 2018. Human activity prediction in smart home environments with lstm neural networks. In 2018 14th international conference on intelligent environments (IE), pages 40–47. IEEE. find
  29. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton T… find
  30. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  31. Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and Michael L Littman. 2014. Practical trigger-action programming in the smart home. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 803–812. find
  32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  33. Hongru Wang, Rui Wang, Boyang Xue, Heming Xia, Jingtao Cao, Zeming Liu, Jeff Z. Pan, and Kam-Fai Wong. 2024a. AppBench: Planning of multiple APIs from various APPs for complex user instruction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15322–153… find
  34. Huiming Wang, Liying Cheng, Wenxuan Zhang, De Wen Soh, and Lidong Bing. 2024b. Order-agnostic data augmentation for few-shot named entity recognition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7792–7807, Bangkok, Thailan… find
  35. Mark Weiser. 1999. The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev., 3(3):3–11. find
  36. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… find
  37. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  38. Ziqi Yin, Mingxin Zhang, and Daisuke Kawahara. 2024. Harmony: A home agent for responsive management and action optimization with a locally deployed large language model. Preprint, arXiv:2410.14252. — [arXiv:2410.14252](https://arxiv.org/abs/2410.14252) find
  39. Haoxiang Yu, Jie Hua, and Christine Julien. 2021. Analysis of ifttt recipes to study how humans use internet-of-things (iot) devices. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, SenSys ’21, page 537–541. ACM. find
  40. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018…
  41. Li Zeng, Yingyu Shan, Zeming Liu, Jiashu Yao, and Yuhang Guo. 2024. FAME: Towards factual multi-task model editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15992–16011, Miami, Florida, USA. Association for Computational Linguistics. find
  42. Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. 2024. CodeAgent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1…
  43. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbou…
  44. Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. 2023. Toolqa: A dataset for llm question answering with external tools. Advances in Neural Information Processing Systems, 36:50117– 50143.
  45. Rolf A Zwaan and Gabriel A Radvansky. 1998. Situation models in language comprehension and memory. Psychological bulletin, 123(2):162. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shan-2026-pec-home-interpretation-progressively-elliptical/