Elamur Research⌕
☰
home / papers / 2026 / NavA³: Understanding Any Instruction, Navigating Anywhere, Finding Anything

NavA³: Understanding Any Instruction, Navigating Anywhere, Finding Anything

Paper · 2026 · Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang et al.

Summary

The paper tackles long-horizon embodied navigation driven by high-level human requests, such as asking for coffee, rather than step-by-step directions (VLN) or a fixed object category (ObjectNav). In this setting, a robot must work out which object would satisfy the request, guess where it is likely to be, and find the specific instance that matches any spatial constraints. NavA³ splits this into two levels. At the global level, a Reasoning-VLM reads the instruction alongside an annotated global 3D scene and picks the target object and the most promising room. At the local level, a Pointing-VLM called NaviAfford chooses exploration waypoints, scans panoramic views, and points to the target. The pointed location is then converted into robot coordinates. NaviAfford was trained on a newly collected set of 1.0M spatial-aware object affordance samples. The authors report state-of-the-art navigation results and successful runs on both wheeled and quadruped robots. The work matters because it moves robot navigation closer to the vague, open-vocabulary way people actually phrase requests.

Key points

  • Task definition: an episode succeeds when the agent ends within 1 meter of the correct target object and keeps it in line of sight.
  • Affordance accuracy: NaviAfford averages 70.8%, compared with 55.9% for the previous best model, RoboPoint.
  • Navigation success rate: 68.0% with NaviAfford as the Pointing-VLM, versus 57.5% (RoboPoint), 32.0% (GPT-4o) and 16.0% (Qwen2.5-VL-72B).
  • Ablation: swapping the Reasoning-VLM for weaker models, including LLaVA-NeXT-7B, cuts performance by 28.0 to 36.0 points, which suggests strong reasoning is essential.
  • Stated limitations: the system depends on reliable RGB-D depth, which struggles with reflective or transparent surfaces and poor lighting. Its modular two-stage design, rather than end-to-end action prediction, may also add latency.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓National University of SingaporeSingapore · Singapore180 works · 34,304 citations🔬Institute of AutomationBeijing · China70 works · 14,251 citations🔬Beijing Academy of Artificial IntelligenceBeijing · China32 works · 7,511 citations🎓Tsinghua Shenzhen International Graduate SchoolShenzhen · China28 works · 3,035 citations🎓Tongji UniversityShanghai · China19 works · 2,162 citations🏢XiaomiBeijing · China9 works · 334 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding and Shanghang Zhang. 2026. NavA³: Understanding Any Instruction, Navigating Anywhere, Finding Anything. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.37

@inproceedings{zhang2026understanding,
  title = {{NavA³: Understanding Any Instruction, Navigating Anywhere, Finding Anything}},
  author = {Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding and Shanghang Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.37},
  url = {https://doi.org/10.18653/v1/2026.acl-long.37}
}

References

42 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, and Si Liu. 2025. Octonav: Towards generalist embodied navigation. arXiv preprint arXiv:2506.09839. — [arXiv:2506.09839](https://arxiv.org/abs/2506.09839) find
  2. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. https://docs.anthropic.com/zh-CN/release-notes/claude-apps. ↗
  3. Zeying Gong, Rong Li, Tianshuai Hu, Ronghe Qiu, Lingdong Kong, Lingfeng Zhang, Yiyi Ding, Leying Zhang, and Junwei Liang. 2025. Stairway to success: Zero-shot floor-aware object-goal navigation via llm-driven coarse-to-fine exploration. arXiv preprint arXiv:2505.23019. — [arXiv:2505.23019](https://arxiv.org/abs/2505.23019) find
  4. Agrim Gupta, Piotr Dollar, and Ross Girshick. 2019. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364. find
  5. Xiaoshuai Hao, Huaihai Lyu, Lingfeng Zhang, Rui Liu, Dayan Wu, Jing Zhang, and Long Chen. 2026. H2rbm: Can leveraging human videos enhance performance and generalizability in robotic bimanual manipulation? Pattern Recognition, page 113637. find
  6. Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Yanbiao Ma, Yunfeng Diao, Ziyu Jia, Wenbo Ding, Hangjun Ye, and Long Chen. 2025a. Roboafford++: A generative ai-enhanced dataset for multimodal affordance learning in robotic manipulation and navigation. arXiv preprint arXiv:2511.12436. — [arXiv:2511.12436](https://arxiv.org/abs/2511.12436) find
  7. Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, and 1 others. 2025b. Mimo-embodied: X-embodied foundation model technical report. arXiv preprint arXiv:2511.16518. — [arXiv:2511.16518](https://arxiv.org/abs/2511.16518) find
  8. Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, and Stephen Gould. 2021. Vln bert: A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 1643–1653. disentangled reasoning. IEEE Transactions on Patter… find
  9. Peiran Liu, Qiang Zhang, Daojie Peng, Lingfeng Zhang, Yihao Qin, Hang Zhou, Jun Ma, Renjing Xu, and Yiding Ji. 2025a. Toponav: Topological graphs as a key enabler for advanced object navigation. arXiv preprint arXiv:2509.01364. — [arXiv:2509.01364](https://arxiv.org/abs/2509.01364) find
  10. Sichao Liu, Jianjing Zhang, Robert X Gao, Xi Vincent Wang, and Lihui Wang. 2024. Vision-language model-driven scene understanding and robotic object manipulation. In IEEE 20th International Conference on Automation Science and Engineering, pages 21–26. IEEE. find
  11. Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, and 1 others. 2025b. Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning. arXiv preprint arXiv:25… — [arXiv:2501.10074](https://arxiv.org/abs/2501.10074) find
  12. Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. 2025. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. In Conference on Robot Learning, pages 2049–2060. find
  13. Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. 2024. Discuss before moving: Visual language navigation via multi-expert discussions. In IEEE International Conference on Robotics and Automation, pages 17380–17387. IEEE. find
  14. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  15. Steven D Morad, Roberto Mecca, Rudra PK Poudel, Stephan Liwicki, and Roberto Cipolla. 2021. Embodied visual navigation with automatic curriculum learning in real environments. IEEE Robotics and Automation Letters, 6(2):683–690. find
  16. Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, and 1 others. 2025. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. In Proceedings of the Computer Vision and Pattern Recognition Conference… find
  17. Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, and 1 others. 2024. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In IEEE International Conferenc… find
  18. Lingdong Kong, Shaoyuan Xie, Zeying Gong, Ye Li, Meng Chu, Ao Liang, Yuhao Dong, Tianshuai Hu, Ronghe Qiu, Rong Li, and 1 others. 2026. The robosense challenge: Sense anything, navigate anywhere, adapt across platforms. arXiv preprint arXiv:2601.05014. — [arXiv:2601.05014](https://arxiv.org/abs/2601.05014) find
  19. Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. 2024. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895. find
  20. Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, and Xiaodan Liang. 2025. Navcot: Boosting llm-based vision-and-language navigation via learning Zhangyang Qi, Zhixiong Zhang, Yizhou Yu, Jiaqi Wang, and Hengshuang Zhao. 2025. Vln-r1: Vision-language… — [arXiv:2506.17221](https://arxiv.org/abs/2506.17221) find
  21. Ram Ramrakhya, Dhruv Batra, Erik Wijmans, and Abhishek Das. 2023. Pirlnav: Pretraining with imitation and rl finetuning for objectnav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17896–17906. find
  22. Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. 2025. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752. — [arXiv:2503.20752](https://arxiv.org/abs/2503.20752) find
  23. Yingbo Tang, Lingfeng Zhang, Shuyi Zhang, Yinuo Zhao, and Xiaoshuai Hao. 2025. Roboafford: A dataset and benchmark for enhancing object and spatial affordance learning in robot manipulation. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 12706–12713. find
  24. Justin Wasserman, Girish Chowdhary, Abhinav Gupta, and Unnat Jain. 2024. Exploitation-guided exploration for semantic embodied navigation. In IEEE International Conference on Robotics and Automation, pages 2901–2908. IEEE. find
  25. Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. 2025a. Reinforcing spatial reasoning in vision-language models with interwoven thinking and visual drawing. arXiv preprint arXiv:2506.09965. — [arXiv:2506.09965](https://arxiv.org/abs/2506.09965) find
  26. Yujie Wu, Huaihai Lyu, Yingbo Tang, Lingfeng Zhang, Zhihui Zhang, Wei Zhou, and Siqi Hao. 2025b. Evaluating gpt-4o’s embodied intelligence: A comprehensive empirical study. Authorea Preprints. find
  27. Bangguo Yu, Hamidreza Kasaei, and Ming Cao. 2023. L3mvn: Leveraging large language models for visual target navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3554–3560. IEEE. find
  28. Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. 2025. Robopoint: A vision-language model for spatial affordance prediction in robotics. In Conference on Robot Learning, pages 4005–4020. find
  29. Simon Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Peter Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, and 1 others. 2024. Fine-tuning large vision-language models as decision-making agents via reinforcement learning. Advances in neural information processing systems, 37:110935–110971. find
  30. Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang, Zhizheng Zhang, and He Wang. 2024a. Uni-navid: A video-based vision-language-action model for unifying embodied navigation tasks. arXiv preprint arXiv:2412.06224. — [arXiv:2412.06224](https://arxiv.org/abs/2412.06224) find
  31. Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. 2024b. Navid: Video-based vlm plans the next step for vision-and-language navigation. In Robotics: Science and Systems. find
  32. Lingfeng Zhang, Haoxiang Fu, Xiaoshuai Hao, Shuyi Zhang, Qiang Zhang, Rui Liu, Long Chen, and Wenbo Ding. 2026a. What you see is what you reach: Towards spatial navigation with high-level human instructions. find
  33. Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang, Shanghang Zhang, and Renjing Xu. 2025a. Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and-language navigation. In The 63rd Annual Meeting of the Ass… find
  34. Lingfeng Zhang, Erjia Xiao, Xiaoshuai Hao, Haoxiang Fu, Zeying Gong, Long Chen, Xiaojun Liang, Renjing Xu, Hangjun Ye, and Wenbo Ding. 2025b. Socialnav-map: Dynamic mapping with human trajectory prediction for zero-shot social navigation. arXiv preprint arXiv:2511.12232. find
  35. Lingfeng Zhang, Erjia Xiao, Yuchen Zhang, Haoxiang Fu, Ruibin Hu, Yanbiao Ma, Wenbo Ding, Long Chen, Hangjun Ye, and Xiaoshuai Hao. 2025c. Team xiaomi ev-ad vla: Caption-guided retrieval system for cross-modal drone navigation–technical report for iros 2025 robosense challenge track 4. arXiv preprin… — [arXiv:2510.02728](https://arxiv.org/abs/2510.02728) find
  36. Lingfeng Zhang, Qiang Zhang, Hao Wang, Erjia Xiao, Zixuan Jiang, Honglei Chen, and Renjing Xu. 2024c. Trihelper: Zero-shot object navigation with dynamic assistance. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 10035–10042. IEEE. find
  37. Lingfeng Zhang, Yuchen Zhang, Hongsheng Li, Haoxiang Fu, Yingbo Tang, Hangjun Ye, Long Chen, Xiaojun Liang, Xiaoshuai Hao, and Wenbo Ding. 2025d. Is your vlm sky-ready? a comprehensive spatial intelligence benchmark for uav navigation. arXiv preprint arXiv:2511.13269. — [arXiv:2511.13269](https://arxiv.org/abs/2511.13269) find
  38. Qiang Zhang, Jiahao Ma, Peiran Liu, Shuai Shi, Zeran Su, Zifan Wang, Jingkai Sun, Wei Cui, Jialin Yu, Gang Han, and 1 others. 2026b. Mesh-mimic: Geometry-aware humanoid motion learning through 3d scene reconstruction. arXiv preprint arXiv:2602.15733. — [arXiv:2602.15733](https://arxiv.org/abs/2602.15733) find
  39. Shuyi Zhang, Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Pengwei Wang, Zhongyuan Wang, Hongxuan Ma, and Shanghang Zhang. 2025e. Video-cot: A comprehensive dataset for spatiotemporal understanding of videos based on chain-of-thought. arXiv preprint arXiv:2506.08817. — [arXiv:2506.08817](https://arxiv.org/abs/2506.08817) find
  40. Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. 2024a. Towards learning a generalist model for embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13624–13634. find
  41. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024b. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372.
  42. Gengze Zhou, Yicong Hong, and Qi Wu. 2024. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-nava-3-understanding-any-instruction/