Elamur Research⌕
☰
home / papers / 2019 / Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout

Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout

Paper · 2019 · Hao Tan, Licheng Yu and Mohit Bansal

Summary

The paper addresses instruction-guided navigation in the Room-to-Room (R2R) task, which is built on real Matterport3D indoor scans. Agents for this task usually lose much of their accuracy in houses they never saw during training. The authors propose a two-stage training scheme.

Stage 1. An encoder-decoder agent is trained with a combination of imitation learning and reinforcement learning. The encoder is a Bi-LSTM over the instruction. The decoder is an attentive LSTM over ResNet panoramic features.

Stage 2. The agent is fine-tuned on synthetic (environment, path, instruction) triplets produced by back-translation with a speaker model.

The authors' analysis shows that the real bottleneck of back-translation is the small number of distinct training environments, not the amount of data. To address this, they introduce "environmental dropout." It masks visual feature channels consistently across views and viewpoints, so that each mask behaves like a new house. The same mask is shared by the speaker and the follower.

The resulting agent took first place on the private test set of the VLN R2R leaderboard in every evaluation setting. This shows that cheaply simulating environment diversity can substantially narrow the gap between seen and unseen environments.

Key points

  • Back-translation with environmental dropout raises the best supervised (IL+RL) model by 5.7%, roughly three times the gain of plain back-translation. Feature-level dropout performs 3.8% worse and barely improves on the baseline.
  • Giving the speaker and the follower separately sampled masks noticeably degrades results, so the shared mask is essential.
  • Data-scaling experiments over the 60 training environments suggest a ceiling of about 52% success rate, even if all 190K routes in those environments were human-annotated. Adding new environments helps more than adding data.
  • In the pre-exploration setting on validation-unseen environments, back-translation reaches 61.9% success rate, compared with 46.5% when using only training environments. Adding environmental dropout lifts this to 64.5%.
  • With Matterport3D semantic views as input, results rise from 49.5% (IL+RL) to 52.0% with environmental dropout. Code is released at github.com/airsplay/R2R-EnvDrop.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of North Carolina at Chapel HillChapel Hill · United States37 works · 14,332 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

Cite

Cite as Hao Tan, Licheng Yu and Mohit Bansal. 2019. Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1268

@inproceedings{tan2019learning,
  title = {{Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout}},
  author = {Hao Tan, Licheng Yu and Mohit Bansal},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1268},
  url = {https://doi.org/10.18653/v1/n19-1268}
}

References

41 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. 2018a. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757. — [arXiv:1807.06757](https://arxiv.org/abs/1807.06757) find
  2. Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018b. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vi… our card
  3. Jacob Andreas and Dan Klein. 2015. Alignment-based compositional semantics for instruction following. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1165–1174. find
  4. Yoav Artzi and Luke Zettlemoyer. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Transactions of the Association of Computational Linguistics, 1:49–62. our card
  5. Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-follower models for vision-and-language navigation. In Advances in Neural Information Processing Systems, pages 3318–3329. find
  6. Dhiraj Gandhi, Lerrel Pinto, and Abhinav Gupta. 2017. Learning to fly by crashing. IROS. find
  7. Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. 2018. Iqa: Visual question answering in interactive environments. In CVPR. find
  8. Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. 2017. Cognitive mapping and planning for visual navigation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2616–2625. find
  9. Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316. — [arXiv:1604.07316](https://arxiv.org/abs/1604.07316) find
  10. Kotaro Hayashi, Daisuke Sakamoto, Takayuki Kanda, Masahiro Shiomi, Satoshi Koizumi, Hiroshi Ishiguro, Tsukasa Ogasawara, and Norihiro Hagita. 2007. Humanoid robots as a passive-social medium-a field experiment at a train station. In Human-Robot Interaction (HRI), 2007 2nd ACM/IEEE International Conf… find
  11. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Mat-terport3D: Learning from RGB-D data in indoor environments. International Conference on 3D Vision (3DV). find
  12. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778. find
  13. Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov. 2018. Gated-attention architectures for task-oriented language grounding. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  14. David L Chen and Raymond J Mooney. 2011. Learning to interpret natural language navigation instructions from observations. In AAAI, volume 2, pages 1–2. find
  15. Andrew Correa, Matthew R Walter, Luke Fletcher, Jim Glass, Seth Teller, and Randall Davis. 2010. Multimodal interaction with an autonomous forklift. In Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, pages 243– 250. IEEE Press. find
  16. Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. 2012. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. find
  17. Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018. Iterative back-translation for neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18–24. our card
  18. Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end training of deep visuomotor policies. JMLR. find
  19. Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan Al-Regib, Zsolt Kira, Richard Socher, and Caiming Xiong. 2019. Self-monitoring navigation agent via auxiliary progress estimation. find
  20. Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. 2018. Embodied question answering. In CVPR). find
  21. Hongyuan Mei, Mohit Bansal, and Matthew R Walter. 2016. Listen, attend, and walk: Neural mapping of navigational instructions to action sequences. In AAAI, volume 1, page 2. find
  22. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500. find
  23. Dipendra Misra, John Langford, and Yoav Artzi. 2017. Mapping instructions and visual observations to actions with reinforcement learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1004–1015. find
  24. Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, pages 1928–1937. find
  25. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In International Conference on Learning Representations. our card
  26. David L Poole and Alan K Mackworth. 2010. Artificial Intelligence: foundations of computational agents. Cambridge University Press. find
  27. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. Style transfer through back-translation. In Proc. ACL. our card
  28. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. find
  29. Xin Wang, Wenhan Xiong, Hongmin Wang, and William Yang Wang. 2018a. Look before you leap: Bridging model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Pro… find
  30. Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018b. Switchout: an efficient data augmentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856–861. our card
  31. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. In ICLR. our card
  32. Yi Wu, Yuxin Wu, Georgia Gkioxari, and Yuandong Tian. 2018. Building generalizable agents with a realistic and rich 3d environment. arXiv preprint arXiv:1801.02209. — [arXiv:1801.02209](https://arxiv.org/abs/1801.02209) find
  33. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252. our card
  34. Yuke Zhu, Daniel Gordon, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, and Ali Farhadi. 2017a. Visual semantic planning using deep successor representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 483–492. find
  35. Fereshteh Sadeghi and Sergey Levine. 2017. CAD2RL: Real single-image flight without a single real image. RSS. find
  36. Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. 2017b. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In Robotics and Automation (ICRA), 2017 IEEE International Conference on, pages 3357–3364. IEEE. find
  37. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 86–96. find
  38. Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. 2017. Revisiting unreasonable effectiveness of data in deep learning era. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 843–852. IEEE. find
  39. Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth J Teller, and Nicholas Roy. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. In AAAI, volume 1, page 2. find
  40. Michael L Walters, Kerstin Dautenhahn, Sarah N Woods, and Kheng Lee Koay. 2007. Robotic etiquette: results from user studies involving a fetch and carry task. In Proceedings of the ACM/IEEE international conference on Human-robot interaction, pages 317–324. ACM. find
  41. Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang. 2019. Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2019-learning-navigate-unseen-environments-back/