Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2507.06261
Summary
This technical report from Google's Gemini Team describes the Gemini 2.X generation: Gemini 2.5 Pro and 2.5 Flash, plus the earlier, cheaper Gemini 2.0 Flash and Flash-Lite. The goal is a set of models that covers every trade-off point between capability and cost, with thinking-based reasoning, native multimodal input (text, image, audio, video), very long context and built-in tool use, so the models can drive agentic systems.
All models are sparse mixture-of-experts (MoE) transformers. Pre-training data comes from web documents, code, images, audio and video, with better filtering and deduplication. The smaller models are distilled from a larger teacher. Post-training uses SFT, reward modeling and more compute for RL, with verifiable and model-based rewards. Training ran on TPUv5p with new fault-tolerance mechanisms.
Gemini 2.5 Pro reaches state-of-the-art results on coding and reasoning benchmarks and can handle about 3 hours of video. Safety testing found it did not cross any Critical Capability Level. The authors also argue that progress now outpaces the creation of hard new benchmarks.
Key points
- The Gemini 2.5 models accept inputs of about 1M tokens, produce up to 64K output tokens, use dynamic thinking and have a January 2025 knowledge cutoff. Gemini 2.5 Flash lets users set a controllable thinking budget.
- Distillation for Flash-size models approximates the teacher's next-token distribution with a k-sparse distribution, which reduces storage cost.
- Training ran on 8960-chip TPUv5p pods across several datacenters, coordinated by the Pathways system:
- 93.4% of run time went to TPU computation.
- Slice-level elasticity kept throughput at roughly 97% during hardware failures.
- Replay-based SDC detection re-ran about 0.25% of steps, and 6% of those turned out to be real hardware corruption.
- Gemini 2.5 Pro scores more than 120 LMArena points above Gemini 1.5 Pro. Within one year, Gemini Pro's Aider Polyglot score rose roughly 5x, and its SWE-bench verified score doubled.
- Evaluations of CBRN, cybersecurity, ML R&D and deceptive alignment risks, including external red teaming, showed capability gains but no Critical Capability Level reached.
Cited by
- Qwen3-VL Technical Report
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Accelerating scientific discovery with Co-Scientist
- Arabic Sentence Segmentation Across Genres and Punctuation Conditions
- Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- Qwen3-Omni Technical Report
Cite
@misc{comanici2025gemini,
title = {{Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities}},
author = {Comanici, Gheorghe and Bieber, Eric and Schaekermann, Mike and Pasupat, Ice and Sachdeva, Noveen and Dhillon, Inderjit and Blistein, Marcel and Ram, Ori and Zhang, Dan and Rosen, Evan and Marris, Luke and Petulla, Sam and Gaffney, Colin and Aharoni, Asaf and Lintz, Nathan and Pais, Tiago Cardal and Jacobsson, Henrik and Szpektor, Idan and Jiang, Nan-Jiang and Haridasan, Krishna and Omran, Ahmed and Saunshi, Nikunj and Bahri, Dara and Mishra, Gaurav and Chu, Eric and Boyd, Toby and Hekman, Brad and Parisi, Aaron and Zhang, Chaoyi and Kawintiranon, Kornraphop and Bedrax-Weiss, Tania and Wang, Oliver and Xu, Ya and Purkiss, Ollie and Mendlovic, Uri and Deutel, Ilaï and Nguyen, Nam and Langley, Adam and Korn, Flip and Rossazza, Lucia and Ramé, Alexandre and Waghmare, Sagar and Miller, Helen and Byrd, Nathan and Sheshan, Ashrith and Hadsell, Raia and Bhardwaj, Sangnie and Janus, Pawel and Rissa, Tero and Horgan, Dan and Abdagic, Alvin and Belenki, Lior and Allingham, James and Singh, Anima and Guidroz, Theo and Srinivasan, Srivatsan and Schmit, Herman and Chiafullo, Kristen and Elisseeff, Andre and Jha, Nilpa and Kolhar, Prateek and Berrada, Leonard and Ding, Frank and Si, Xiance and Mallick, Shrestha Basu and Och, Franz and Erell, Sofia and Ni, Eric and Latkar, Tejasi and Yang, Sherry and Sirkovic, Petar and Feng, Ziqiang and Leland, Robert and Hornung, Rachel and Wu, Gang and Blundell, Charles and Alvari, Hamidreza and Huang, Po-Sen and Yip, Cathy and Deur, Sanja and Liu, Li and Surita, Gabriela and Duque, Pablo and Damen, Dima and Jia, Johnson and Guez, Arthur and Mircea, Markus and Sinha, Animesh and Magni, Alberto and Stradomski, Paweł and Marian, Tal and Galić, Vlado and Chen, Wenhu and Husain, Hisham and Singhal, Achintya and Grewe, Dominik and Aubet, François-Xavier and Song, Shuang and Blanco, Lorenzo and Rechis, Leland and Ho, Lewis and Munoz, Rich and Zheng, Kelvin and Hamrick, Jessica and Mather, Kevin and Taitelbaum, Hagai and Rutherford, Eliza and Lei, Yun and Chen, Kuangyuan and Shukla, Anand and Moreira, Erica and Doi, Eric and Isik, Berivan and Shabat, Nir and Rogozińska, Dominika and Kolipaka, Kashyap and Chang, Jason and Vušak, Eugen and Venkatachary, Srinivasan and Noghabi, Shadi and Bharti, Tarun and Jun, Younghoon and Zaks, Aleksandr and Green, Simon and Challagundla, Jeshwanth and Wong, William and Mohammad, Muqthar and Hirsch, Dean and Cheng, Yong and Naim, Iftekhar and Proleev, Lev and Vincent, Damien and Singh, Aayush and Krikun, Maxim and Krishnan, Dilip and Ghahramani, Zoubin and Atias, Aviel and Aggarwal, Rajeev and Kirov, Christo and Vytiniotis, Dimitrios and Koh, Christy and Chronopoulou, Alexandra and Dogra, Pawan and Ion, Vlad-Doru and Tyen, Gladys and Lee, Jason and Weissenberger, Felix and Strohman, Trevor and Balakrishna, Ashwin and Rae, Jack and Velic, Marko and Liedekerke, Raoul de and Elyada, Oded and Yuan, Wentao and Liu, Canoee and Shani, Lior and Kishchenko, Sergey and Alessio, Bea and Li, Yandong and Song, Richard and Kwei, Sam and Jankowski, Orion and Pappu, Aneesh and Namiki, Youhei and Ma, Yenai and Tripuraneni, Nilesh and Cherry, Colin and Ikonomidis, Marissa and Ling, Yu-Cheng and Ji, Colin and Westberg, Beka and Wright, Auriel and Yu, Da and Parkinson, David and Ramaswamy, Swaroop and Connor, Jerome and Yeganeh, Soheil Hassas and Grover, Snchit and Kenwright, George and Litchev, Lubo and Apps, Chris and Tomala, Alex and Halim, Felix and Castro-Ros, Alex and Li, Zefei and Boral, Anudhyan and Sho, Pauline and Yarom, Michal and Malmi, Eric and Klinghoffer, David and Lin, Rebecca and Ansell, Alan and S, Pradeep Kumar and Zhao, Shubin and Zuo, Siqi and Santoro, Adam and Cheng, Heng-Tze and Demmessie, Solomon and Liu, Yuchi and Brichtova, Nicole and Culp, Allie and Braun, Nathaniel and Graur, Dan and Ng, Will and Mehta, Nikhil and Phillips, Aaron and Sundberg, Patrik and Godbole, Varun and Liu, Fangyu and Katariya, Yash and Rim, David and Seyedhosseini, Mojtaba and Ammirati, Sean and Valfridsson, Jonas and Malihi, Mahan and Knight, Timothy and Toor, Andeep and Lampe, Thomas and Ittycheriah, Abe and Chiang, Lewis and Yeung, Chak and Fréchette, Alexandre and Rao, Jinmeng and Wang, Huisheng and Srivastava, Himanshu and Zhang, Richard and Rhodes, Rocky and Brand, Ariel and Weesner, Dean and Figotin, Ilya and Gimeno, Felix and Fellinger, Rachana and Marcenac, Pierre and Leal, José and Marcus, Eyal and Cotruta, Victor and Cabrera, Rodrigo and Luo, Sheryl and Garrette, Dan and Axelrod, Vera and Baltateanu, Sorin and Barker, David and Chen, Dongkai and Toma, Horia and Ingram, Ben and Riesa, Jason and Kulkarni, Chinmay and Zhang, Yujing and Liu, Hongbin and Wang, Chao and Polacek, Martin and Wu, Will and Hui, Kai and Reyes, Adrian N and Su, Yi and Barnes, Megan and Malhi, Ishaan and Siddiqui, Anfal and Feng, Qixuan and Damaschin, Mihai and Pighin, Daniele and Steiner, Andreas and Yang, Samuel and Boppana, Ramya Sree and Ivanov, Simeon and Kandoor, Arun and Shah, Aditya and Mujika, Asier and Huang, Da and Choquette-Choo, Christopher A. and Patel, Mohak and Yu, Tianhe and Creswell, Toni and Jerry and Liu and Barros, Catarina and Razeghi, Yasaman and Roy, Aurko and Culliton, Phil and Xiong, Binbin and Pan, Jiaqi and Strohmann, Thomas and Powell, Tolly and Seal, Babi and DeCarlo, Doug and Shyam, Pranav and Katircioglu, Kaan and Wang, Xuezhi and Hardin, Cassidy and Odisho, Immanuel and Broder, Josef and Chang, Oscar and Nair, Arun and Shtefan, Artem and O'Brien, Maura and Agarwal, Manu and Potluri, Sahitya and Goyal, Siddharth and Jhindal, Amit and Thakur, Saksham and Stuken, Yury and Lyon, James and Toutanova, Kristina and Feng, Fangxiaoyu and Wu, Austin and Horn, Ben and Wang, Alek and Cullum, Alex and Taubman, Gabe and Shrivastava, Disha and Shi, Chongyang and Tomlinson, Hamish and Patel, Roma and Tu, Tao and Oflazer, Ada Maksutaj and Pongetti, Francesco and Yang, Mingyao and Taïga, Adrien Ali and Perot, Vincent and Pierse, Nuo Wang and Han, Feng and Drori, Yoel and Iturrate, Iñaki and Chakrabarti, Ayan and Yeung, Legg and Dopson, Dave and Chen, Yi-ting and Kulshreshtha, Apoorv and Guo, Tongfei and Pham, Philip and Schuster, Tal and Chen, Junquan and Polozov, Alex and Xing, Jinwei and Zhou, Huanjie and Kacham, Praneeth and Kukliansky, Doron and Miech, Antoine and Yaroshenko, Sergey and Chi, Ed and Douglas, Sholto and Fei, Hongliang and Blondel, Mathieu and Myla, Preethi and Madmoni, Lior and Wu, Xing and Keysers, Daniel and Kjems, Kristian and Albuquerque, Isabela and Yu, Lijun and D'sa, Joel and Plantan, Michelle and Ionescu, Vlad and Elias, Jaume Sanchez and Gupta, Abhirut and Vuyyuru, Manish Reddy and Alcober, Fred and Zhou, Tong and Ji, Kaiyang and Hartmann, Florian and Puttagunta, Subha and Song, Hugo and Amid, Ehsan and Stefanoiu, Anca and Lee, Andrew and Pucciarelli, Paul and Wang, Emma and Raul, Amit and Petrov, Slav and Tian, Isaac and Anklin, Valentin and Nti, Nana and Gomes, Victor and Schumacher, Max and Vesom, Grace and Panagopoulos, Alex and Bousmalis, Konstantinos and Andor, Daniel and Jacob, Josh and Zhang, Yuan and Rosgen, Bill and Kecman, Matija and Tung, Matthew and Belias, Alexandra and Goodman, Noah and Covington, Paul and Wieder, Brian and Saxena, Nikita and Davoodi, Elnaz and Huang, Muhuan and Maddineni, Sharath and Roulet, Vincent and Campbell-Ajala, Folawiyo and Sessa, Pier Giuseppe and Xintian and Wu and Lai, Guangda and Collins, Paul and Haig, Alex and Sakenas, Vytenis and Xu, Xiaowei and Giustina, Marissa and Shafey, Laurent El and Charoenpanit, Pichi and Garg, Shefali and Ainslie, Joshua and Severson, Boone and Arenas, Montse Gonzalez and Pathak, Shreya and Rajayogam, Sujee and Feng, Jie and Bakker, Michiel and Li, Sheng and Wichers, Nevan and Rogers, Jamie and Geng, Xinyang and Li, Yeqing and Jagerman, Rolf and Jia, Chao and Olmert, Nadav and Sharon, David and Mauger, Matthew and Mariserla, Sandeep and Ma, Hongxu and Mohabey, Megha and Kim, Kyuyeun and Andreev, Alek and Pollom, Scott and Love, Juliette and Jain, Vihan and Agrawal, Priyanka and Schroecker, Yannick and Fortin, Alisa and Warmuth, Manfred and Liu, Ji and Leach, Andrew and Blok, Irina and Girirajan, Ganesh Poomal and Aharoni, Roee and Uria, Benigno and Sozanschi, Andrei and Goldberg, Dan and Ionita, Lucian and Ribeiro, Marco Tulio and Zlocha, Martin and Birodkar, Vighnesh and Lachgar, Sami and Yuan, Liangzhe and Choudhury, Himadri and Ginsberg, Matt and Zheng, Fei and Dibb, Gregory and Graves, Emily and Lokhande, Swachhand and Rasskin, Gabriel and Muraru, George-Cristian and Quick, Corbin and Tata, Sandeep and Sermanet, Pierre and Chawla, Aditya and Karo, Itay and Wang, Yan and Zhang, Susan and Keller, Orgad and Dragan, Anca and Su, Guolong and Chou, Ian and Liu, Xi and Tao, Yiqing and Prabhakara, Shruthi and Wilson, Marc and Liu, Ruibo and Wang, Shibo and Evans, Georgie and Du, David and Castaño, Alfonso and Prasad, Gautam and Mahdy, Mona El and Gerlach, Sebastian and Reid, Machel and Kahn, Jarrod and Zait, Amir and Pillai, Thanumalayan Sankaranarayana and Ulrich, Thatcher and Wang, Guanyu and Wassenberg, Jan and Farkash, Efrat and Yalasangi, Kiran and Wang, Congchao and Bauza, Maria and Bucher, Simon and Liu, Ting and Yan, Jun and Leung, Gary and Sindhwani, Vikas and Barnes, Parker and Singh, Avi and Jurin, Ivan and Chang, Jichuan and Bhumihar, Niket Kumar and Eiger, Sivan and Citovsky, Gui and Withbroe, Ben and Li, Zhang and Xue, Siyang and Santo, Niccolò Dal and Stoyanov, Georgi and Raimond, Yves and Zheng, Steven and Gao, Yilin and Listík, Vít and Kwasiborski, Sławek and Saputro, Rachel and Ozturel, Adnan and Mallya, Ganesh and Majmundar, Kushal and West, Ross and Caron, Paul and Wei, Jinliang and Castrejon, Lluis and Vikram, Sharad and Ramachandran, Deepak and Dhawan, Nikhil and Park, Jiho and Smoot, Sara and Driessche, George van den and Blau, Yochai and Malik, Chase and Liang, Wei and Hirsch, Roy and Santos, Cicero Nogueira dos and Weinstein, Eugene and Oord, Aäron van den and Lall, Sid and FitzGerald, Nicholas and Jiang, Zixuan and Yang, Xuan and Webster, Dale and Elqursh, Ali and Pope, Aedan and Rotival, Georges and Raposo, David and Zhu, Wanzheng and Dean, Jeff and Alabed, Sami and Tran, Dustin and Gupta, Arushi and Gleicher, Zach and Austin, Jessica and Rosseel, Edouard and Umekar, Megh and Das, Dipanjan and Sun, Yinghao and Chen, Kai and Misiunas, Karolis and Zhou, Xiang and Di, Yixian and Loo, Alyssa and Newlan, Josh and Li, Bo and Ramasesh, Vinay and Xu, Ying and Chen, Alex and Gandhe, Sudeep and Soricut, Radu and Gupta, Nikita and Hu, Shuguang and El-Sayed, Seliem and Garcia, Xavier and Brusilovsky, Idan and Chen, Pu-Chin and Bolt, Andrew and Huang, Lu and Gurney, Alex and Zhang, Zhiying and Pritzel, Alexander and Wilkiewicz, Jarek and Seybold, Bryan and Shamanna, Bhargav Kanagal and Fischer, Felix and Dean, Josef and Gill, Karan and Mcilroy, Ross and Bhowmick, Abhishek and Selier, Jeremy and Yang, Antoine and Cheng, Derek and Magay, Vladimir and Tan, Jie and Varma, Dhriti and Walder, Christian and Kocisky, Tomas and Nakashima, Ryo and Natsev, Paul and Kwong, Mike and Gog, Ionel and Zhang, Chiyuan and Dieleman, Sander and Jimma, Thomas and Ryabtsev, Andrey and Brahma, Siddhartha and Steiner, David and Du, Dayou and Žužul, Ante and Žanić, Mislav and Raghavachari, Mukund and Gierke, Willi and Zheng, Zeyu and Petrova, Dessie and Dauphin, Yann and Liu, Yuchuan and Kessler, Ido and Hand, Steven and Duvarney, Chris and Kim, Seokhwan and Lee, Hyo and Hussenot, Léonard and Hui, Jeffrey and Smith, Josh and Jain, Deepali and Xia, Jiawei and Tomar, Gaurav Singh and Amiri, Keyvan and Phan, Du and Fuchs, Fabian and Weyand, Tobias and Tomasev, Nenad and Cordell, Alexandra and Liu, Xin and Mallinson, Jonathan and Joshi, Pankaj and Crawford, Andy and Suggala, Arun and Chien, Steve and Fernando, Nick and Sanchez-Vargas, Mariella and Williams, Duncan and Crone, Phil and Luo, Xiyang and Karpov, Igor and Shan, Jyn and Thurk, Terry and Strudel, Robin and Voigtlaender, Paul and Patil, Piyush and Dozat, Tim and Khodaei, Ali and Singla, Sahil and Ambroszczyk, Piotr and Wu, Qiyin and Chang, Yifan and Roark, Brian and Hegde, Chaitra and Ding, Tianli and Filos, Angelos and Wu, Zhongru and Pinto, André Susano and Liu, Shuang and Khanna, Saarthak and Pandey, Aditya and Mcloughlin, Siobhan and Li, Qiujia and Haves, Sam and Zhou, Allan and Buchatskaya, Elena and Leal, Isabel and Boursac, Peter de and Akazawa, Nami and Anderson, Nina and Chen, Terry and Somandepalli, Krishna and Liang, Chen and Goenka, Sheela and Winkler, Stephanie and Grushetsky, Alexander and Ding, Yifan and Smith, Jamie and Ye, Fan and Pont-Tuset, Jordi and Li, Eric and Li, Ruichao and Golany, Tomer and Wegner, Dawid and Jiang, Tao and Barak, Omer and Shangguan, Yuan and Vértes, Eszter and Wong, Renee and Bornschein, Jörg and Tudor, Alex and Bevilacqua, Michele and Schaul, Tom and Rawat, Ankit Singh and Zhao, Yang and Axiotis, Kyriakos and Meng, Lei and McLean, Cory and Lai, Jonathan and Beattie, Jennifer and Kushman, Nate and Liu, Yaxin and Kutzman, Blair and Lang, Fiona and Ye, Jingchen and Netrapalli, Praneeth and Mishra, Pushkar and Khan, Myriam and Goel, Megha and Willoughby, Rob and Tian, David and Zhuang, Honglei and Chen, JD and Tsai, Zak and Kementsietsidis, Tasos and Khare, Arjun and Keeling, James and Xu, Keyang and Waters, Nathan and Altché, Florent and Popat, Ashok and Mittal, Bhavishya and Saxton, David and Badawy, Dalia El and Mathieu, Michael and Zheng, Zheng and Zhou, Hao and Ranka, Nishant and Shin, Richard and Duan, Qingnan and Salimans, Tim and Mihailescu, Ioana and Shaham, Uri and Chang, Ming-Wei and Assael, Yannis and Dikkala, Nishanth and Izzard, Martin and Cohen-Addad, Vincent and Graves, Cat and Feinberg, Vlad and Chung, Grace and Strouse, DJ and Karmon, Danny and Sharifzadeh, Sahand and Ashwood, Zoe and Pham, Khiem and Blanton, Jon and Vasiloff, Alex and Barber, Jarred and Geller, Mark and Zhou, Aurick and Zubach, Fedir and Huang, Tzu-Kuo and Zhang, Lei and Gupta, Himanshu and Young, Matt and Proskurnia, Julia and Votel, Ronny and Gabeur, Valentin and Barcik, Gabriel and Tripathi, Aditya and Yu, Hongkun and Yan, Geng and Changpinyo, Beer and Pavetić, Filip and Coyle, Amy and Fujii, Yasuhisa and Mendez, Jorge Gonzalez and Zhou, Tianhao and Rajamani, Harish and Hechtman, Blake and Cao, Eddie and Juan, Da-Cheng and Tan, Yi-Xuan and Dalibard, Valentin and Du, Yilun and Clay, Natalie and Yao, Kaisheng and Jia, Wenhao and Vijaykumar, Dimple and Zhou, Yuxiang and Bai, Xinyi and Hung, Wei-Chih and Pecht, Steven and Todorov, Georgi and Khadke, Nikhil and Gupta, Pramod and Lahoti, Preethi and Autef, Arnaud and Duddu, Karthik and Lee-Thorp, James and Bykovsky, Alexander and Misiunas, Tautvydas and Flennerhag, Sebastian and Thangaraj, Santhosh and McGiffin, Jed and Nado, Zack and Kunesch, Markus and Noever, Andreas and Hertz, Amir and Liang, Marco and Stone, Victor and Palmer, Evan and Daruki, Samira and Pramanik, Arijit and Põder, Siim and Kyker, Austin and Khan, Mina and Sluzhaev, Evgeny and Ritter, Marvin and Ruderman, Avraham and Zhou, Wenlei and Nagpal, Chirag and Vodrahalli, Kiran and Necula, George and Barham, Paul and Pavlick, Ellie and Hartford, Jay and Shafran, Izhak and Zhao, Long and Mikuła, Maciej and Eccles, Tom and Shimokawa, Hidetoshi and Garg, Kanav and Vilnis, Luke and Chen, Hanwen and Shumailov, Ilia and Lee, Kuang-Huei and Abdelhamed, Abdelrahman and Xie, Meiyan and Cohen, Vered and Hlavnova, Ester and Malkin, Dan and Sitawarin, Chawin and Lottes, James and Coquinot, Pauline and Yu, Tianli and Kumar, Sandeep and Zhang, Jingwei and Mahendru, Aroma and Ahmed, Zafarali and Martens, James and Chen, Tao and Boag, Aviel and Peng, Daiyi and Devin, Coline and Klimovskiy, Arseniy and Phuong, Mary and Vainstein, Danny and Xie, Jin and Ramabhadran, Bhuvana and Howard, Nathan and Yu, Xinxin and Goswami, Gitartha and Cui, Jingyu and Shleifer, Sam and Pinto, Mario and Yeh, Chih-Kuan and Yang, Ming-Hsuan and Javanmardi, Sara and Ethier, Dan and Lee, Chace and Orbay, Jordi and Kotecha, Suyog and Bromberg, Carla and Shaw, Pete and Thornton, James and Rosenthal, Adi Gerzi and Gu, Shane and Thomas, Matt and Gemp, Ian and Ayyar, Aditya and Ushio, Asahi and Selvan, Aarush and Wee, Joel and Liu, Chenxi and Majzoubi, Maryam and Yu, Weiren and Abernethy, Jake and Liechty, Tyler and Pan, Renke and Nguyen, Hoang and Qiong and Hu and Perrin, Sarah and Arora, Abhinav and Pitler, Emily and Wang, Weiyi and Shivakumar, Kaushik and Prost, Flavien and Limonchik, Ben and Wang, Jing and Gao, Yi and Cour, Timothee and Buch, Shyamal and Gui, Huan and Ivanova, Maria and Neubeck, Philipp and Chan, Kelvin and Kim, Lucy and Chen, Huizhong and Goyal, Naman and Chung, Da-Woon and Liu, Lu and Su, Yao and Petrushkina, Anastasia and Shen, Jiajun and Joulin, Armand and Xu, Yuanzhong and Lin, Stein Xudong and Kulizhskaya, Yana and Chelba, Ciprian and Vasudevan, Shobha and Collins, Eli and Bashlovkina, Vasilisa and Lu, Tony and Fritz, Doug and Park, Jongbin and Zhou, Yanqi and Su, Chen and Tanburn, Richard and Sushkov, Mikhail and Rasquinha, Mitchelle and Li, Jinning and Prendki, Jennifer and Li, Yiming and LV, Pallavi and Sharma, Shriya and Fitoussi, Hen and Huang, Hui and Dai, Andrew and Dao, Phuong and Burrows, Mike and Prior, Henry and Qin, Danfeng and Pundak, Golan and Sjoesund, Lars Lowe and Khurshudov, Art and Zhu, Zhenkai and Webson, Albert and Kemp, Elizabeth and Tan, Tat and Agrawal, Saurabh and Sargsyan, Susie and Cheng, Liqun and Stephan, Jim and Kwiatkowski, Tom and Reid, David and Byravan, Arunkumar and Michaely, Assaf Hurwitz and Heess, Nicolas and Zhou, Luowei and Goenka, Sonam and Carpenter, Viral and Levskaya, Anselm and Wang, Bo and Roberts, Reed and Leblond, Rémi and Chikkerur, Sharat and Ginzburg, Stav and Chang, Max and Riachi, Robert and Chuqiao and Xu and Borsos, Zalán and Pliskin, Michael and Pawar, Julia and Lustman, Morgane and Kirkwood, Hannah and Anand, Ankit and Chaudhary, Aditi and Kalb, Norbert and Milan, Kieran and Augenstein, Sean and Goldie, Anna and Prince, Laurel and Raman, Karthik and Sun, Yanhua and Xia, Vivian and Cohen, Aaron and Huo, Zhouyuan and Camp, Josh and Ellis, Seher and Zilka, Lukas and Torres, David Vilar and Patel, Lisa and Arora, Sho and Chan, Betty and Adler, Jonas and Ayoub, Kareem and Liang, Jacky and Jamil, Fayaz and Jiang, Jiepu and Baumgartner, Simon and Sun, Haitian and Karov, Yael and Akulov, Yaroslav and Zheng, Hui and Cai, Irene and Fantacci, Claudio and Rubin, James and Acha, Alex Rav and Wang, Mengchao and D'Souza, Nina and Sathyanarayana, Rohit and Dai, Shengyang and Rowe, Simon and Simanovsky, Andrey and Goldman, Omer and Kuang, Yuheng and Pan, Xiaoyue and Rosenberg, Andrew and Rojas-Esponda, Tania and Dutta, Praneet and Zeng, Amy and Jurenka, Irina and Farquhar, Greg and Bansal, Yamini and Iqbal, Shariq and Roelofs, Becca and Joung, Ga-Young and Beak, Parker and Ryu, Changwan and Poplin, Ryan and Wu, Yan and Alayrac, Jean-Baptiste and Buthpitiya, Senaka and Ronneberger, Olaf and Habtegebriel, Caleb and Li, Wei and Cavallaro, Paul and Wei, Aurora and Bensky, Guy and Denk, Timo and Ganapathy, Harish and Stanway, Jeff and Joshi, Pratik and Bertolini, Francesco and Lo, Jessica and Ma, Olivia and Charles, Zachary and Sampemane, Geta and Sahni, Himanshu and Chen, Xu and Askham, Harry and Gaddy, David and Young, Peter and Tan, Jiewen and Eyal, Matan and Bražinskas, Arthur and Zhong, Li and Wu, Zhichun and Epstein, Mark and Bailey, Kai and Hard, Andrew and Lee, Kamyu and Goldshtein, Sasha and Ruiz, Alex and Badawi, Mohammed and Lochbrunner, Matthias and Kearns, JK and Brown, Ashley and Pardo, Fabio and Weber, Theophane and Yang, Haichuan and Jiang, Pan-Pan and Akin, Berkin and Fu, Zhao and Wainwright, Marcus and Zou, Chi and Gaba, Meenu and Manzagol, Pierre-Antoine and Kan, Wendy and Song, Yang and Zainullina, Karina and Lin, Rui and Ko, Jeongwoo and Deshmukh, Salil and Jindal, Apoorv and Svensson, James and Tyam, Divya and Zhao, Heri and Kaeser-Chen, Christine and Baird, Scott and Moradi, Pooya and Hall, Jamie and Guo, Qiuchen and Tsang, Vincent and Liang, Bowen and Pereira, Fernando and Ganesh, Suhas and Korotkov, Ivan and Adamek, Jakub and Thiagarajan, Sridhar and Tran, Vinh and Chen, Charles and Tar, Chris and Jain, Sanil and Dasgupta, Ishita and Bilal, Taylan and Reitter, David and Zhao, Kai and Vezzani, Giulia and Gehman, Yasmin and Mehta, Pulkit and Beltrone, Lauren and Dotiwalla, Xerxes and Guadarrama, Sergio and Abbas, Zaheer and Karp, Stefani and Georgiev, Petko and Ferng, Chun-Sung and Brockschmidt, Marc and Peng, Liqian and Hirnschall, Christoph and Verma, Vikas and Bi, Yingying and Xiao, Ying and Dabush, Avigail and Xu, Kelvin and Wallis, Phil and Parker, Randall and Wang, Qifei and Xu, Yang and Safarli, Ilkin and Tewari, Dinesh and Zhang, Yin and Kim, Seungyeon and Gesmundo, Andrea and Thomas, Mackenzie and Levi, Sergey and Chowdhury, Ahmed and Rao, Kanishka and Garst, Peter and Conway-Rahman, Sam and Ran, Helen and McKinney, Kay and Xiao, Zhisheng and Yu, Wenhao and Agrawal, Rohan and Stjerngren, Axel and Ionescu, Catalin and Chen, Jingjing and Sharma, Vivek and Chiu, Justin and Liu, Fei and Franko, Ken and Sanford, Clayton and Cai, Xingyu and Michel, Paul and Ganapathy, Sanjay and Labanowski, Jane and Garrett, Zachary and Vargas, Ben and Sun, Sean and Gale, Bryan and Buschmann, Thomas and Desjardins, Guillaume and Ghelani, Nimesh and Jain, Palak and Verma, Mudit and Asawaroengchai, Chulayuth and Eisenschlos, Julian and Harlalka, Jitendra and Kazawa, Hideto and Metzler, Don and Howland, Joshua and Jian, Ying and Ades, Jake and Shah, Viral and Gangwani, Tynan and Lee, Seungji and Ring, Roman and Hernandez, Steven M. and Reich, Dean and Sinha, Amer and Sathe, Ashutosh and Kovac, Joe and Gill, Ashleah and Kannan, Ajay and D'olimpio, Andrea and Sevenich, Martin and Whang, Jay and Kim, Been and Sim, Khe Chai and Chen, Jilin and Zhang, Jiageng and Lall, Shuba and Matias, Yossi and Jia, Bill and Friesen, Abe and Nasso, Sara and Thapliyal, Ashish and Perozzi, Bryan and Yu, Ting and Shekhawat, Anna and Huda, Safeen and Grabowski, Peter and Wang, Eric and Sreevatsa, Ashwin and Dib, Hilal and Hassen, Mehadi and Schuh, Parker and Milutinovic, Vedrana and Welty, Chris and Quinn, Michael and Shah, Ali and Wang, Bangju and Barth-Maron, Gabe and Frye, Justin and Axelsson, Natalie and Zhu, Tao and Ma, Yukun and Giannoumis, Irene and Sedghi, Hanie and Ye, Chang and Luan, Yi and Aydin, Kevin and Chandra, Bilva and Sampathkumar, Vivek and Huang, Ronny and Lavrenko, Victor and Eleryan, Ahmed and Hong, Zhi and Hansen, Steven and Carthy, Sara Mc and Samanta, Bidisha and Ćevid, Domagoj and Wang, Xin and Li, Fangtao and Voznesensky, Michael and Hoffman, Matt and Terzis, Andreas and Sehwag, Vikash and Fidel, Gil and He, Luheng and Cai, Mu and He, Yanzhang and Feng, Alex and Nikoltchev, Martin and Phatale, Samrat and Chase, Jason and Lawton, Rory and Zhang, Ming and Ouyang, Tom and Tragut, Manuel and Manshadi, Mehdi Hafezi and Narayanan, Arjun and Shen, Jiaming and Gao, Xu and Bolukbasi, Tolga and Roy, Nick and Li, Xin and Golovin, Daniel and Panait, Liviu and Qin, Zhen and Han, Guangxing and Anthony, Thomas and Kudugunta, Sneha and Patraucean, Viorica and Ray, Aniket and Chen, Xinyun and Yang, Xiaochen and Bhatia, Tanuj and Talluri, Pranav and Morris, Alex and Ražnatović, Andrija and Brownfield, Bethanie and An, James and Peng, Sheng and Kane, Patrick and Zheng, Ce and Duduta, Nico and Kessinger, Joshua and Noraky, James and Liu, Siqi and Rong, Keran and Veličković, Petar and Rush, Keith and Goldin, Alex and Wei, Fanny and Garlapati, Shiva Mohan Reddy and Pantofaru, Caroline and Kwon, Okwan and Ni, Jianmo and Noland, Eric and Trapani, Julia Di and Beaufays, Françoise and Roy, Abhijit Guha and Chow, Yinlam and Turker, Aybuke and Cideron, Geoffrey and Mei, Lantao and Clark, Jon and Dou, Qingyun and Bošnjak, Matko and Leith, Ralph and Du, Yuqing and Yazdanbakhsh, Amir and Nasr, Milad and Kwak, Chester and Sheth, Suraj Satishkumar and Kaskasoli, Alex and Anand, Ankesh and Lakshminarayanan, Balaji and Jerome, Sammy and Bieber, David and Chu, Chun-Te and Senges, Alexandre and Shen, Tianxiao and Sridhar, Mukund and Ndebele, Ndaba and Beyret, Benjamin and Mohamed, Shakir and Chen, Mia and Freitag, Markus and Guo, Jiaxian and Liu, Luyang and Roit, Paul and Chen, Heng and Yan, Shen and Stone, Tom and Co-Reyes, JD and Cole, Jeremy and Scellato, Salvatore and Azizi, Shekoofeh and Hashemi, Hadi and Jin, Alicia and Iyer, Anand and Valentine, Marcella and György, András and Ahuja, Arun and Diaz, Daniel Hernandez and Lee, Chen-Yu and Clement, Nathan and Kong, Weize and Garmon, Drew and Watts, Ishaan and Bhatia, Kush and Gupta, Khyatti and Miecnikowski, Matt and Vallet, Hugo and Taly, Ankur and Loper, Edward and Joshi, Saket and Atwood, James and Chick, Jo and Collier, Mark and Iliopoulos, Fotis and Trostle, Ryan and Gunel, Beliz and Leal-Cavazos, Ramiro and Hrafnkelsson, Arnar Mar and Guzman, Michael and Ju, Xiaoen and Forbes, Andy and Emond, Jesse and Chauhan, Kushal and Caine, Ben and Xiao, Li and Zeng, Wenjun and Moufarek, Alexandre and Murphy, Daniel and Meng, Maya and Gupta, Nitish and Riedel, Felix and Das, Anil and Lawal, Elijah and Narayan, Shashi and Sosea, Tiberiu and Swirhun, James and Friso, Linda and Neyshabur, Behnam and Lu, Jing and Girgin, Sertan and Wunder, Michael and Yvinec, Edouard and Pyne, Aroonalok and Carbune, Victor and Rijhwani, Shruti and Guo, Yang and Doshi, Tulsee and Briukhov, Anton and Bain, Max and Hitron, Ayal and Wang, Xuanhui and Gupta, Ashish and Chen, Ke and Du, Cosmo and Zhang, Weiyang and Shah, Dhruv and Akula, Arjun and Dylla, Max and Kachra, Ashyana and Kuo, Weicheng and Zou, Tingting and Wang, Lily and Xu, Luyao and Zhu, Jifan and Snyder, Justin and Menon, Sachit and Firat, Orhan and Mordatch, Igor and Yuan, Yuan and Ponomareva, Natalia and Blevins, Rory and Moore, Lawrence and Wang, Weijun and Chen, Phil and Scholz, Martin and Dwornik, Artur and Lin, Jason and Li, Sicheng and Antognini, Diego and I, Te and Song, Xiaodan and Miller, Matt and Kalra, Uday and Raveret, Adam and Akerlund, Oscar and Wu, Felix and Nystrom, Andrew and Godbole, Namrata and Liu, Tianqi and DeBalsi, Hannah and Zhao, Jewel and Liu, Buhuang and Caciularu, Avi and Lax, Lauren and Khandelwal, Urvashi and Langston, Victoria and Bailey, Eric and Lattanzi, Silvio and Wang, Yufei and Kovelamudi, Neel and Mondal, Sneha and Guruganesh, Guru and Hua, Nan and Roval, Ofir and Wesołowski, Paweł and Ingale, Rishikesh and Halcrow, Jonathan and Sohn, Tim and Angermueller, Christof and Raad, Bahram and Stickgold, Eli and Lu, Eva and Kosik, Alec and Xie, Jing and Lillicrap, Timothy and Huang, Austin and Zhang, Lydia Lihui and Paulus, Dominik and Farabet, Clement and Wertheim, Alex and Wang, Bing and Joshi, Rishabh and Ko, Chu-ling and Wu, Yonghui and Agrawal, Shubham and Lin, Lily and Sheng, XiangHai and Sung, Peter and Breland-King, Tyler and Butterfield, Christina and Gawde, Swapnil and Singh, Sumeet and Zhang, Qiao and Apte, Raj and Shetty, Shilpa and Hutter, Adrian and Li, Tao and Salesky, Elizabeth and Lebron, Federico and Kanerva, Jonni and Paganini, Michela and Nguyen, Arthur and Vallu, Rohith and Peter, Jan-Thorsten and Velury, Sarmishta and Kao, David and Hoover, Jay and Bortsova, Anna and Bishop, Colton and Jakobovits, Shoshana and Agostini, Alessandro and Agarwal, Alekh and Liu, Chang and Kwong, Charles and Tavakkol, Sasan and Bica, Ioana and Greve, Alex and GP, Anirudh and Marcus, Jake and Hou, Le and Duerig, Tom and Moroshko, Rivka and Lacey, Dave and Davis, Andy and Amelot, Julien and Wang, Guohui and Kim, Frank and Strinopoulos, Theofilos and Wan, Hui and Lan, Charline Le and Krishnan, Shankar and Tang, Haotian and Humphreys, Peter and Bai, Junwen and Shtacher, Idan Heimlich and Machado, Diego and Pang, Chenxi and Burke, Ken and Liu, Dangyi and Aravamudhan, Renga and Song, Yue and Hirst, Ed and Singh, Abhimanyu and Jou, Brendan and Bai, Liang and Piccinno, Francesco and Fu, Chuyuan Kelly and Alazard, Robin and Meiri, Barak and Winter, Daniel and Chen, Charlie and Zhang, Mingda and Heitkaemper, Jens and Lambert, John and Lee, Jinhyuk and Frömmgen, Alexander and Rogulenko, Sergey and Nair, Pranav and Niemczyk, Paul and Bulyenov, Anton and Xu, Bibo and Shemtov, Hadar and Zadimoghaddam, Morteza and Toropov, Serge and Wirth, Mateo and Dai, Hanjun and Gollapudi, Sreenivas and Zheng, Daniel and Kurakin, Alex and Lee, Chansoo and Bullard, Kalesha and Serrano, Nicolas and Balazevic, Ivana and Li, Yang and Schalkwyk, Johan and Murphy, Mark and Zhang, Mingyang and Sequeira, Kevin and Datta, Romina and Agrawal, Nishant and Sutton, Charles and Attaluri, Nithya and Chiang, Mencher and Farhan, Wael and Thornton, Gregory and Lin, Kate and Choma, Travis and Nguyen, Hung and Dasgupta, Kingshuk and Robinson, Dirk and Comşa, Iulia and Riley, Michael and Pillai, Arjun and Mustafa, Basil and Golan, Ben and Zandieh, Amir and Lespiau, Jean-Baptiste and Porter, Billy and Ross, David and Rajayogam, Sujeevan and Agarwal, Mohit and Venugopalan, Subhashini and Shahriari, Bobak and Yan, Qiqi and Xu, Hao and Tobin, Taylor and Dubov, Pavel and Shi, Hongzhi and Recasens, Adrià and Kovsharov, Anton and Borgeaud, Sebastian and Dery, Lucio and Vasanth, Shanthal and Gribovskaya, Elena and Qiu, Linhai and Mahdieh, Mahdis and Skut, Wojtek and Nielsen, Elizabeth and Zheng, CJ and Yu, Adams and Bostock, Carrie Grimes and Gupta, Shaleen and Archer, Aaron and Rawles, Chris and Davies, Elinor and Svyatkovskiy, Alexey and Tsai, Tomy and Halpern, Yoni and Reisswig, Christian and Wydrowski, Bartek and Chang, Bo and Puigcerver, Joan and Taege, Mor Hazan and Li, Jian and Schnider, Eva and Li, Xinjian and Dena, Dragos and Xu, Yunhan and Telang, Umesh and Shi, Tianze and Zen, Heiga and Kastner, Kyle and Ko, Yeongil and Subramaniam, Neesha and Kumar, Aviral and Blois, Pete and Dai, Zhuyun and Wieting, John and Lu, Yifeng and Zeldes, Yoel and Xie, Tian and Hauth, Anja and Ţifrea, Alexandru and Li, Yuqi and El-Husseini, Sam and Abolafia, Dan and Zhou, Howard and Ding, Wen and Ghalebikesabi, Sahra and Guía, Carlos and Maksai, Andrii and Weisz, Ágoston and Arik, Sercan and Sukhanov, Nick and Świetlik, Aga and Jia, Xuhui and Yu, Luo and Wang, Weiyue and Brand, Mark and Bloxwich, Dawn and Kirmani, Sean and Chen, Zhe and Go, Alec and Sprechmann, Pablo and Kannen, Nithish and Carin, Alen and Sandhu, Paramjit and Edkins, Isabel and Nooteboom, Leslie and Gupta, Jai and Maggiore, Loren and Azizi, Javad and Pritch, Yael and Yin, Pengcheng and Gupta, Mansi and Tarlow, Danny and Smith, Duncan and Ivanov, Desi and Babaeizadeh, Mohammad and Goel, Ankita and Kambala, Satish and Chu, Grace and Kastelic, Matej and Liu, Michelle and Soltau, Hagen and Stone, Austin and Agrawal, Shivani and Kim, Min and Soparkar, Kedar and Tadepalli, Srinivas and Bunyan, Oskar and Soh, Rachel and Kannan, Arvind and Kim, DY and Chen, Blake JianHang and Halumi, Afief and Roy, Sudeshna and Wang, Yulong and Sercinoglu, Olcan and Gibson, Gena and Bhatnagar, Sijal and Sano, Motoki and Dincklage, Daniel von and Ren, Qingchun and Mitrevski, Blagoj and Olšák, Mirek and She, Jennifer and Doersch, Carl and Jilei and Wang and Liu, Bingyuan and Tan, Qijun and Yakar, Tamar and Warkentin, Tris and Ramirez, Alex and Lebsack, Carl and Dillon, Josh and Mathews, Rajiv and Cobley, Tom and Wu, Zelin and Chen, Zhuoyuan and Simon, Jon and Nath, Swaroop and Sainath, Tara and Bendebury, Alexei and Julian, Ryan and Mankalale, Bharath and Ćurko, Daria and Zacchello, Paulo and Brown, Adam R. and Sodhia, Kiranbir and Howard, Heidi and Caelles, Sergi and Gupta, Abhinav and Evans, Gareth and Bulanova, Anna and Katzen, Lesley and Goldenberg, Roman and Tsitsulin, Anton and Stanton, Joe and Schillings, Benoit and Kovalev, Vitaly and Fry, Corey and Shah, Rushin and Lin, Kuo and Upadhyay, Shyam and Li, Cheng and Radpour, Soroush and Maggioni, Marcello and Xiong, Jing and Haas, Lukas and Brennan, Jenny and Kamath, Aishwarya and Savinov, Nikolay and Nagrani, Arsha and Yacovone, Trevor and Kappedal, Ryan and Andriopoulos, Kostas and Lao, Li and Li, YaGuang and Rozhdestvenskiy, Grigory and Hashimoto, Kazuma and Audibert, Andrew and Austin, Sophia and Rodriguez, Daniel and Ruoss, Anian and Honke, Garrett and Karkhanis, Deep and Xiong, Xi and Wei, Qing and Huang, James and Leng, Zhaoqi and Premachandran, Vittal and Bileschi, Stan and Evangelopoulos, Georgios and Mensink, Thomas and Pavagadhi, Jay and Teplyashin, Denis and Chang, Paul and Xue, Linting and Tanzer, Garrett and Goldman, Sally and Patel, Kaushal and Li, Shixin and Wiesner, Jeremy and Zheng, Ivy and Stewart-Binks, Ian and Han, Jie and Li, Zhi and Luo, Liangchen and Lenc, Karel and Lučić, Mario and Xue, Fuzhao and Mullins, Ryan and Guseynov, Alexey and Chang, Chung-Ching and Galatzer-Levy, Isaac and Zhang, Adam and Bingham, Garrett and Hu, Grace and Hartman, Ale and Ma, Yue and Griffith, Jordan and Irpan, Alex and Radebaugh, Carey and Yue, Summer and Fan, Lijie and Ungureanu, Victor and Sorokin, Christina and Teufel, Hannah and Li, Peiran and Anil, Rohan and Paparas, Dimitris and Wang, Todd and Lin, Chu-Cheng and Peng, Hui and Shum, Megan and Petrovic, Goran and Brady, Demetra and Nguyen, Richard and Macherey, Klaus and Li, Zhihao and Singh, Harman and Yenugula, Madhavi and Iinuma, Mariko and Chen, Xinyi and Kopparapu, Kavya and Stern, Alexey and Dave, Shachi and Thekkath, Chandu and Perot, Florence and Kumar, Anurag and Li, Fangda and Xiao, Yang and Bilotti, Matthew and Bateni, Mohammad Hossein and Noble, Isaac and Lee, Lisa and Vázquez-Reina, Amelio and Salazar, Julian and Yang, Xiaomeng and Wang, Boyu and Gruzewska, Ela and Rao, Anand and Raghuram, Sindhu and Xu, Zheng and Ben-David, Eyal and Mei, Jieru and Dalmia, Sid and Zhang, Zhaoyi and Liu, Yuchen and Bansal, Gagan and Pankov, Helena and Schwarcz, Steven and Burns, Andrea and Chan, Christine and Sanghai, Sumit and Liang, Ricky and Liang, Ethan and He, Antoine and Stuart, Amy and Narayanan, Arun and Zhu, Yukun and Frank, Christian and Fatemi, Bahar and Sabne, Amit and Lang, Oran and Bhattacharya, Indro and Settle, Shane and Wang, Maria and McMahan, Brendan and Tacchetti, Andrea and Soares, Livio Baldini and Hadian, Majid and Cabi, Serkan and Chung, Timothy and Putikhin, Nikita and Li, Gang and Chen, Jeremy and Tarango, Austin and Michalewski, Henryk and Kazemi, Mehran and Masoom, Hussain and Sheftel, Hila and Shivanna, Rakesh and Vadali, Archita and Comanescu, Ramona and Reid, Doug and Moore, Joss and Neelakantan, Arvind and Sander, Michaël and Herzig, Jonathan and Rosenberg, Aviv and Dehghani, Mostafa and Choi, JD and Fink, Michael and Hayes, Reid and Ge, Eric and Weng, Shitao and Ho, Chia-Hua and Karro, John and Krishna, Kalpesh and Thiet, Lam Nguyen and Skerry-Ryan, Amy and Eppens, Daniel and Andreetto, Marco and Sarma, Navin and Bonacina, Silvano and Ayan, Burcu Karagol and Nawhal, Megha and Shan, Zhihao and Dusenberry, Mike and Thakoor, Shantanu and Gubbi, Sagar and Nguyen, Duc Dung and Tsarfaty, Reut and Albanie, Samuel and Mitrović, Jovana and Gandhi, Meet and Chen, Bo-Juen and Epasto, Alessandro and Stephanov, Georgi and Jin, Ye and Gehman, Samuel and Amini, Aida and Weber, Jack and Behbahani, Feryal and Xu, Shawn and Allamanis, Miltos and Chen, Xi and Ott, Myle and Sha, Claire and Jastrzebski, Michal and Qi, Hang and Greene, David and Wu, Xinyi and Toki, Abodunrinwa and Vlasic, Daniel and Shapiro, Jane and Kotikalapudi, Ragha and Shen, Zhe and Saeki, Takaaki and Xie, Sirui and Cassirer, Albin and Bharadwaj, Shikhar and Kiyono, Tatsuya and Bhojanapalli, Srinadh and Rosenfeld, Elan and Ritter, Sam and Mao, Jieming and Oliveira, João Gabriel and Egyed, Zoltan and Bandemer, Bernd and Parisotto, Emilio and Kinoshita, Keisuke and Pluto, Juliette and Maniatis, Petros and Li, Steve and Guo, Yaohui and Ghiasi, Golnaz and Tarbouriech, Jean and Chatterjee, Srimon and Jin, Julie and Katrina and Xu and Palomaki, Jennimaria and Arnold, Séb and Sewak, Madhavi and Piccinini, Federico and Sharma, Mohit and Albrecht, Ben and Purser-haskell, Sean and Vaswani, Ashwin and Chen, Chongyan and Wisniewski, Matheus and Cao, Qin and Aslanides, John and Phu, Nguyet Minh and Sieb, Maximilian and Agubuzu, Lauren and Zheng, Anne and Sohn, Daniel and Selvi, Marco and Andreassen, Anders and Subudhi, Krishan and Eruvbetine, Prem and Woodman, Oliver and Mery, Tomas and Krause, Sebastian and Ren, Xiaoqi and Ma, Xiao and Luo, Jincheng and Chen, Dawn and Fan, Wei and Griffiths, Henry and Schuler, Christian and Li, Alice and Zhang, Shujian and Sarr, Jean-Michel and Luo, Shixin and Patana, Riccardo and Watson, Matthew and Naboulsi, Dani and Collins, Michael and Sidhwani, Sailesh and Hoogeboom, Emiel and Silver, Sharon and Caveness, Emily and Zhao, Xiaokai and Rodriguez, Mikel and Deines, Maxine and Bai, Libin and Griffin, Patrick and Tagliasacchi, Marco and Xue, Emily and Babbula, Spandana Raj and Pang, Bo and Ding, Nan and Shen, Gloria and Peake, Elijah and Crocker, Remi and Raghvendra, Shubha Srinivas and Swisher, Danny and Han, Woohyun and Singh, Richa and Wu, Ling and Pchelin, Vladimir and Munkhdalai, Tsendsuren and Alon, Dana and Bacon, Geoff and Robles, Efren and Bulian, Jannis and Johnson, Melvin and Powell, George and Ferreira, Felipe Tiengo and Li, Yaoyiran and Benzing, Frederik and Velimirović, Mihajlo and Soyer, Hubert and Kong, William and Tony and Nguyên and Yang, Zhen and Liu, Jeremiah and Amersfoort, Joost van and Gillick, Daniel and Sun, Baochen and Rauschmayr, Nathalie and Zhang, Katie and Zhan, Serena and Zhou, Tao and Frolov, Alexey and Yang, Chengrun and Vnukov, Denis and Rouillard, Louis and Li, Hongji and Mandhane, Amol and Fallen, Nova and Venkataraman, Rajesh and Hu, Clara Huiyi and Brennan, Jennifer and Lee, Jenny and Chang, Jerry and Sundermeyer, Martin and Pan, Zhufeng and Ke, Rosemary and Tong, Simon and Fabrikant, Alex and Bono, William and Gu, Jindong and Foley, Ryan and Mao, Yiran and Delakis, Manolis and Bhaswar, Dhruva and Frostig, Roy and Li, Nick and Zipori, Avital and Hope, Cath and Kozlova, Olga and Mishra, Swaroop and Djolonga, Josip and Schiff, Craig and Merey, Majd Al and Briakou, Eleftheria and Morgan, Peter and Wan, Andy and Hassidim, Avinatan and Skerry-Ryan, RJ and Sengupta, Kuntal and Jasarevic, Mary and Kallakuri, Praveen and Kunkle, Paige and Brennan, Hannah and Lieber, Tom and Mansoor, Hassan and Walker, Julian and Zhang, Bing and Xie, Annie and Žužić, Goran and Chukwuka, Adaeze and Druinsky, Alex and Cho, Donghyun and Yao, Rui and Naeem, Ferjad and Butt, Shiraz and Kim, Eunyoung and Jia, Zhipeng and Jordan, Mandy and Lelkes, Adam and Kurzeja, Mark and Wang, Sophie and Zhao, James and Over, Andrew and Chakladar, Abhishek and Prasetya, Marcel and Jha, Neha and Ganapathy, Sriram and Cong, Yale and Shroff, Prakash and Saroufim, Carl and Miryoosefi, Sobhan and Hammad, Mohamed and Nasir, Tajwar and Xi, Weijuan and Gao, Yang and Maeng, Young and Hora, Ben and Cheng, Chin-Yi and Haghani, Parisa and Lewenberg, Yoad and Lu, Caden and Matysiak, Martin and Raisinghani, Naina and Wang, Huiyu and Baugher, Lexi and Sukthankar, Rahul and Giang, Minh and Schultz, John and Fiedel, Noah and Chen, Minmin and Lee, Cheng-Chun and Dey, Tapomay and Zheng, Hao and Paul, Shachi and Smith, Celine and Ly, Andy and Wang, Yicheng and Bansal, Rishabh and Perz, Bartek and Ricco, Susanna and Blank, Stasha and Keshava, Vaishakh and Sharma, Deepak and Chow, Marvin and Lad, Kunal and Jalan, Komal and Osindero, Simon and Swanson, Craig and Scott, Jacob and Ilić, Anastasija and Li, Xiaowei and Jonnalagadda, Siddhartha Reddy and Soudagar, Afzal Shama and Xiong, Yan and Batsaikhan, Bat-Orgil and Jarrett, Daniel and Kumar, Naveen and Shah, Maulik and Lawlor, Matt and Waters, Austin and Graham, Mark and May, Rhys and Ramos, Sabela and Lefdal, Sandra and Cankara, Zeynep and Cano, Nacho and O'Donoghue, Brendan and Borovik, Jed and Liu, Frederick and Grimstad, Jordan and Alnahlawi, Mahmoud and Tsihlas, Katerina and Hudson, Tom and Grigorev, Nikolai and Jia, Yiling and Huang, Terry and Igwe, Tobenna Peter and Lebedev, Sergei and Tang, Xiaodan and Krivokon, Igor and Garcia, Frankie and Tan, Melissa and Jia, Eric and Stys, Peter and Vashishth, Shikhar and Liang, Yu and Venkatraman, Balaji and Gu, Chenjie and Kementsietsidis, Anastasios and Zhu, Chen and Jung, Junehyuk and Bai, Yunfei and Hosseini, Mohammad Javad and Ahmed, Faruk and Gupta, Aditya and Yuan, Xin and Ashraf, Shereen and Nigam, Shitij and Vasudevan, Gautam and Awasthi, Pranjal and Gilady, Adi Mayrav and Mariet, Zelda and Eskander, Ramy and Li, Haiguang and Hu, Hexiang and Garrido, Guillermo and Schlattner, Philippe and Zhang, George and Saxena, Rohun and Dević, Petar and Muralidharan, Kritika and Murthy, Ashwin and Zhou, Yiqian and Choi, Min and Wongpanich, Arissa and Wang, Zhengdong and Shah, Premal and Xu, Yuntao and Huang, Yiling and Spencer, Stephen and Chen, Alice and Cohan, James and Wang, Junjie and Tompson, Jonathan and Wu, Junru and Haroun, Ruba and Li, Haiqiong and Huergo, Blanca and Yang, Fan and Yin, Tongxin and Wendt, James and Bendersky, Michael and Chaabouni, Rahma and Snaider, Javier and Ferret, Johan and Jindal, Abhishek and Thompson, Tara and Xue, Andrew and Bishop, Will and Phal, Shubham Milind and Sharma, Archit and Sung, Yunhsuan and Radhakrishnan, Prabakar and Shomrat, Mo and Ingle, Reeve and Vij, Roopali and Gilmer, Justin and Istin, Mihai Dorin and Sobell, Sam and Lu, Yang and Nottage, Emily and Sadigh, Dorsa and Willcock, Jeremiah and Zhang, Tingnan and Xu, Steve and Brown, Sasha and Lee, Katherine and Wang, Gary and Zhu, Yun and Tay, Yi and Kim, Cheolmin and Gutierrez, Audrey and Sharma, Abhanshu and Xian, Yongqin and Seo, Sungyong and Cui, Claire and Pochernina, Elena and Baetu, Cip and Jastrzębski, Krzysztof and Ly, Mimi and Elhawaty, Mohamed and Suh, Dan and Sezener, Eren and Wang, Pidong and Yuen, Nancy and Tucker, George and Cai, Jiahao and Yang, Zuguang and Wang, Cindy and Muzio, Alex and Qian, Hai and Yoo, Jae and Lockhart, Derek and McKee, Kevin R. and Guo, Mandy and Mehrotra, Malika and Mendonça, Artur and Mehta, Sanket Vaibhav and Ben, Sherry and Tekur, Chetan and Mu, Jiaqi and Zhu, Muye and Krakovna, Victoria and Lee, Hongrae and Maschinot, AJ and Cevey, Sébastien and Choe, HyunJeong and Bai, Aijun and Srinivasan, Hansa and Gasaway, Derek and Young, Nick and Siegler, Patrick and Holtmann-Rice, Dan and Piratla, Vihari and Baumli, Kate and Yogev, Roey and Hofer, Alex and Hasselt, Hado van and Grant, Svetlana and Chervonyi, Yuri and Silver, David and Hogue, Andrew and Agarwal, Ayushi and Wang, Kathie and Singh, Preeti and Flynn, Four and Lipschultz, Josh and David, Robert and Bellot, Lizzetth and Yang, Yao-Yuan and Le, Long and Graziano, Filippo and Olszewska, Kate and Hui, Kevin and Maurya, Akanksha and Parotsidis, Nikos and Chen, Weijie and Oguntebi, Tayo and Kelley, Joe and Baddepudi, Anirudh and Mauerer, Johannes and Shaw, Gregory and Siegman, Alex and Yang, Lin and Shetty, Shravya and Roy, Subhrajit and Song, Yunting and Stokowiec, Wojciech and Burnell, Ryan and Savant, Omkar and Busa-Fekete, Robert and Miao, Jin and Ghosh, Samrat and MacDermed, Liam and Lippe, Phillip and Dektiarev, Mikhail and Behrman, Zach and Mentzer, Fabian and Nguyen, Kelvin and Wei, Meng and Verma, Siddharth and Knutsen, Chris and Dasari, Sudeep and Yan, Zhipeng and Mitrichev, Petr and Wang, Xingyu and Shejwalkar, Virat and Austin, Jacob and Sunkara, Srinivas and Potti, Navneet and Virin, Yan and Wright, Christian and Liu, Gaël and Riva, Oriana and Pot, Etienne and Kochanski, Greg and Le, Quoc and Balasubramaniam, Gargi and Dhar, Arka and Liao, Yuguo and Bloniarz, Adam and Shukla, Divyansh and Cole, Elizabeth and Lee, Jong and Zhang, Sheng and Kafle, Sushant and Vashishtha, Siddharth and Mahmoudieh, Parsa and Chen, Grace and Hoffmann, Raphael and Srinivasan, Pranesh and Lago, Agustin Dal and Shalom, Yoav Ben and Wang, Zi and Elabd, Michael and Sharma, Anuj and Oh, Junhyuk and Kothawade, Suraj and Le, Maigo and Monteiro, Marianne and Yang, Shentao and Alarakyia, Kaiz and Geirhos, Robert and Mincu, Diana and Garnes, Håvard and Kobayashi, Hayato and Mariooryad, Soroosh and Krasowiak, Kacper and Zhixin and Lai and Mourad, Shibl and Wang, Mingqiu and Bu, Fan and Aharoni, Ophir and Chen, Guanjie and Goyal, Abhimanyu and Zubov, Vadim and Bapna, Ankur and Dabir, Elahe and Kothari, Nisarg and Lamerigts, Kay and Cao, Nicola De and Shar, Jeremy and Yew, Christopher and Kulkarni, Nitish and Mahaarachchi, Dre and Joshi, Mandar and Zhu, Zhenhai and Lichtarge, Jared and Zhou, Yichao and Muckenhirn, Hannah and Selo, Vittorio and Vinyals, Oriol and Chen, Peter and Brohan, Anthony and Mehta, Vaibhav and Cogan, Sarah and Wang, Ruth and Geri, Ty and Ko, Wei-Jen and Chen, Wei and Viola, Fabio and Shivam, Keshav and Wang, Lisa and Elish, Madeleine Clare and Popa, Raluca Ada and Pereira, Sébastien and Liu, Jianqiao and Koster, Raphael and Kim, Donnie and Zhang, Gufeng and Ebrahimi, Sayna and Talukdar, Partha and Zheng, Yanyan and Poklukar, Petra and Mikhalap, Ales and Johnson, Dale and Vijayakumar, Anitha and Omernick, Mark and Dibb, Matt and Dubey, Ayush and Hu, Qiong and Suman, Apurv and Aggarwal, Vaibhav and Kornakov, Ilya and Xia, Fei and Lowe, Wing and Kolganov, Alexey and Xiao, Ted and Nikolaev, Vitaly and Hemingray, Steven and Li, Bonnie and Iljazi, Joana and Rybiński, Mikołaj and Sandhu, Ballie and Lu, Peggy and Luong, Thang and Jenatton, Rodolphe and Govindaraj, Vineetha and Hui and Li and Dulac-Arnold, Gabriel and Park, Wonpyo and Wang, Henry and Modi, Abhinit and Pouget-Abadie, Jean and Greller, Kristina and Gupta, Rahul and Berry, Robert and Ramachandran, Prajit and Xie, Jinyu and McCafferty, Liam and Wang, Jianling and Gupta, Kilol and Lim, Hyeontaek and Bratanič, Blaž and Brock, Andy and Akolzin, Ilia and Sproch, Jim and Karliner, Dan and Kim, Duhyeon and Goedeckemeyer, Adrian and Shazeer, Noam and Schmid, Cordelia and Calandriello, Daniele and Bhatia, Parul and Choromanski, Krzysztof and Montgomery, Ceslee and Dua, Dheeru and Ramalho, Ana and King, Helen and Gao, Yue and Nguyen, Lynn and Lindner, David and Pitta, Divya and Johnson, Oleaser and Salama, Khalid and Ardila, Diego and Han, Michael and Farnese, Erin and Odoom, Seth and Wang, Ziyue and Ding, Xiangzhuo and Rink, Norman and Smith, Ray and Lehri, Harshal Tushar and Cohen, Eden and Vats, Neera and He, Tong and Gopavarapu, Parthasarathy and Paszke, Adam and Patel, Miteyan and Gansbeke, Wouter Van and Loher, Lucia and Castro, Luis and Voitovich, Maria and Glehn, Tamara von and George, Nelson and Niklaus, Simon and Eaton-Rosen, Zach and Rakićević, Nemanja and Jue, Erik and Perel, Sagi and Zhang, Carrie and Bahat, Yuval and Pouget, Angéline and Xing, Zhi and Huot, Fantine and Shenoy, Ashish and Bos, Taylor and Coriou, Vincent and Richter, Bryan and Noy, Natasha and Wang, Yaqing and Ontanon, Santiago and Qin, Siyang and Makarchuk, Gleb and Hassabis, Demis and Li, Zhuowan and Sharma, Mandar and Venkatesan, Kumaran and Kemaev, Iurii and Daniel, Roxanne and Huang, Shiyu and Shah, Saloni and Ponce, Octavio and Warren and Chen and Faruqui, Manaal and Wu, Jialin and Andačić, Slavica and Payrits, Szabolcs and McDuff, Daniel and Hume, Tom and Cao, Yuan and Tessler, MH and Wang, Qingze and Wang, Yinan and Rendulic, Ivor and Agustsson, Eirikur and Johnson, Matthew and Lando, Tanya and Howard, Andrew and Padmanabhan, Sri Gayatri Sundara and Daswani, Mayank and Banino, Andrea and Kilgore, Michael and Heek, Jonathan and Ji, Ziwei and Caceres, Alvaro and Li, Conglong and Kassner, Nora and Vlaskin, Alexey and Liu, Zeyu and Grills, Alex and Hou, Yanhan and Sukkerd, Roykrong and Cheon, Gowoon and Shetty, Nishita and Markeeva, Larisa and Stanczyk, Piotr and Iyer, Tejas and Gong, Yuan and Gao, Shawn and Gopalakrishnan, Keerthana and Blyth, Tim and Reynolds, Malcolm and Bhoopchand, Avishkar and Bilenko, Misha and Gharibian, Dero and Zayats, Vicky and Faust, Aleksandra and Singh, Abhinav and Ma, Min and Jiao, Hongyang and Vijayanarasimhan, Sudheendra and Aroyo, Lora and Yadav, Vikas and Chakera, Sarah and Kakarla, Ashwin and Meshram, Vilobh and Gregor, Karol and Botea, Gabriela and Senter, Evan and Jia, Dawei and Kovacs, Geza and Sharma, Neha and Baur, Sebastien and Kang, Kai and He, Yifan and Zhuo, Lin and Kostelac, Marija and Laish, Itay and Peng, Songyou and O'Bryan, Louis and Kasenberg, Daniel and Rao, Girish Ramchandra and Leurent, Edouard and Zhang, Biao and Stevens, Sage and Salazar, Ana and Zhang, Ye and Lobov, Ivan and Walker, Jake and Porter, Allen and Redshaw, Morgan and Ke, Han and Rao, Abhishek and Lee, Alex and Lam, Hoi and Moffitt, Michael and Kim, Jaeyoun and Qiao, Siyuan and Koo, Terry and Dadashi, Robert and Song, Xinying and Sundararajan, Mukund and Xu, Peng and Kawamoto, Chizu and Zhong, Yan and Barbu, Clara and Reddy, Apoorv and Verzetti, Mauro and Li, Leon and Papamakarios, George and Klimczak-Plucińska, Hanna and Cassin, Mary and Kavukcuoglu, Koray and Swavely, Rigel and Vaucher, Alain and Zhao, Jeffrey and Hemsley, Ross and Tschannen, Michael and Ge, Heming and Menghani, Gaurav and Yu, Yang and Ha, Natalie and He, Wei and Wu, Xiao and Song, Maggie and Sterneck, Rachel and Zinke, Stefan and Calian, Dan A. and Marsden, Annie and Ruiz, Alejandro Cruzado and Hessel, Matteo and Gueta, Almog and Lee, Benjamin and Farris, Brian and Gupta, Manish and Li, Yunjie and Saleh, Mohammad and Misra, Vedant and Xiao, Kefan and Mendolicchio, Piermaria and Buttimore, Gavin and Krayvanova, Varvara and Nayakanti, Nigamaa and Wiethoff, Matthew and Pande, Yash and Mirhoseini, Azalia and Lao, Ni and Liu, Jasmine and Hua, Yiqing and Chen, Angie and Malkov, Yury and Kalashnikov, Dmitry and Gupta, Shubham and Audhkhasi, Kartik and Zhai, Yuexiang and Kopalle, Sudhindra and Jain, Prateek and Ofek, Eran and Meyer, Clemens and Baatarsukh, Khuslen and Strejček, Hana and Qian, Jun and Freedman, James and Figueira, Ricardo and Sokolik, Michal and Bachem, Olivier and Lin, Raymond and Kharrat, Dia and Hidey, Chris and Xu, Pingmei and Duan, Dennis and Li, Yin and Ersoy, Muge and Everett, Richard and Cen, Kevin and Santamaria-Fernandez, Rebeca and Taubenfeld, Amir and Mackinnon, Ian and Deng, Linda and Zablotskaia, Polina and Viswanadha, Shashank and Goel, Shivanker and Yates, Damion and Deng, Yunxiao and Choy, Peter and Chen, Mingqing and Sinha, Abhishek and Mossin, Alex and Wang, Yiming and Szlam, Arthur and Hao, Susan and Rubenstein, Paul Kishan and Toksoz-Exley, Metin and Aperghis, Miranda and Zhong, Yin and Ahn, Junwhan and Isard, Michael and Lacombe, Olivier and Luisier, Florian and Anastasiou, Chrysovalantis and Kalley, Yogesh and Prabhu, Utsav and Dunleavy, Emma and Bijwadia, Shaan and Mao-Jones, Justin and Chen, Kelly and Pasumarthi, Rama and Wood, Emily and Dostmohamed, Adil and Hurley, Nate and Simsa, Jiri and Parrish, Alicia and Pajarskas, Mantas and Harvey, Matt and Skopek, Ondrej and Kochinski, Yony and Rey, Javier and Rieser, Verena and Zhou, Denny and Lee, Sun Jae and Acharya, Trilok and Li, Guowang and Jiang, Joe and Zhang, Xiaofan and Gipson, Bryant and Mahintorabi, Ethan and Gelmi, Marco and Khajehnouri, Nima and Yeh, Angel and Lee, Kayi and Matthey, Loic and Baker, Leslie and Pham, Trang and Fu, Han and Pak, Alex and Gupta, Prakhar and Vasconcelos, Cristina and Sadovsky, Adam and Walker, Brian and Hsiao, Sissie and Zochbauer, Patrik and Marzoca, Andreea and Velan, Noam and Zeng, Junhao and Baechler, Gilles and Driess, Danny and Jain, Divya and Huang, Yanping and Tao, Lizzie and Maggs, John and Levine, Nir and Schneider, Jon and Gemzer, Erika and Petit, Samuel and Han, Shan and Fisher, Zach and Zelle, Dustin and Biles, Courtney and Ie, Eugene and Fadeeva, Asya and Liu, Casper and Franco, Juliana Vicente and Collister, Adrian and Zhang, Hao and Wang, Renshen and Zhao, Ruizhe and Kieliger, Leandro and Shuster, Kurt and Zhu, Rui and Gong, Boqing and Chan, Lawrence and Sun, Ruoxi and Basu, Sujoy and Zimmermann, Roland and Hayes, Jamie and Bapna, Abhishek and Snoek, Jasper and Yang, Weel and Datta, Puranjay and Abdallah, Jad Al and Kilgour, Kevin and Li, Lu and Mah, SQ and Jun, Yennie and Rivière, Morgane and Karmarkar, Abhijit and Spalink, Tammo and Huang, Tao and Gonzalez, Lucas and Tran, Duc-Hieu and Nowak, Averi and Palowitch, John and Chadwick, Martin and Talius, Ellie and Mehta, Harsh and Sellam, Thibault and Fränken, Philipp and Nicosia, Massimo and He, Kyle and Kini, Aditya and Amos, David and Basu, Sugato and Jobe, Harrison and Shaw, Eleni and Xu, Qiantong and Evans, Colin and Ikeda, Daisuke and Yan, Chaochao and Jin, Larry and Wang, Lun and Yadav, Sachin and Labzovsky, Ilia and Sampath, Ramesh and Ma, Ada and Schumann, Candice and Siddhant, Aditya and Shah, Rohin and Youssef, John and Agarwal, Rishabh and Dabney, Natalie and Tonioni, Alessio and Ambar, Moran and Li, Jing and Guyon, Isabelle and Li, Benny and Soergel, David and Fang, Boya and Karadzhov, Georgi and Udrescu, Cristian and Trinh, Trieu and Raunak, Vikas and Noury, Seb and Guo, Dee and Gupta, Sonal and Finkelstein, Mara and Petek, Denis and Liang, Lihao and Billock, Greg and Sun, Pei and Wood, David and Song, Yiwen and Yu, Xiaobin and Matejovicova, Tatiana and Cohen, Regev and Andra, Kalyan and D'Ambrosio, David and Deng, Zhiwei and Nallatamby, Vincent and Songhori, Ebrahim and Dangovski, Rumen and Lampinen, Andrew and Botadra, Pankil and Hillier, Adam and Cao, Jiawei and Baddi, Nagabhushan and Kuncoro, Adhi and Yoshino, Toshihiro and Bhagatwala, Ankit and Ranzato, Marcáurelio and Schaeffer, Rylan and Liu, Tianlin and Ye, Shuai and Sarvana, Obaid and Nham, John and Kuang, Chenkai and Gao, Isabel and Baek, Jinoo and Mittal, Shubham and Wahid, Ayzaan and Gergely, Anita and Ni, Bin and Feldman, Josh and Muir, Carrie and Lamblin, Pascal and Macherey, Wolfgang and Dyer, Ethan and Kilpatrick, Logan and Campos, Víctor and Bhutani, Mukul and Fort, Stanislav and Ahmad, Yanif and Severyn, Aliaksei and Chatziprimou, Kleopatra and Ferludin, Oleksandr and Dimarco, Mason and Kusupati, Aditya and Heyward, Joe and Bahir, Dan and Villela, Kevin and Millican, Katie and Marcus, Dror and Bahargam, Sanaz and Unlu, Caglar and Roth, Nicholas and Wei, Zichuan and Gopal, Siddharth and Ghoshal, Deepanway and Lee, Edward and Lin, Sharon and Lees, Jennie and Lee, Dayeong and Hosseini, Anahita and Fan, Connie and Neel, Seth and Wu, Marcus and Altun, Yasemin and Cai, Honglong and Piqueras, Enrique and Woodward, Josh and Bissacco, Alessandro and Haykal, Salem and Bordbar, Mahyar and Sundaram, Prasha and Hodkinson, Sarah and Toyama, Daniel and Polovets, George and Myers, Austin and Sinha, Anu and Levinboim, Tomer and Krishnakumar, Kashyap and Chhaparia, Rachita and Sholokhova, Tatiana and Gundavarapu, Nitesh Bharadwaj and Jawahar, Ganesh and Qureshi, Haroon and Hu, Jieru and Momchev, Nikola and Rahtz, Matthew and Wu, Renjie and S, Aishwarya P and Dhamdhere, Kedar and Guo, Meiqi and Gupta, Umang and Eslami, Ali and Schain, Mariano and Blokzijl, Michiel and Welling, David and Orr, Dave and Bolelli, Levent and Perez-Nieves, Nicolas and Sirotenko, Mikhail and Prasad, Aman and Kar, Arjun and Pigem, Borja De Balle and Terzi, Tayfun and Weisz, Gellért and Ghosh, Dipankar and Mavalankar, Aditi and Madeka, Dhruv and Daugaard, Kaspar and Adam, Hartwig and Shah, Viraj and Berman, Dana and Tran, Maggie and Baker, Steven and Andrejczuk, Ewa and Chole, Grishma and Raboshchuk, Ganna and Mirzazadeh, Mahdi and Kagohara, Thais and Wu, Shimu and Schallhart, Christian and Orlando, Bernett and Wang, Chen and Rrustemi, Alban and Xiong, Hao and Liu, Hao and Vezer, Arpi and Ramsden, Nolan and Chang, Shuo-yiin and Mudgal, Sidharth and Li, Yan and Vieillard, Nino and Hoshen, Yedid and Ahmad, Farooq and Slone, Ambrose and Hua, Amy and Potikha, Natan and Rossini, Mirko and Stritar, Jon and Prakash, Sushant and Wang, Zifeng and Dong, Xuanyi and Nazari, Alireza and Nehoran, Efrat and Tekelioglu, Kaan and Li, Yinxiao and Badola, Kartikeya and Funkhouser, Tom and Li, Yuanzhen and Yerram, Varun and Ganeshan, Ramya and Formoso, Daniel and Langner, Karol and Shi, Tian and Li, Huijian and Yamamori, Yumeya and Panda, Amayika and Saade, Alaa and Scarpati, Angelo Scorza and Breaux, Chris and Carey, CJ and Zhou, Zongwei and Hsieh, Cho-Jui and Bridgers, Sophie and Butryna, Alena and Gupta, Nishesh and Tulsyan, Vaibhav and Woo, Sanghyun and Eltyshev, Evgenii and Grathwohl, Will and Parks, Chanel and Benjamin, Seth and Panigrahy, Rina and Dodhia, Shenil and Freitas, Daniel De and Sauer, Chris and Song, Will and Alet, Ferran and Tolins, Jackson and Paduraru, Cosmin and Zhou, Xingyi and Albert, Brian and Zhang, Zizhao and Shu, Lei and Bansal, Mudit and Nguyen, Sarah and Globerson, Amir and Xiao, Owen and Manyika, James and Hennigan, Tom and Rong, Rong and Matak, Josip and Bakalov, Anton and Sharma, Ankur and Sinopalnikov, Danila and Pierson, Andrew and Roller, Stephen and Brown, Geoff and Gao, Mingcen and Fukuzawa, Toshiyuki and Ghafouri, Amin and Vassigh, Kenny and Barr, Iain and Wang, Zhicheng and Korsun, Anna and Jayaram, Rajesh and Ren, Lijie and Zaman, Tim and Khan, Samira and Lunts, Yana and Deutsch, Dan and Uthus, Dave and Katz, Nitzan and Samsikova, Masha and Khalifa, Amr and Sethi, Nikhil and Sun, Jiao and Tang, Luming and Alon, Uri and Luo, Xianghong and Yu, Dian and Nayyar, Abhishek and Petrini, Bryce and Truong, Will and Hellendoorn, Vincent and Chinaev, Nikolai and Alberti, Chris and Wang, Wei and Hu, Jingcao and Mirrokni, Vahab and Balashankar, Ananth and Aharon, Avia and Mehta, Aahil and Iscen, Ahmet and Kready, Joseph and Manning, Lucas and Mohananey, Anhad and Chen, Yuankai and Tripathi, Anshuman and Wu, Allen and Petrovski, Igor and Hwang, Dawsen and Baeuml, Martin and Chandrakaladharan, Shreyas and Liu, Yuan and Coaguila, Rey and Chen, Maxwell and Ma, Sally and Tafti, Pouya and Tatineni, Susheel and Spitz, Terry and Ye, Jiayu and Vicol, Paul and Rosca, Mihaela and Puigdomènech, Adrià and Yahav, Zohar and Ghemawat, Sanjay and Lin, Hanzhao and Kirk, Phoebe and Nabulsi, Zaid and Brin, Sergey and Bohnet, Bernd and Caluwaerts, Ken and Veerubhotla, Aditya Srikanth and Zheng, Dan and Dai, Zihang and Petrov, Petre and Xu, Yichong and Mehran, Ramin and Xu, Zhuo and Zintgraf, Luisa and Choi, Jiho and Hombaiah, Spurthi Amba and Thoppilan, Romal and Reddi, Sashank and Lew, Lukasz and Li, Li and Webster, Kellie and Sawhney, KP and Lamprou, Lampros and Shakeri, Siamak and Lunayach, Mayank and Chen, Jianmin and Bagri, Sumit and Salcianu, Alex and Chen, Ying and Donchev, Yani and Magister, Charlotte and Nørly, Signe and Rodrigues, Vitor and Izo, Tomas and Noga, Hila and Zou, Joe and Köppe, Thomas and Zhou, Wenxuan and Lee, Kenton and Long, Xiangzhu and Eisenbud, Danielle and Chen, Anthony and Schenck, Connor and To, Chi Ming and Zhong, Peilin and Taropa, Emanuel and Truong, Minh and Levy, Omer and Martins, Danilo and Zhang, Zhiyuan and Semturs, Christopher and Zhang, Kelvin and Yakubovich, Alex and Moreno, Pol and McConnaughey, Lara and Lu, Di and Redmond, Sam and Weerts, Lotte and Bitton, Yonatan and Refice, Tiziana and Lacasse, Nicolas and Conmy, Arthur and Tallec, Corentin and Odell, Julian and Forbes-Pollard, Hannah and Socala, Arkadiusz and Hoech, Jonathan and Kohli, Pushmeet and Walton, Alanna and Wang, Rui and Sazanovich, Mikita and Zhu, Kexin and Kapishnikov, Andrei and Galt, Rich and Denton, Matthew and Murdoch, Ben and Sikora, Caitlin and Mohamed, Kareem and Wei, Wei and First, Uri and McConnell, Tim and Cobo, Luis C. and Qin, James and Avrahami, Thi and Balle, Daniel and Watanabe, Yu and Louis, Annie and Kraft, Adam and Ariafar, Setareh and Gu, Yiming and Rives, Eugénie and Yoon, Charles and Rusu, Andrei and Cobon-Kerr, James and Hahn, Chris and Luo, Jiaming and Yuvein and Zhu and Ahuja, Niharika and Benenson, Rodrigo and Kaufman, Raphaël Lopez and Yu, Honglin and Hightower, Lloyd and Zhang, Junlin and Ni, Darren and Hendricks, Lisa Anne and Wang, Gabby and Yona, Gal and Jain, Lalit and Barrio, Pablo and Bhupatiraju, Surya and Velusamy, Siva and Dafoe, Allan and Riedel, Sebastian and Thomas, Tara and Yuan, Zhe and Bellaiche, Mathias and Panthaplackel, Sheena and Kloboves, Klemen and Jauhari, Sarthak and Akbulut, Canfer and Davchev, Todor and Gladchenko, Evgeny and Madras, David and Chuklin, Aleksandr and Hill, Tyrone and Yuan, Quan and Madhavan, Mukundan and Leonhard, Luke and Scandinaro, Dylan and Chen, Qihang and Niu, Ning and Douillard, Arthur and Damoc, Bogdan and Onoe, Yasumasa and Pedregosa, Fabian and Bertsch, Fred and Leichner, Chas and Pagadora, Joseph and Malmaud, Jonathan and Ponda, Sameera and Twigg, Andy and Duzhyi, Oleksii and Shen, Jingwei and Wang, Miaosen and Garg, Roopal and Chen, Jing and Evci, Utku and Lee, Jonathan and Liu, Leon and Kojima, Koji and Yamaguchi, Masa and Rajendran, Arunkumar and Piergiovanni, AJ and Rajendran, Vinodh Kumar and Fornoni, Marco and Ibagon, Gabriel and Ragan, Harry and Khan, Sadh MNM and Blitzer, John and Bunner, Andrew and Sun, Guan and Kosakai, Takahiro and Lundberg, Scott and Elue, Ndidi and Guu, Kelvin and Park, SK and Park, Jane and Narayanaswamy, Arunachalam and Wu, Chengda and Mudigonda, Jayaram and Cohn, Trevor and Mu, Hairong and Kumar, Ravi and Graesser, Laura and Zhang, Yichi and Killam, Richard and Zhuang, Vincent and Giménez, Mai and Jishi, Wael Al and Ley-Wild, Ruy and Zhai, Alex and Osawa, Kazuki and Cedillo, Diego and Liu, Jialu and Upadhyay, Mayank and Sieniek, Marcin and Sharma, Roshan and Paine, Tom and Angelova, Anelia and Addepalli, Sravanti and Parada, Carolina and Majumder, Kingshuk and Lamp, Avery and Kumar, Sanjiv and Deng, Xiang and Myaskovsky, Artiom and Sabolić, Tea and Dudek, Jeffrey and York, Sarah and Quitry, Félix de Chaumont and Nie, Jiazhong and Cattle, Dee and Gunjan, Alok and Piot, Bilal and Khawaja, Waleed and Bang, Seojin and Wang, Simon and Khodadadeh, Siavash and R, Raghavender and Rawlani, Praynaa and Powell, Richard and Lee, Kevin and Griesser, Johannes and Oh, GS and Magalhaes, Cesar and Li, Yujia and Tokumine, Simon and Vogel, Hadas Natalie and Hsu, Dennis and BC, Arturo and Jindal, Disha and Cohen, Matan and Yang, Zi and Yuan, Junwei and Cesare, Dario de and Bruguier, Tony and Xu, Jun and Roy, Monica and Jacovi, Alon and Belov, Dan and Arya, Rahul and Meadowlark, Phoenix and Cohen-Ganor, Shlomi and Ye, Wenting and Morris-Suzuki, Patrick and Banzal, Praseem and Song, Gan and Ponnuramu, Pranavaraj and Zhang, Fred and Scrivener, George and Zaiem, Salah and Rochman, Alif Raditya and Han, Kehang and Ghazi, Badih and Lee, Kate and Drath, Shahar and Suo, Daniel and Girgis, Antonious and Shenoy, Pradeep and Nguyen, Duy and Eck, Douglas and Gupta, Somit and Yan, Le and Carreira, Joao and Gulati, Anmol and Sang, Ruoxin and Mirylenka, Daniil and Cooney, Emma and Chou, Edward and Ling, Mingyang and Fan, Cindy and Coleman, Ben and Tubone, Guilherme and Kumar, Ravin and Baldridge, Jason and Hernandez-Campos, Felix and Lazaridou, Angeliki and Besley, James and Yona, Itay and Bulut, Neslihan and Wellens, Quentin and Pierigiovanni, AJ and George, Jasmine and Green, Richard and Han, Pu and Tao, Connie and Clark, Geoff and You, Chong and Abdolmaleki, Abbas and Fu, Justin and Chen, Tongzhou and Chaugule, Ashwin and Chandorkar, Angad and Rahman, Altaf and Thompson, Will and Koanantakool, Penporn and Bernico, Mike and Ren, Jie and Vlasov, Andrey and Vassilvitskii, Sergei and Kula, Maciej and Liang, Yizhong and Kim, Dahun and Huang, Yangsibo and Ye, Chengxi and Lepikhin, Dmitry and Helmholz, Wesley},
year = {2025},
doi = {10.48550/arxiv.2507.06261},
eprint = {2507.06261},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.2507.06261}
}References
- R. Anil, G. Pereyra, A. Passos, R. Ormandi, G. E. Dahl, and G. E. Hinton. Large scale distributed neural network training through online distillation, 2018. URL https://arxiv.org/abs/1804 .03235 . ↗
- R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, et al. PaLM 2 technical report, 2023. URL https://arxiv.org/abs/2305.10403 . ↗ our card
- Anthropic. Claude’s extended thinking, 2025. URL https://www.anthropic.com/research/v isible-extended-thinking . ↗
- A. Baddepudi, A. Yang, and M. Lučić. Advancing the frontier of video understanding with Gemini 2.5, 2025. URL https://developers.googleblog.com/en/gemini-2-5-video-understan ding/ . ↗
- Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, et al. Constitutional ai: Harmlessness from ai feedback, 2022. URL https://arxiv.org/abs/2212.08073 . ↗ our card
- M. Balunović, J. Dekoninck, I. Petrov, N. Jovanović, and M. Vechev. Matharena: Evaluating llms on uncontaminated math competitions, 2025. URL https://arxiv.org/abs/2505.23281 . ↗
- P. Barham, A. Chowdhery, J. Dean, S. Ghemawat, S. Hand, D. Hurt, M. Isard, H. Lim, R. Pang, S. Roy, et al. Pathways: Asynchronous distributed dataflow for ml. Proceedings of Machine Learning and Systems, 4:430–449, 2022. URL https://proceedings.mlr.press/v162/barham22a.html . ↗
- A. Beutel, K. Xiao, J. Heidecke, and L. Weng. Diverse and effective red teaming with auto-generated rewards and multi-step reinforcement learning, 2024. URL https://arxiv.org/abs/2412.1 8693 . ↗
- S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, et al. Pythia: A suite for analyzing large language models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, 2023. URL https://proceedings.mlr.press/v202/biderman23a.h tml . ↗
- N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang. Quantifying memorization across neural language models. In 2022 IEEE Symposium on Security and Privacy (SP), pages 1113–1130, 2022. URL https://arxiv.org/abs/2202.07646 . ↗
- W.-L. Chiang, L. Zheng, Y. Sheng, A. N. Angelopoulos, T. Li, D. Li, B. Zhu, H. Zhang, M. Jordan, J. E. Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning, 2024. URL https://arxiv.org/abs/ 2306.05685 . ↗
- A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022. URL https://arxiv.org/abs/2204.02311 . ↗
- N. Chowdhury, J. Aung, C. J. Shern, O. Jaffe, D. Sherburn, G. Starace, E. Mays, R. Dias, M. Aljubeh, M. Glaese, C. E. Jimenez, J. Yang, L. Ho, T. Patwardhan, K. Liu, and A. Madry. Introducing SWE-bench verified, 2024. URL https://openai.com/index/introducing-swe-bench-verif ied/ . ↗
- A. Clark, D. de las Casas, A. Guy, A. Mensch, M. Paganini, J. Hoffmann, B. Damoc, B. Hechtman, T. Cai, S. Borgeaud, G. van den Driessche, E. Rutherford, T. Hennigan, M. Johnson, K. Millican, A. Cassirer, C. Jones, E. Buchatskaya, D. Budden, L. Sifre, S. Osindero, O. Vinyals, J. Rae, E. Elsen, K. Kav… — [arXiv:2202.01169](https://arxiv.org/abs/2202.01169) find
- CodeGemma Team, H. Zhao, J. Hui, J. Howland, N. Nguyen, S. Zuo, A. Hu, C. A. Choquette-Choo, J. Shen, J. Kelley, K. Bansal, L. Vilnis, M. Wirth, P. Michel, P. Choy, P. Joshi, R. Kumar, S. Hashmi, S. Agrawal, Z. Gong, J. Fine, T. Warkentin, A. J. Hartman, B. Ni, K. Korevec, K. Schaefer, and S. Huffma… find
- A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna. Fleurs: Few-shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. find our card
- M. Dehghani, J. Djolonga, B. Mustafa, P. Padlewski, J. Heek, J. Gilmer, A. P. Steiner, M. Caron, R. Geirhos, I. Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning, pages 7480–7512. PMLR, 2023. URL https: //proceedings.mlr.press… find
- T. Doshi. Build rich, interactive web apps with an updated Gemini 2.5 Pro, 2025a. URL https: //blog.google/products/gemini/gemini-2-5-pro-updates/ . find
- T. Doshi. Gemini 2.5: Our most intelligent models are getting even better, 2025b. URL https: //blog.google/technology/google-deepmind/google-gemini-updates-io-2025/ . find
- N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al. GLaM: Efficient scaling of language models with mixture-of-experts. arXiv preprint arXiv:2112.06905, 2021. URL https://arxiv.org/abs/2112.06905 . ↗ our card
- W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021. URL https://arxiv.org/ abs/2101.03961 . ↗
- C. Fu, Y. Dai, Y. Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24108–24118, 2025. URL https… find our card
- P. Gauthier. Aider Polyglot Coding Leaderboard, 2025. URL https://aider.chat/docs/leader boards/ . ↗
- Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. URL https://arxiv.org/abs/2403.05530 . ↗ our card
- Gemini Team, Google. Gemini Deep Research, 2024. URL https://gemini.google/overview /deep-research/ . ↗
- Gemma Team. Gemma: Open Models Based on Gemini Research and Technology, 2024. URL https://arxiv.org/abs/2403.08295 . ↗
- O. Goldman, U. Shaham, D. Malkin, S. Eiger, A. Hassidim, Y. Matias, J. Maynez, A. M. Gilady, J. Riesa, S. Rijhwani, L. Rimell, I. Szpektor, R. Tsarfaty, and M. Eyal. Eclektic: a novel challenge set for evaluation of cross-lingual knowledge transfer, 2025. URL https://arxiv.org/abs/2502.2 1228 . ↗
- Google DeepMind. Frontier safety framework, February 2025a. URL https://deepmind.google/ discover/governance/frontier-safety-framework/ . ↗
- Google DeepMind. Gemini 2.0 Flash-Lite, 2025b. URL https://deepmind.google/models/gem ini/flash-lite/ . ↗
- A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, et al. The Llama 3 Herd of Models, 2024. URL https://arxiv.org/abs/2407.21783 . ↗
- D. Hassabis. Our vision for building a universal AI assistant, 2025. URL https://blog.google/ technology/google-deepmind/gemini-universal-ai-assistant/ . ↗
- G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531 . ↗ our card
- K. Hu, P. Wu, F. Pu, W. Xiao, Y. Zhang, X. Yue, B. Li, and Z. Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos, 2025. URL https://arxiv.org/abs/25 01.13826 . ↗
- S. Hughes, M. Bae, and M. Li. Vectara Hallucination Leaderboard, nov 2023. URL https://github .com/vectara/hallucination-leaderboard . find
- D. Ippolito, F. Tramer, M. Nasr, C. Zhang, M. Jagielski, K. Lee, C. A. Choquette-Choo, and N. Carlini. Preventing verbatim memorization in language models gives a false sense of privacy, 2022. URL https://arxiv.org/abs/2210.17546 . — [arXiv:2210.17546](https://arxiv.org/abs/2210.17546) ↗
- A. Jacovi, A. Wang, C. Alberti, C. Tao, J. Lipovetz, K. Olszewska, L. Haas, M. Liu, N. Keating, A. Bloniarz, C. Saroufim, C. Fry, D. Marcus, D. Kukliansky, G. S. Tomar, J. Swirhun, J. Xing, L. Wang, M. Gurumurthy, M. Aaron, M. Ambar, R. Fellinger, R. Wang, R. Sims, Z. Zhang, S. Goldshtein, and D. Da… find
- A. Jacovi, A. Wang, C. Alberti, C. Tao, J. Lipovetz, K. Olszewska, L. Haas, M. Liu, N. Keating, A. Bloniarz, et al. The facts grounding leaderboard: Benchmarking llms’ ability to ground responses to long-form input. arXiv preprint arXiv:2501.03200, 2025. URL https://arxiv.org/abs/2501.03200 . ↗
- N. Jain, K. Han, A. Gu, W.-D. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. URL https://arxiv.org/abs/2403.07974 . ↗ our card
- A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. URL https://arxiv.org/abs/2401.04088 . ↗
- C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=VTF8yNQM66 . ↗ our card
- K. Kampf and N. Brichtova. Experiment with Gemini 2.0 Flash native image generation, 2025. URL https://developers.googleblog.com/en/experiment-with-gemini-20-flash-nat ive-image-generation/ . ↗
- K. Kavukcuoglu. Gemini 2.0 is now available to everyone, 2025. URL https://blog.google/te chnology/google-deepmind/gemini-model-updates-february-2025 . ↗
- L. Kilpatrick. Gemini 2.5 Pro Preview: even better coding performance, 2025. URL https://deve lopers.googleblog.com/en/gemini-2-5-pro-io-improved-coding-performance . find
- S. Kudugunta, I. Caswell, B. Zhang, X. Garcia, C. A. Choquette-Choo, K. Lee, D. Xin, A. Kusupati, R. Stella, A. Bapna, and O. Firat. MADLAD-400: A Multilingual And Document-Level Large Audited Dataset, 2023. URL https://arxiv.org/abs/2309.04662 . ↗ our card
- J. M. Laurent, J. D. Janizek, M. Ruzo, M. M. Hinks, M. J. Hammerling, S. Narayanan, et al. LAB-Bench: Measuring capabilities of language models for biology research, 2024. URL https: //arxiv.org/abs/2407.10362 . find
- LearnLM Team. Evaluating Gemini in an Arena for Learning, 2025. URL https://goo.gle/Lear nLM-May25 . ↗
- J. Lee, A. Chen, Z. Dai, D. Dua, D. S. Sachan, M. Boratko, Y. Luan, S. M. Arnold, V. Perot, S. Dalmia, et al. Can long-context language models subsume retrieval, rag, sql, and more? arXiv preprint arXiv:2406.13121, 2024. URL https://arxiv.org/abs/2406.13121 . ↗
- J. Lei, T. L. Berg, and M. Bansal. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems, 34:11846–11858, 2021. find
- D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen. GShard: Scaling giant models with conditional computation and automatic sharding. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=qr we7XHTmYb . ↗
- N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, et al. The WMDP benchmark: Measuring and reducing malicious use with unlearning, 2024. URL https://arxiv.org/abs/2403.03218 . ↗
- L. Liu, X. Liu, J. Gao, W. Chen, and J. Han. Understanding the difficulty of training transformers. arXiv preprint arXiv:2004.08249, 2020. URL https://arxiv.org/abs/2004.08249 . ↗
- LMArena Team. Webdev arena, 2025. URL https://web.lmarena.ai/leaderboard . ↗
- S. B. Mallick and L. Kilpatrick. Gemini 2.0: Flash, Flash-Lite and Pro, 2025. URL https://develo pers.googleblog.com/en/gemini-2-family-expands/ . find
- A. Mehrotra, M. Zampetakis, P. Kassianik, B. Nelson, H. Anderson, Y. Singer, and A. Karbasi. Tree of attacks: Jailbreaking black-box llms automatically, 2024. URL https://arxiv.org/abs/2312 .02119 . ↗ our card
- I. Molybog, P. Albert, M. Chen, Z. DeVito, D. Esiobu, N. Goyal, P. Koura, S. Narang, A. Poulton, R. Silva, et al. A theory on adam instability in large-scale machine learning. arXiv preprint arXiv:2304.09871, 2023. URL https://arxiv.org/abs/2304.09871 . ↗
- A. Nagrani, S. Menon, A. Iscen, S. Buch, R. Mehran, N. Jha, A. Hauth, Y. Zhu, C. Vondrick, M. Sirotenko, C. Schmid, and T. Weyand. Minerva: Evaluating complex video reasoning, 2025a. URL https: //arxiv.org/abs/2505.00681 . — [arXiv:2505.00681](https://arxiv.org/abs/2505.00681) find
- A. Nagrani, M. Zhang, R. Mehran, R. Hornung, N. B. Gundavarapu, N. Jha, A. Myers, X. Zhou, B. Gong, C. Schmid, M. Sirotenko, Y. Zhu, and T. Weyand. Neptune: The long orbit to benchmarking long video understanding, 2025b. URL https://arxiv.org/abs/2412.09582 . — [arXiv:2412.09582](https://arxiv.org/abs/2412.09582) ↗
- M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee. Scalable extraction of training data from (production) language models, 2023. URL https://arxiv.org/abs/2311.17035 . ↗ our card
- P. Padlewski, M. Bain, M. Henderson, Z. Zhu, N. Relan, H. Pham, D. Ong, K. Aleksiev, A. Ormazabal, S. Phua, E. Yeo, E. Lamprecht, Q. Liu, Y. Wang, E. Chen, D. Fu, L. Li, C. Zheng, C. de Masson d’Autume, D. Yogatama, M. Artetxe, and Y. Tay. Vibe-eval: A hard evaluation suite for measuring progress of… find
- A. Pappu, B. Porter, I. Shumailov, and J. Hayes. Measuring memorization in RLHF for code completion. arXiv preprint arXiv:2406.11715, 2024. URL https://arxiv.org/abs/2406.11715 . — [arXiv:2406.11715](https://arxiv.org/abs/2406.11715) ↗
- V. Patraucean, L. Smaira, A. Gupta, A. Recasens, L. Markeeva, D. Banarse, S. Koppula, M. Malinowski, Y. Yang, C. Doersch, et al. Perception test: A diagnostic benchmark for multimodal video models. Advances in Neural Information Processing Systems, 36:42748–42761, 2023. find
- E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving. Red teaming language models with language models, 2022. URL https://arxiv.org/abs/2202 .03286 . ↗ our card
- L. Phan et al. Humanity’s last exam, 2025. URL https://arxiv.org/abs/2501.14249 . ↗ our card
- M. Phuong, M. Aitchison, E. Catt, S. Cogan, A. Kaskasoli, V. Krakovna, D. Lindner, M. Rahtz, Y. Assael, S. Hodkinson, et al. Evaluating frontier models for dangerous capabilities, 2024. URL https: //arxiv.org/abs/2403.13793 . find
- M. Phuong, R. S. Zimmermann, Z. Wang, D. Lindner, V. Krakovna, S. Cogan, A. Dafoe, L. Ho, and R. Shah. Evaluating frontier models for stealth and situational awareness, 2025. URL https: //arxiv.org/abs/2505.01420 . — [arXiv:2505.01420](https://arxiv.org/abs/2505.01420) find
- S. Pichai. Google I/O 2025: From research to reality, 2025. URL https://blog.google/techno logy/ai/io-2025-keynote/ . ↗
- C. Plizzari, A. Tonioni, Y. Xian, A. Kulshrestha, and F. Tombari. Omnia de egotempo: Benchmarking temporal understanding of multi-modal llms in egocentric videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24129–24138, 2025. find
- D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman. Gqqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. find
- C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. S. Pinto, D. Keysers, and N. Houlsby. Scaling vision with sparse mixture of experts, 2021. URL https://arxiv.org/abs/ 2106.05974 . ↗
- J. Roberts, M. R. Taesiri, A. Sharma, A. Gupta, S. Roberts, I. Croitoru, S.-V. Bogolin, J. Tang, F. Langer, V. Raina, et al. ZeroBench: An impossible visual benchmark for contemporary large multimodal models. arXiv preprint arXiv:2502.09696, 2025. — [arXiv:2502.09696](https://arxiv.org/abs/2502.09696) find
- M. Rodriguez, R. A. Popa, L. Liang, A. Wang, M. Rahtz, A. Kaskasoli, A. Dafoe, and F. Flynn. A framework for evaluating emerging cyberattack capabilities of AI, 2025. URL https://arxiv. org/abs/2503.11917 . — [arXiv:2503.11917](https://arxiv.org/abs/2503.11917) find
- S. Roller, S. Sukhbaatar, J. Weston, et al. Hash layers for large sparse models. Advances in Neural Information Processing Systems, 34:17555–17566, 2021. URL https://proceedings.neurips. cc/paper/2021/file/883e881bc596359e0c5112411858a74b-Paper.pdf . ↗
- M. Samvelyan, S. C. Raparthy, A. Lupu, E. Hambro, A. H. Markosyan, M. Bhatt, Y. Mao, M. Jiang, J. Parker-Holder, J. Foerster, T. Rocktäschel, and R. Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024. URL https://arxiv.org/abs/2402.16822 . — [arXiv:2402.16822](https://arxiv.org/abs/2402.16822) ↗
- R. Shah, A. Irpan, A. M. Turner, A. Wang, A. Conmy, D. Lindner, J. Brown-Cohen, L. Ho, N. Nanda, R. A. Popa, R. Jain, R. Greig, S. Albanie, S. Emmons, S. Farquhar, S. Krier, S. Rajamanoharan, S. Bridgers, T. Ijitoye, T. Everitt, V. Krakovna, V. Varma, V. Mikulik, Z. Kenton, D. Orr, S. Legg, N. Goodm… — [arXiv:2504.01849](https://arxiv.org/abs/2504.01849) find
- D. Sharon. Upload and edit your images directly in the Gemini app, 2025. URL https://blog.goo gle/products/gemini/image-editing/ . ↗
- N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In ICLR (Poster). OpenReview.net, 2017. URL https://arxiv.org/abs/1701.06538 . ↗
- C. Shi, S. Lin, S. Song, J. Hayes, I. Shumailov, I. Yona, J. Pluto, A. Pappu, C. A. Choquette-Choo, M. Nasr, C. Sitawarin, G. Gibson, A. Terzis, and J. F. Flynn. Lessons from defending gemini against indirect prompt injections, 2025. URL https://arxiv.org/abs/2505.14534 . — [arXiv:2505.14534](https://arxiv.org/abs/2505.14534) ↗
- S. Singh, A. Romanou, C. Fourrier, D. I. Adelani, J. G. Ngui, D. Vila-Suero, P. Limkonchotiwat, K. Marchisio, W. Q. Leong, Y. Susanto, R. Ng, S. Longpre, W.-Y. Ko, M. Smith, A. Bosselut, A. Oh, A. F. T. Martins, L. Choshen, D. Ippolito, E. Ferrante, M. Fadaee, B. Ermis, and S. Hooker. Global mmlu: U… find our card
- R. Stein. Expanding AI Overviews and introducing AI Mode, 2025. URL https://blog.google/ products/search/ai-mode-search . ↗
- I. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit, M. Lucic, and A. Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision, 2021. find
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Cu… find
- K. Vodrahalli, S. Ontanon, N. Tripuraneni, K. Xu, S. Jain, R. Shivanna, J. Hui, N. Dikkala, M. Kazemi, B. Fatemi, et al. Michelangelo: Long context evaluations beyond haystacks via latent structure queries. arXiv preprint arXiv:2409.12640, 2024. URL https://arxiv.org/abs/2409.12640 . ↗
- B. Wang. NotebookLM now lets you listen to a conversation about your sources , 2024. URL https://blog.google/technology/ai/notebooklm-audio-overviews . ↗
- C. Wang, A. Wu, and J. Pino. Covost 2: A massively multilingual speech-to-text translation corpus, 2020. find
- W. Wang, Z. He, W. Hong, Y. Cheng, X. Zhang, J. Qi, X. Gu, S. Huang, B. Xu, Y. Dong, M. Ding, and J. Tang. Lvbench: An extreme long video understanding benchmark, 2024. URL https: //arxiv.org/abs/2406.08035 . find
- X. Wang, J. Wu, J. Chen, L. Li, Y.-F. Wang, and W. Y. Wang. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4581–4591, 2019. find
- J. Wei, K. Nguyen, H. W. Chung, Y. J. Jiao, S. Papay, A. Glaese, J. Schulman, and W. Fedus. Measuring short-form factuality in large language models. arXiv preprint arXiv:2411.04368, 2024. URL https://arxiv.org/abs/2411.04368 . ↗ our card
- L. Weidinger, J. Barnhart, J. Brennan, C. Butterfield, S. Young, W. Hawkins, et al. Holistic safety and responsibility evaluations of advanced ai models, 2024. URL https://arxiv.org/abs/2404.1 4068 . ↗
- H. Wijk, T. Lin, J. Becker, S. Jawhar, N. Parikh, T. Broadley, L. Chan, M. Chen, J. Clymer, J. Dhyani, et al. RE-Bench: Evaluating frontier ai r&d capabilities of language model agents against human experts, 2025. URL https://arxiv.org/abs/2411.15114 . ↗
- M. Wortsman, P. J. Liu, L. Xiao, K. Everett, A. Alemi, B. Adlam, J. D. Co-Reyes, I. Gur, A. Kumar, R. Novak, et al. Small-scale proxies for large-scale transformer training instabilities. arXiv preprint arXiv:2309.14322, 2023. URL https://arxiv.org/abs/2309.14322 . ↗
- J. Yang, A. Prabhakar, K. Narasimhan, and S. Yao. Intercode: Standardizing and benchmarking interactive coding with execution feedback, 2023. URL https://arxiv.org/abs/2306.14898 . ↗
- Z. Yu, D. Xu, J. Yu, T. Yu, Z. Zhao, Y. Zhuang, and D. Tao. ActivityNet-QA: A dataset for understanding complex web videos via question answering. In AAAI, 2019. find our card
- X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567, 2… find our card
- Zerokid. Pokemon Red Version - Guide and Walkthrough (GB), 2024. URL https://gamefaqs.g amespot.com/gameboy/367023-Pokémon-red-version/faqs/64175 . ↗
- S. Zhai, T. Likhomanenko, E. Littwin, D. Busbridge, J. Ramapuram, Y. Zhang, J. Gu, and J. M. Susskind. Stabilizing transformer training by preventing attention entropy collapse. In International Conference on Machine Learning, pages 40770–40803. PMLR, 2023. URL https://proceedings.mlr.pr ess/v202/zh… ↗
- J. Zhang. Gemini Plays Pokemon Twitch Stream, 2025. URL https://www.twitch.tv/gemini \_plays\_pokemon/about . ↗
- S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. URL https://arxiv.org/abs/2205.01068 . ↗
- L. Zhou, C. Xu, and J. J. Corso. Towards automatic learning of procedures from web instructional videos. In AAAI Conference on Artificial Intelligence, pages 7590–7598, 2018. URL https://www. aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17344 . \#\# 7. Contributors and Acknowledgments Contributors Gh… find