🧑‍🔬 Who cites Peter Hawkins

540 citations of their works from 531 papers of our library (of the 149,439 whose bibliographies we read) · 65 with a page here, the most cited first · ← Peter Hawkins

1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsIEEE/CVF International Conference on Computer Vision (ICCV)Ze Liu, Yutong Lin, Yue Cao, et al.2021 · 33,968 citations
2. Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerarXiv:1910.10683Colin Raffel, Noam Shazeer, Adam Roberts, et al.2019 · 11,325 citations
3. PaLM: Scaling Language Modeling with PathwaysarXiv:2204.02311Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.2022 · 4,753 citations
4. Scaling Laws for Neural Language ModelsarXiv:2001.08361Jared Kaplan, Sam McCandlish, Tom Henighan, et al.2020 · 3,434 citations
5. On the Opportunities and Risks of Foundation ModelsarXiv:2108.07258Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.2021 · 3,160 citations
6. RoFormer: Enhanced Transformer with Rotary Position EmbeddingarXiv:2104.09864Jianlin Su, Yu Lu, Shengfeng Pan, et al.2021 · 3,028 citations
7. ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsarXiv:1909.11942Zhenzhong Lan, Mingda Chen, Sebastian Goodman, et al.2019 · 2,806 citations
8. Efficient Transformers: A SurveyACM Computing SurveysYi Tay, Mostafa Dehghani, Dara Bahri, et al.2020 · 1,539 citations
9. FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningarXiv:2307.08691Tri Dao2023 · 1,240 citations
10. Reformer: The Efficient TransformerarXiv:2001.04451Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya2020 · 829 citations
11. FNet: Mixing Tokens with Fourier TransformsConference of the North American Chapter of the Association for Computational Linguistics (NAACL)James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.2022 · 597 citations
12. AudioLM: a Language Modeling Approach to Audio GenerationIEEE/ACM Transactions on Audio Speech and Language ProcessingZalan Borsos, Raphael Marinier, Damien Vincent, et al.2022 · 581 citations
13. Massively Multilingual Neural Machine Translation in the Wild: Findings and ChallengesarXiv:1907.05019Naveen Arivazhagan, Ankur Bapna, Orhan Firat, et al.2019 · 564 citations
14. Simple, Scalable Adaptation for Neural Machine TranslationConference on Empirical Methods in Natural Language Processing (EMNLP)Ankur Bapna, Orhan Firat2019 · 530 citations
15. GLaM: Efficient Scaling of Language Models with Mixture-of-ExpertsarXiv:2112.06905Nan Du, Yanping Huang, Andrew M. Dai, et al.2021 · 496 citations
16. The Falcon Series of Open Language ModelsarXiv:2311.16867Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, et al.2023 · 443 citations
17. Long Range Arena: A Benchmark for Efficient TransformersarXiv:2011.04006Yi Tay, Mostafa Dehghani, Samira Abnar, et al.2020 · 414 citations
18. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionConference on Neural Information Processing Systems (NeurIPS)Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.2024 · 400 citations
19. LongT5: Efficient Text-To-Text Transformer for Long SequencesFindings of the Association for Computational LinguisticsMandy Guo, Joshua Ainslie, David Uthus, et al.2022 · 399 citations
20. PyTorch Distributed: Experiences on Accelerating Data Parallel TrainingarXiv:2006.15704Shen Li, Yanli Zhao, Rohan Varma, et al.2020 · 348 citations
21. Efficient Attentions for Long Document SummarizationConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Luyang Huang, Shuyang Cao, Nikolaus Parulian, et al.2021 · 336 citations
22. ZeRO-Offload: Democratizing Billion-Scale Model TrainingarXiv:2101.06840Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, et al.2021 · 309 citations
23. Socratic Models: Composing Zero-Shot Multimodal Reasoning with LanguagearXiv:2204.00598Andy Zeng, Maria Attarian, Brian Ichter, et al.2022 · 302 citations
24. Challenges and Applications of Large Language ModelsarXiv:2307.10169Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.2023 · 301 citations
25. Canine: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics (TACL)Jonathan H. Clark, Dan Garrette, Iulia Turc, et al.2022 · 301 citations
26. Reducing Activation Recomputation in Large Transformer ModelsarXiv:2205.05198Vijay Korthikanti, Jared Casper, Sangkug Lym, et al.2022 · 273 citations
27. Perceiver: General Perception with Iterative AttentionarXiv:2103.03206Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.2021 · 269 citations
28. Transformers learn in-context by gradient descentarXiv:2212.07677Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, et al.2022 · 247 citations
29. Charformer: Fast Character Transformers via Gradient-based Subword TokenizationarXiv:2106.12672Yi Tay, Vinh Q. Tran, Sebastian Ruder, et al.2021 · 186 citations
30. Adapting Language Models to Compress ContextsConference on Empirical Methods in Natural Language Processing (EMNLP)Alexis Chevalier, Alexander Wettig, Anirudh Ajith, et al.2023 · 182 citations
31. SCROLLS: Standardized CompaRison Over Long Language SequencesConference on Empirical Methods in Natural Language Processing (EMNLP)Uri Shaham, Elad Segal, Maor Ivgi, et al.2022 · 165 citations
32. PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel ComputationarXiv:2104.12369Wei Zeng, Xiaozhe Ren, Teng Su, et al.2021 · 161 citations
33. Efficient Methods for Natural Language Processing: A SurveyTransactions of the Association for Computational Linguistics (TACL)Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.2023 · 145 citations
34. Rethinking embedding coupling in pre-trained language modelsarXiv:2010.12821Hyung Won Chung, Thibault Févry, Henry Tsai, et al.2020 · 142 citations
35. A Length-Extrapolatable TransformerAnnual Meeting of the Association for Computational Linguistics (ACL)Yutao Sun, Li Dong, Barun Patra, et al.2023 · 118 citations
36. Learning from Task DescriptionsConference on Empirical Methods in Natural Language Processing (EMNLP)Orion Weller, Nicholas Lourie, Matt Gardner, et al.2020 · 112 citations
37. Do Long-Range Language Models Actually Use Long-Range Context?Conference on Empirical Methods in Natural Language Processing (EMNLP)Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, et al.2021 · 101 citations
38. How to Train Long-Context Language Models (Effectively)Annual Meeting of the Association for Computational Linguistics (ACL)Tianyu Gao, Alexander Wettig, Howard Yen, et al.2025 · 89 citations
39. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?Findings of the Association for Computational LinguisticsYi Tay, Mostafa Dehghani, Samira Abnar, et al.2023 · 86 citations
40. MoBA: Mixture of Block Attention for Long-Context LLMsConference on Neural Information Processing Systems (NeurIPS)Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.2025 · 80 citations
41. Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text SummarizationComputational LinguisticsMd Tahmid Rahman Laskar, Enamul Hoque, Jimmy Xiangji Huang2022 · 76 citations
42. Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction TuningarXiv:2309.05444Ted Zadouri, Ahmet Ustun, Arash Ahmadian, et al.2023 · 69 citations
43. The Devil in Linear TransformerConference on Empirical Methods in Natural Language Processing (EMNLP)Zhen Qin, Xiaodong Han, Weixuan Sun, et al.2022 · 67 citations
44. MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning AttentionarXiv:2506.13585MiniMax, Aili Chen, Aonian Li, et al.2025 · 66 citations
45. A Comprehensive Survey on Long Context Language ModelingarXiv:2503.17407Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.2025 · 64 citations
46. ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationarXiv:2112.12731Shuohuan Wang, Yu Sun, Yang Xiang, et al.2021 · 63 citations
47. Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language ModelsarXiv:2402.02244Xindi Wang, Mahsa Salmani, Parsa Omidi, et al.2024 · 60 citations
48. Simple linear attention language models balance the recall-throughput tradeoffarXiv:2402.18668Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.2024 · 59 citations
49. Scaling Laws of RoPE-based ExtrapolationarXiv:2310.05209Xiaoran Liu, Hang Yan, Shuo Zhang, et al.2023 · 54 citations
50. Long-Context Language Modeling with Parallel Context EncodingAnnual Meeting of the Association for Computational Linguistics (ACL)Howard Yen, Tianyu Gao, Danqi Chen2024 · 44 citations
51. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormerarXiv:2307.14995Zhen Qin, Dong Li, Weigao Sun, et al.2023 · 33 citations
52. TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked TextConference on Empirical Methods in Natural Language Processing (EMNLP)Songshuo Lu, Hua Wang, Yutian Rong, et al.2025 · 32 citations
53. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language ModelsarXiv:2401.04658Zhen Qin, Weigao Sun, Dong Li, et al.2024 · 31 citations
54. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language ProcessingConference on Empirical Methods in Natural Language Processing (EMNLP)Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 27 citations
55. Block-State TransformersarXiv:2306.09539Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.2023 · 20 citations
56. Linearizing Large Language ModelsarXiv:2405.06640Jean Mercat, Igor Vasiljevic, Sedrick Keh, et al.2024 · 16 citations
57. Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Zhisong Zhang, Yan Wang, Xinting Huang, et al.2025 · 14 citations
58. PanGu-π: Enhancing Language Model Architectures via Nonlinearity CompensationarXiv:2312.17276Yunhe Wang, Hanting Chen, Yehui Tang, et al.2023 · 10 citations
59. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language UnderstandingarXiv:2205.10687Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 9 citations
60. Just read twice: closing the recall gap for recurrent language modelsarXiv:2407.05483Simran Arora; Aman Timalsina; Aaryan Singhal; Benjamin Spector; Sabri Eyuboglu; Xinyi Zhao; Ashish Rao; Atri Rudra;, Christopher Re2024 · 8 citations
61. An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with DiacriticsarXiv:2208.11484Aly Mostafa, Omar Mohamed, Ali Ashraf, et al.2022 · 3 citations
62. Cross-layer Attention Sharing for Pre-trained Large Language ModelsTransactions of the Association for Computational Linguistics (TACL)Yongyu Mu, Yuzhang Wu, Yuchun Fan, et al.2026 · 2 citations
63. MiniMax Sparse AttentionarXiv:2606.13392Xunhao Lai, Weiqi Xu, Yufeng Yang, et al.2026 · 2 citations
64. From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code CompletionAnnual Meeting of the Association for Computational Linguistics (ACL)Yanli Wang, Yanlin Wang, Bowen Zhang, et al.2026
65. SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language ModelsJoint Conference on Lexical and Computational Semantics (*SEM)Dong Liu, Yanxuan Yu2026

466 more citing papers of our library have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/peter-hawkins/cited-by/