🔗 Who cites «Rethinking Attention with Performers»

387 papers of our library cite it (of the 149,439 whose bibliographies we read) · 49 with a page here, the most cited first · ← back to «Rethinking Attention with Performers»

1. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsIEEE/CVF International Conference on Computer Vision (ICCV)Ze Liu, Yutong Lin, Yue Cao, et al.2021 · 33,968 citations
2. PaLM: Scaling Language Modeling with PathwaysarXiv:2204.02311Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.2022 · 4,753 citations
3. On the Opportunities and Risks of Foundation ModelsarXiv:2108.07258Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.2021 · 3,160 citations
4. RoFormer: Enhanced Transformer with Rotary Position EmbeddingarXiv:2104.09864Jianlin Su, Yu Lu, Shengfeng Pan, et al.2021 · 3,028 citations
5. Efficient Transformers: A SurveyACM Computing SurveysYi Tay, Mostafa Dehghani, Dara Bahri, et al.2020 · 1,539 citations
6. FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningarXiv:2307.08691Tri Dao2023 · 1,240 citations
7. FNet: Mixing Tokens with Fourier TransformsConference of the North American Chapter of the Association for Computational Linguistics (NAACL)James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.2022 · 597 citations
8. AudioLM: a Language Modeling Approach to Audio GenerationIEEE/ACM Transactions on Audio Speech and Language ProcessingZalan Borsos, Raphael Marinier, Damien Vincent, et al.2022 · 581 citations
9. Long Range Arena: A Benchmark for Efficient TransformersarXiv:2011.04006Yi Tay, Mostafa Dehghani, Samira Abnar, et al.2020 · 414 citations
10. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionConference on Neural Information Processing Systems (NeurIPS)Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.2024 · 400 citations
11. LongT5: Efficient Text-To-Text Transformer for Long SequencesFindings of the Association for Computational LinguisticsMandy Guo, Joshua Ainslie, David Uthus, et al.2022 · 399 citations
12. Efficient Attentions for Long Document SummarizationConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Luyang Huang, Shuyang Cao, Nikolaus Parulian, et al.2021 · 336 citations
13. Socratic Models: Composing Zero-Shot Multimodal Reasoning with LanguagearXiv:2204.00598Andy Zeng, Maria Attarian, Brian Ichter, et al.2022 · 302 citations
14. Challenges and Applications of Large Language ModelsarXiv:2307.10169Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.2023 · 301 citations
15. Canine: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics (TACL)Jonathan H. Clark, Dan Garrette, Iulia Turc, et al.2022 · 301 citations
16. Perceiver: General Perception with Iterative AttentionarXiv:2103.03206Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.2021 · 269 citations
17. Transformers learn in-context by gradient descentarXiv:2212.07677Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, et al.2022 · 247 citations
18. Charformer: Fast Character Transformers via Gradient-based Subword TokenizationarXiv:2106.12672Yi Tay, Vinh Q. Tran, Sebastian Ruder, et al.2021 · 186 citations
19. Adapting Language Models to Compress ContextsConference on Empirical Methods in Natural Language Processing (EMNLP)Alexis Chevalier, Alexander Wettig, Anirudh Ajith, et al.2023 · 182 citations
20. SCROLLS: Standardized CompaRison Over Long Language SequencesConference on Empirical Methods in Natural Language Processing (EMNLP)Uri Shaham, Elad Segal, Maor Ivgi, et al.2022 · 165 citations
21. Efficient Methods for Natural Language Processing: A SurveyTransactions of the Association for Computational Linguistics (TACL)Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.2023 · 145 citations
22. A Length-Extrapolatable TransformerAnnual Meeting of the Association for Computational Linguistics (ACL)Yutao Sun, Li Dong, Barun Patra, et al.2023 · 118 citations
23. Do Long-Range Language Models Actually Use Long-Range Context?Conference on Empirical Methods in Natural Language Processing (EMNLP)Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, et al.2021 · 101 citations
24. How to Train Long-Context Language Models (Effectively)Annual Meeting of the Association for Computational Linguistics (ACL)Tianyu Gao, Alexander Wettig, Howard Yen, et al.2025 · 89 citations
25. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?Findings of the Association for Computational LinguisticsYi Tay, Mostafa Dehghani, Samira Abnar, et al.2023 · 86 citations
26. MoBA: Mixture of Block Attention for Long-Context LLMsConference on Neural Information Processing Systems (NeurIPS)Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.2025 · 80 citations
27. Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text SummarizationComputational LinguisticsMd Tahmid Rahman Laskar, Enamul Hoque, Jimmy Xiangji Huang2022 · 76 citations
28. The Devil in Linear TransformerConference on Empirical Methods in Natural Language Processing (EMNLP)Zhen Qin, Xiaodong Han, Weixuan Sun, et al.2022 · 67 citations
29. MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning AttentionarXiv:2506.13585MiniMax, Aili Chen, Aonian Li, et al.2025 · 66 citations
30. A Comprehensive Survey on Long Context Language ModelingarXiv:2503.17407Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.2025 · 64 citations
31. Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language ModelsarXiv:2402.02244Xindi Wang, Mahsa Salmani, Parsa Omidi, et al.2024 · 60 citations
32. Simple linear attention language models balance the recall-throughput tradeoffarXiv:2402.18668Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.2024 · 59 citations
33. Scaling Laws of RoPE-based ExtrapolationarXiv:2310.05209Xiaoran Liu, Hang Yan, Shuo Zhang, et al.2023 · 54 citations
34. Long-Context Language Modeling with Parallel Context EncodingAnnual Meeting of the Association for Computational Linguistics (ACL)Howard Yen, Tianyu Gao, Danqi Chen2024 · 44 citations
35. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormerarXiv:2307.14995Zhen Qin, Dong Li, Weigao Sun, et al.2023 · 33 citations
36. TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked TextConference on Empirical Methods in Natural Language Processing (EMNLP)Songshuo Lu, Hua Wang, Yutian Rong, et al.2025 · 32 citations
37. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language ModelsarXiv:2401.04658Zhen Qin, Weigao Sun, Dong Li, et al.2024 · 31 citations
38. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language ProcessingConference on Empirical Methods in Natural Language Processing (EMNLP)Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 27 citations
39. Block-State TransformersarXiv:2306.09539Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.2023 · 20 citations
40. Linearizing Large Language ModelsarXiv:2405.06640Jean Mercat, Igor Vasiljevic, Sedrick Keh, et al.2024 · 16 citations
41. Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Zhisong Zhang, Yan Wang, Xinting Huang, et al.2025 · 14 citations
42. PanGu-π: Enhancing Language Model Architectures via Nonlinearity CompensationarXiv:2312.17276Yunhe Wang, Hanting Chen, Yehui Tang, et al.2023 · 10 citations
43. Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language UnderstandingarXiv:2205.10687Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.2022 · 9 citations
44. Just read twice: closing the recall gap for recurrent language modelsarXiv:2407.05483Simran Arora; Aman Timalsina; Aaryan Singhal; Benjamin Spector; Sabri Eyuboglu; Xinyi Zhao; Ashish Rao; Atri Rudra;, Christopher Re2024 · 8 citations
45. An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with DiacriticsarXiv:2208.11484Aly Mostafa, Omar Mohamed, Ali Ashraf, et al.2022 · 3 citations
46. Cross-layer Attention Sharing for Pre-trained Large Language ModelsTransactions of the Association for Computational Linguistics (TACL)Yongyu Mu, Yuzhang Wu, Yuchun Fan, et al.2026 · 2 citations
47. MiniMax Sparse AttentionarXiv:2606.13392Xunhao Lai, Weiqi Xu, Yufeng Yang, et al.2026 · 2 citations
48. From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code CompletionAnnual Meeting of the Association for Computational Linguistics (ACL)Yanli Wang, Yanlin Wang, Bowen Zhang, et al.2026
49. SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language ModelsJoint Conference on Lexical and Computational Semantics (*SEM)Dong Liu, Yanxuan Yu2026

338 more papers of our library cite it; they have no page here yet.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/cited-by/