Elamur Research⌕
☰
home / papers / FlashAttention-2: Faster Attention with Better… / cited by

Who cites «FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning»

1,097 papers of our library cite it (of the 149,439 whose bibliographies we read) · 150 with a page here, the most cited first · page 1 of 2 · 1–100 · ← back to «FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning»

1 2

  1. A Survey of Large Language ModelsFrontiers of Computer ScienceWayne Xin Zhao, Kun Zhou, Junyi Li, et al.2023 · 2,875 citations
  2. Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any ResolutionarXiv:2409.12191Peng Wang, Shuai Bai, Sinan Tan, et al.2024 · 860 citations
  3. Baichuan 2: Open Large-scale Language ModelsarXiv:2309.10305Aiyuan Yang, Bin Xiao, Bingning Wang, et al.2023 · 517 citations
  4. The Falcon Series of Open Language ModelsarXiv:2311.16867Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, et al.2023 · 443 citations
  5. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical DomainsFindings of the Association for Computational LinguisticsYanis Labrak, Adrien Bazoge, Emmanuel Morin, et al.2024 · 409 citations
  6. Yi: Open Foundation Models by 01.AIarXiv:2403.04652Alex Young, Bei Chen, Chao Li, et al.2024 · 408 citations
  7. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionConference on Neural Information Processing Systems (NeurIPS)Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.2024 · 400 citations
  8. RULER: What's the Real Context Size of Your Long-Context Language Models?arXiv:2404.06654Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, et al.2024 · 400 citations
  9. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and InferencearXiv:2412.13663Benjamin Warner, Antoine Chaffin, Benjamin Clavié, et al.2024 · 358 citations
  10. SnapKV: LLM Knows What You are Looking for Before GenerationConference on Neural Information Processing Systems (NeurIPS)Yuhong Li, Yingbing Huang, Bowen Yang, et al.2024 · 355 citations
  11. TinyLlama: An Open-Source Small Language ModelarXiv:2401.02385Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, et al.2024 · 352 citations
  12. Textbooks Are All You Need II: phi-1.5 technical reportarXiv:2309.05463Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, et al.2023 · 336 citations
  13. MEDITRON-70B: Scaling Medical Pretraining for Large Language ModelsarXiv:2311.16079Zeming Chen, Alejandro Hernandez Cano, Angelika Romanou, et al.2023 · 315 citations
  14. InternLM2 Technical ReportarXiv:2403.17297Zheng Cai, Maosong Cao, Haojiong Chen, et al.2024 · 301 citations
  15. Llemma: An Open Language Model For MathematicsarXiv:2310.10631Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, et al.2023 · 286 citations
  16. An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language ModelsLecture notes in computer scienceLiang Chen, Haozhe Zhao, Tianyu Liu, et al.2024 · 279 citations
  17. LongLoRA: Efficient Fine-tuning of Long-Context Large Language ModelsarXiv:2309.12307Yukang Chen, Shengju Qian, Haotian Tang, et al.2023 · 268 citations
  18. Fine-Tuning LLaMA for Multi-Stage Text RetrievalInternational ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)Xueguang Ma, Liang Wang, Nan Yang, et al.2023 · 257 citations
  19. Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts ArchitecturearXiv:2306.16092Jiaxi Cui; Munan Ning; Zongjian Li; Bohua Chen; Yang Yan; Hao Li; Bin Ling; Yonghong Tian;, Li Yuan2023 · 248 citations
  20. mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text RetrievalConference on Empirical Methods in Natural Language Processing (EMNLP)Xin Zhang, Yanzhao Zhang, Dingkun Long, et al.2024 · 237 citations
  21. ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem SolvingarXiv:2309.17452Zhibin Gou, Zhihong Shao, Yeyun Gong, et al.2023 · 225 citations
  22. Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language ModelsIEEE Computer Society Conference on Computer Vision and Pattern RecognitionMatt Deitke, Christopher Clark, Sangho Lee, et al.2024 · 216 citations
  23. LIMO: Less is More for ReasoningarXiv:2502.03387Yixin Ye, Zhen Huang, Yang Xiao, et al.2025 · 209 citations
  24. MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse AttentionConference on Neural Information Processing Systems (NeurIPS)Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, et al.2024 · 202 citations
  25. InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and EfficiencyarXiv:2508.18265Weiyun Wang, Zhangwei Gao, Lixin Gu, et al.2025 · 179 citations
  26. KernelBench: Can LLMs Write Efficient GPU Kernels?arXiv:2502.10517Anne Ouyang, Simon Guo, Simran Arora, et al.2025 · 178 citations
  27. What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction TuningarXiv:2312.15685Wei Liu, Weihao Zeng, Keqing He, et al.2023 · 178 citations
  28. Generative Representational Instruction TuningarXiv:2402.09906Niklas Muennighoff, Hongjin Su, Liang Wang, et al.2024 · 172 citations
  29. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer ModelsarXiv:2309.14509Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, et al.2023 · 162 citations
  30. The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context LearningarXiv:2312.01552Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, et al.2023 · 150 citations
  31. Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative DecodingAnnual Meeting of the Association for Computational Linguistics (ACL)Jun Zhang, Jue Wang, Huan Li, et al.2024 · 148 citations
  32. Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative DecodingFindings of the Association for Computational LinguisticsHeming Xia, Zhe Yang, Qingxiu Dong, et al.2024 · 146 citations
  33. LongRoPE: Extending LLM Context Window Beyond 2 Million TokensarXiv:2402.13753Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, et al.2024 · 142 citations
  34. Olmo 3arXiv:2512.13961Team Olmo: Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, et al.2025 · 130 citations
  35. Retrieval meets Long Context Large Language ModelsarXiv:2310.03025Peng Xu, Wei Ping, Xianchao Wu, et al.2023 · 130 citations
  36. InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context MemoryConference on Neural Information Processing Systems (NeurIPS)Chaojun Xiao, Pengle Zhang, Xu Han, et al.2024 · 126 citations
  37. Generative Judge for Evaluating AlignmentarXiv:2310.05470Junlong Li, Shichao Sun, Weizhe Yuan, et al.2023 · 125 citations
  38. AlphaMath Almost Zero: Process Supervision without ProcessConference on Neural Information Processing Systems (NeurIPS)Guoxin Chen, Minpeng Liao, Chengxi Li, et al.2024 · 120 citations
  39. Demystifying Long Chain-of-Thought Reasoning in LLMsarXiv:2502.03373Edward Yeo, Yuxuan Tong, Morry Niu, et al.2025 · 119 citations
  40. QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice CodebooksarXiv:2402.04396Albert Tseng, Jerry Chee, Qingyao Sun, et al.2024 · 117 citations
  41. Break the Sequential Dependency of LLM Inference Using Lookahead DecodingarXiv:2402.02057Yichao Fu, Peter Bailis, Ion Stoica, et al.2024 · 114 citations
  42. Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language CapabilitiesarXiv:2404.17790Kazuki Fujii, Taishi Nakamura, Mengsay Loem, et al.2024 · 114 citations
  43. DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming HeadsarXiv:2410.10819Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, et al.2024 · 114 citations
  44. ALLaM: Large Language Models for Arabic and EnglisharXiv:2407.15390M Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, et al.2024 · 113 citations
  45. Data Engineering for Scaling Language Models to 128K ContextarXiv:2402.10171Yao Fu, Rameswar Panda, Xinyao Niu, et al.2024 · 109 citations
  46. LongAlign: A Recipe for Long Context Alignment of Large Language ModelsarXiv:2401.18058Yushi Bai, Xin Lv, Jiajie Zhang, et al.2024 · 98 citations
  47. PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM InferenceFindings of the Association for Computational LinguisticsDongjie Yang, Xiaodong Han, Yan Gao, et al.2024 · 91 citations
  48. Skywork: A More Open Bilingual Foundation ModelarXiv:2310.19341Tianwen Wei, Liang Zhao, Lichang Zhang, et al.2023 · 91 citations
  49. How to Train Long-Context Language Models (Effectively)Annual Meeting of the Association for Computational Linguistics (ACL)Tianyu Gao, Alexander Wettig, Howard Yen, et al.2025 · 89 citations
  50. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive RetrievalarXiv:2407.12883Hongjin Su, Howard Yen, Mengzhou Xia, et al.2024 · 77 citations
  51. Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI ArchitecturesProceedings - International Symposium on Computer Architecture/ProceedingsChenggang Zhao, Chengqi Deng, Chong Ruan, et al.2025 · 74 citations
  52. Seed-TTS: A Family of High-Quality Versatile Speech Generation ModelsarXiv:2406.02430Philip Anastassiou, Jiawei Chen, Jitong Chen, et al.2024 · 71 citations
  53. MagicPIG: LSH Sampling for Efficient LLM GenerationarXiv:2410.16179Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye, et al.2024 · 68 citations
  54. A Comprehensive Survey on Long Context Language ModelingarXiv:2503.17407Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.2025 · 64 citations
  55. Unifying Multimodal Retrieval via Document Screenshot EmbeddingConference on Empirical Methods in Natural Language Processing (EMNLP)Xueguang Ma, Sheng-Chieh Lin, Minghan Li, et al.2024 · 61 citations
  56. XAttention: Block Sparse Attention with Antidiagonal ScoringarXiv:2503.16428Ruyi Xu, Guangxuan Xiao, Haofeng Huang, et al.2025 · 59 citations
  57. Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language ModelingarXiv:2406.07522Liliang Ren, Yang Liu, Yadong Lu, et al.2024 · 59 citations
  58. Zoology: Measuring and Improving Recall in Efficient Language ModelsarXiv:2312.04927Simran Arora; Sabri Eyuboglu; Aman Timalsina; Isys Johnson; Michael Poli; James Zou; Atri Rudra;, Christopher Re2023 · 55 citations
  59. Scaling Laws of RoPE-based ExtrapolationarXiv:2310.05209Xiaoran Liu, Hang Yan, Shuo Zhang, et al.2023 · 54 citations
  60. PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise TrainingarXiv:2309.10400Dawei Zhu, Nan Yang, Liang Wang, et al.2023 · 52 citations
  61. Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired SketchingConference on Empirical Methods in Natural Language Processing (EMNLP)Simon A. Aytes, Jinheon Baek, Sung Ju Hwang2025 · 51 citations
  62. From Decoding to Meta-Generation: Inference-time Algorithms for Large Language ModelsarXiv:2406.16838Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.2024 · 51 citations
  63. MiniMax-01: Scaling Foundation Models with Lightning AttentionarXiv:2501.08313MiniMax, Aonian Li, Bangwei Gong, et al.2025 · 50 citations
  64. WebLINX: Real-World Website Navigation with Multi-Turn DialoguearXiv:2402.05930Xing Han Lu, Zden\vek Kasner, Siva Reddy2024 · 50 citations
  65. Long Context Compression with Activation BeaconarXiv:2401.03462Peitian Zhang, Zheng Liu, Shitao Xiao, et al.2024 · 47 citations
  66. Stable LM 2 1.6B Technical ReportarXiv:2402.17834Marco Bellagente, Jonathan Tow, Dakota Mahan, et al.2024 · 46 citations
  67. Stop Looking for “Important Tokens” in Multimodal Language Models: Duplication Matters MoreConference on Empirical Methods in Natural Language Processing (EMNLP)Zichen Wen, Yifeng Gao, Shaobo Wang, et al.2025 · 44 citations
  68. APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human InterplayConference on Neural Information Processing Systems (NeurIPS)Akshara Prabhakar, Zuxin Liu, Ming Zhu, et al.2025 · 36 citations
  69. WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement LearningConference on Empirical Methods in Natural Language Processing (EMNLP)Zhepei Wei, Wenlin Yao, Yao Liu, et al.2025 · 36 citations
  70. EuroBERT: Scaling Multilingual Encoders for European LanguagesarXiv:2503.05500Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves, et al.2025 · 34 citations
  71. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormerarXiv:2307.14995Zhen Qin, Dong Li, Weigao Sun, et al.2023 · 33 citations
  72. TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked TextConference on Empirical Methods in Natural Language Processing (EMNLP)Songshuo Lu, Hua Wang, Yutian Rong, et al.2025 · 32 citations
  73. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language ModelsarXiv:2401.04658Zhen Qin, Weigao Sun, Dong Li, et al.2024 · 31 citations
  74. Weaver: Foundation Models for Creative WritingarXiv:2401.17268Tiannan Wang; Jiamin Chen; Qingrui Jia; Shuai Wang; Ruoyu Fang; Huilin Wang; Zhaowei Gao; Chunzhao Xie; Chuou Xu; Jihong Dai; Yibin Liu; Jialong Wu; Shengwei Ding; Long Li; Zhiwei Huang; Xinle Deng; Teng Yu; Gangan Ma; Han Xiao; Zixin Chen; Danjun Xiang; Yunxia Wang; Yuanyuan Zhu; Yi Xiao; Jing Wang; Yiru Wang; Siran Ding; Jiayang Huang; Jiayi Xu; Yilihamu Tayier; Zhenyu Hu; Yuan Gao; Chengfeng Zheng; Yueshu Ye; Yihang Li; Lei Wan; Xinyue Jiang; Yujie Wang; Siyu Cheng; Zhule Song; Xiangru Tang; Xiaohua Xu; Ningyu Zhang; Huajun Chen; Yuchen Eleanor Jiang;, Wangchunshu Zhou2024 · 31 citations
  75. Language Models as Science TutorsarXiv:2402.11111Alexis Chevalier, Jiayi Geng, Alexander Wettig, et al.2024 · 30 citations
  76. ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Haochen Tan, Zhijiang Guo, Zhan Shi, et al.2024 · 30 citations
  77. Sailor: Open Language Models for South-East AsiaarXiv:2404.03608Longxu Dou, Qian Liu, Guangtao Zeng, et al.2024 · 27 citations
  78. Binary Classifier Optimization for Large Language Model AlignmentAnnual Meeting of the Association for Computational Linguistics (ACL)Seungjae Jung, Gunsoo Han, Daniel Wontae Nam, et al.2025 · 26 citations
  79. Distillation Scaling LawsarXiv:2502.08606Dan Busbridge, Amitis Shidani, Floris Weers, et al.2025 · 26 citations
  80. TeleChat Technical ReportarXiv:2401.03804Zhongjiang He, Zihan Wang, Xinzhang Liu, et al.2024 · 24 citations
  81. Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm PerspectiveAnnual Meeting of the Association for Computational Linguistics (ACL)Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, et al.2025 · 23 citations
  82. DDK: Distilling Domain Knowledge for Efficient Large Language ModelsConference on Neural Information Processing Systems (NeurIPS)Jiaheng Liu, Chenchen Zhang, Jinyang Guo, et al.2024 · 23 citations
  83. Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token RecyclingAnnual Meeting of the Association for Computational Linguistics (ACL)Xianzhen Luo, Yixuan Wang, Qingfu Zhu, et al.2025 · 22 citations
  84. GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?arXiv:2502.05252Yang Zhou, Hongyi Liu, Zhuoming Chen, et al.2025 · 21 citations
  85. Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from JailbreakingConference on Empirical Methods in Natural Language Processing (EMNLP)Junda Zhu, Lingyong Yan, Shuaiqiang Wang, et al.2025 · 21 citations
  86. NeoBERT: A Next-Generation BERTarXiv:2502.19587Lola Le Breton, Quentin Fournier, Mariam El Mezouar, et al.2025 · 20 citations
  87. Suri: Multi-constraint Instruction Following in Long-form Text GenerationFindings of the Association for Computational LinguisticsChau Minh Pham, Simeng Sun, Mohit Iyyer2024 · 19 citations
  88. Timo: Towards Better Temporal Reasoning for Language ModelsarXiv:2406.14192Zhaochen Su, Jun Zhang, Tong Zhu, et al.2024 · 19 citations
  89. ReasonRank: Empowering Passage Ranking with Strong Reasoning AbilityAnnual Meeting of the Association for Computational Linguistics (ACL)Wenhan Liu, Xinyu Ma, Weiwei Sun, et al.2026 · 18 citations
  90. Orion-14B: Open-source Multilingual Large Language ModelsarXiv:2401.12246Du Chen, Yi Huang, Xiaopu Li, et al.2024 · 17 citations
  91. Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Xuyang Liu, Yiyu Wang, Junpeng Ma, et al.2025 · 16 citations
  92. Dataset Decomposition: Faster LLM Training with Variable Sequence Length CurriculumConference on Neural Information Processing Systems (NeurIPS)Hadi Pouransari, Chun-Liang Li, Jen-Hao Rick Chang, et al.2024 · 16 citations
  93. Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and PerformancearXiv:2507.22448Jingwei Zuo, Maksim Velikanov, Ilyas Chahed, et al.2025 · 15 citations
  94. FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative SamplingAnnual Meeting of the Association for Computational Linguistics (ACL)Weilin Zhao, Tengyu Pan, Xu Han, et al.2025 · 15 citations
  95. LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from ScratchAnnual Meeting of the Association for Computational Linguistics (ACL)Jan Pfister, Julia Wunderle, Andreas Hotho2025 · 15 citations
  96. MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation SystemsConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Nandan Thakur, Suleman Kazi, Ge Luo, et al.2025 · 15 citations
  97. What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best PracticesAnnual Meeting of the Association for Computational Linguistics (ACL)Zhi Chen, Qiguang Chen, Libo Qin, et al.2025 · 15 citations
  98. RAG-Critic: Leveraging Automated Critic-Guided Agentic Workflow for Retrieval Augmented GenerationAnnual Meeting of the Association for Computational Linguistics (ACL)Guanting Dong, Jiajie Jin, Xiaoxi Li, et al.2025 · 14 citations
  99. Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token ReductionFindings of the Association for Computational LinguisticsZhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, et al.2026 · 13 citations
  100. LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMsAnnual Meeting of the Association for Computational Linguistics (ACL)Jianghao Chen, Junhong Wu, Yangyifan Xu, et al.2025 · 12 citations

1 2

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/cited-by/