Elamur Research⌕
☰
home / authors / Albert Gu / cited by

Who cites Albert Gu

1,191 citations of their works from 783 papers of our library (of the 149,439 whose bibliographies we read) · 117 with a page here, the most cited first · page 1 of 2 · 1–100 · ← Albert Gu

1 2

  1. A Survey of Large Language ModelsFrontiers of Computer ScienceWayne Xin Zhao, Kun Zhou, Junyi Li, et al.2023 · 2,875 citations
  2. Lost in the Middle: How Language Models Use Long ContextsTransactions of the Association for Computational Linguistics (TACL)Nelson F. Liu, Kevin Lin, John Hewitt, et al.2023 · 2,584 citations
  3. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextarXiv:2403.05530Gemini Team, Petko Georgiev, Ving Ian Lei, et al.2024 · 1,589 citations
  4. Efficient Transformers: A SurveyACM Computing SurveysYi Tay, Mostafa Dehghani, Dara Bahri, et al.2020 · 1,539 citations
  5. LongBench: A Bilingual, Multitask Benchmark for Long Context UnderstandingarXiv:2308.14508Yushi Bai, Xin Lv, Jiajie Zhang, et al.2023 · 514 citations
  6. Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space ModelarXiv:2401.09417Lianghui Zhu, Bencheng Liao, Qian Zhang, et al.2024 · 500 citations
  7. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisionConference on Neural Information Processing Systems (NeurIPS)Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.2024 · 400 citations
  8. RULER: What's the Real Context Size of Your Long-Context Language Models?arXiv:2404.06654Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, et al.2024 · 400 citations
  9. xLSTM: Extended Long Short-Term MemoryConference on Neural Information Processing Systems (NeurIPS)Maximilian Beck, Korbinian Pöppel, Markus Spanring, et al.2024 · 296 citations
  10. A Review of Large Language Models and Autonomous Agents in ChemistryarXiv:2407.01603Mayk Caldas Ramos, Christopher J. Collison, Andrew D. White2024 · 288 citations
  11. Retentive Network: A Successor to Transformer for Large Language ModelsarXiv:2307.08621Yutao Sun, Li Dong, Shaohan Huang, et al.2023 · 265 citations
  12. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive SurveyarXiv:2403.14608Zeyu Han, Chao Gao, Jinyang Liu, et al.2024 · 240 citations
  13. EAGLE: Speculative Sampling Requires Rethinking Feature UncertaintyarXiv:2401.15077Yuhui Li, Fangyun Wei, Chao Zhang, et al.2024 · 208 citations
  14. MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse AttentionConference on Neural Information Processing Systems (NeurIPS)Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, et al.2024 · 202 citations
  15. DataComp-LM: In search of the next generation of training sets for language modelsConference on Neural Information Processing Systems (NeurIPS)Jeffrey Li, Alex Fang, Georgios Smyrnis, et al.2024 · 191 citations
  16. ShortGPT: Layers in Large Language Models are More Redundant Than You ExpectarXiv:2403.03853Xin Men, Mingyu Xu, Qingyu Zhang, et al.2024 · 169 citations
  17. Mooncake: A KVCache-centric Disaggregated Architecture for LLM ServingFAST 2025Ruoyu Qin, Zheming Li, Weiran He, et al.2024 · 161 citations
  18. A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and TrustworthinessACM Transactions on Intelligent Systems and TechnologyFali Wang, Zhiwei Zhang, Xianren Zhang, et al.2024 · 159 citations
  19. Jamba: A Hybrid Transformer-Mamba Language ModelarXiv:2403.19887Opher Lieber, Barak Lenz, Hofit Bata, et al.2024 · 159 citations
  20. Atom: Low-bit Quantization for Efficient and Accurate LLM ServingMLSys 2024Yilong Zhao, Chien-Yu Lin, Kan Zhu, et al.2023 · 156 citations
  21. Simple and Effective Masked Diffusion Language ModelsConference on Neural Information Processing Systems (NeurIPS)Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, et al.2024 · 152 citations
  22. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video DiffusionConference on Neural Information Processing Systems (NeurIPS)Xun Huang, Zhengqi Li, Guande He, et al.2025 · 146 citations
  23. Efficient Methods for Natural Language Processing: A SurveyTransactions of the Association for Computational Linguistics (TACL)Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.2023 · 145 citations
  24. Long-context LLMs Struggle with Long In-context LearningarXiv:2404.02060Tianle Li, Ge Zhang, Quy Duc Do, et al.2024 · 138 citations
  25. InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context MemoryConference on Neural Information Processing Systems (NeurIPS)Chaojun Xiao, Pengle Zhang, Xu Han, et al.2024 · 126 citations
  26. LongNet: Scaling Transformers to 1,000,000,000 TokensarXiv:2307.02486Jiayu Ding, Shuming Ma, Li Dong, et al.2023 · 123 citations
  27. SaulLM-7B: A pioneering Large Language Model for LawarXiv:2403.03883Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, et al.2024 · 119 citations
  28. Resurrecting Recurrent Neural Networks for Long SequencesarXiv:2303.06349Antonio Orvieto, Samuel L Smith, Albert Gu, et al.2023 · 118 citations
  29. DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming HeadsarXiv:2410.10819Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, et al.2024 · 114 citations
  30. A Survey on Efficient Inference for Large Language ModelsarXiv:2404.14294Zixuan Zhou, Xuefei Ning, Ke Hong, et al.2024 · 113 citations
  31. Efficient Large Language Models: A SurveyarXiv:2312.03863Zhongwei Wan, Xin Wang, Che Liu, et al.2023 · 110 citations
  32. Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to SystemsACM Computing SurveysXupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al.2023 · 110 citations
  33. Data Engineering for Scaling Language Models to 128K ContextarXiv:2402.10171Yao Fu, Rameswar Panda, Xinyao Niu, et al.2024 · 109 citations
  34. Make Your LLM Fully Utilize the ContextConference on Neural Information Processing Systems (NeurIPS)Shengnan An, Zexiong Ma, Zeqi Lin, et al.2024 · 97 citations
  35. Retrieval Head Mechanistically Explains Long-Context FactualityarXiv:2404.15574Wenhao Wu, Yizhong Wang, Guangxuan Xiao, et al.2024 · 94 citations
  36. Efficient Long-Text Understanding with Short-Text ModelsTransactions of the Association for Computational Linguistics (TACL)Maor Ivgi, Uri Shaham, Jonathan Berant2023 · 94 citations
  37. How to Train Long-Context Language Models (Effectively)Annual Meeting of the Association for Computational Linguistics (ACL)Tianyu Gao, Alexander Wettig, Howard Yen, et al.2025 · 89 citations
  38. BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-HaystackConference on Neural Information Processing Systems (NeurIPS)Yuri Kuratov, Aydar Bulatov, Petr Anokhin, et al.2024 · 88 citations
  39. Small Language Models: Survey, Measurements, and InsightsarXiv:2409.15790Zhenyan Lu, Xiang Li, Dongqi Cai, et al.2024 · 88 citations
  40. Benchmark Data Contamination of Large Language Models: A SurveyarXiv:2406.04244Cheng Xu, Shuhao Guan, Derek Greene, et al.2024 · 85 citations
  41. MoBA: Mixture of Block Attention for Long-Context LLMsConference on Neural Information Processing Systems (NeurIPS)Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.2025 · 80 citations
  42. A Primer on the Inner Workings of Transformer-based Language ModelsarXiv:2405.00208Javier Ferrando, Gabriele Sarti, Arianna Bisazza, et al.2024 · 72 citations
  43. (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary TextsTransactions of the Association for Computational Linguistics (TACL)Minghao Wu, Jiahao Xu, Yulin Yuan, et al.2025 · 71 citations
  44. Recent Advances in Speech Language Models: A SurveyAnnual Meeting of the Association for Computational Linguistics (ACL)Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, et al.2025 · 69 citations
  45. An Empirical Study of Mamba-based Language ModelsarXiv:2406.07887Roger Waleffe, Wonmin Byeon, Duncan Riach, et al.2024 · 67 citations
  46. MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning AttentionarXiv:2506.13585MiniMax, Aili Chen, Aonian Li, et al.2025 · 66 citations
  47. A Comprehensive Survey on Long Context Language ModelingarXiv:2503.17407Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.2025 · 64 citations
  48. Layer by Layer: Uncovering Hidden Representations in Language ModelsarXiv:2502.02013Oscar Skean, Md Rifat Arefin, Dan Zhao, et al.2025 · 64 citations
  49. Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language ModelsarXiv:2402.02244Xindi Wang, Mahsa Salmani, Parsa Omidi, et al.2024 · 60 citations
  50. BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language ModelsLanguage Resources and Evaluation Conference (LREC)Zican Dong, Tianyi Tang, Junyi Li, et al.2024 · 59 citations
  51. Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language ModelingarXiv:2406.07522Liliang Ren, Yang Liu, Yadong Lu, et al.2024 · 59 citations
  52. Simple linear attention language models balance the recall-throughput tradeoffarXiv:2402.18668Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.2024 · 59 citations
  53. A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and BeyondarXiv:2503.21614Xiaoye Qu, Yafu Li, Zhao-Chen Su, et al.2025 · 55 citations
  54. Zoology: Measuring and Improving Recall in Efficient Language ModelsarXiv:2312.04927Simran Arora; Sabri Eyuboglu; Aman Timalsina; Isys Johnson; Michael Poli; James Zou; Atri Rudra;, Christopher Re2023 · 55 citations
  55. Kimi Linear: An Expressive, Efficient Attention ArchitecturearXiv:2510.26692Kimi Team, Yu Zhang, Zongyu Lin, et al.2025 · 51 citations
  56. From Decoding to Meta-Generation: Inference-time Algorithms for Large Language ModelsarXiv:2406.16838Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.2024 · 51 citations
  57. MiniMax-01: Scaling Foundation Models with Lightning AttentionarXiv:2501.08313MiniMax, Aonian Li, Bangwei Gong, et al.2025 · 50 citations
  58. Hymba: A Hybrid-head Architecture for Small Language ModelsarXiv:2411.13676Xin Dong, Yonggan Fu, Shizhe Diao, et al.2024 · 50 citations
  59. Language models scale reliably with over-training and on downstream tasksarXiv:2403.08540Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, et al.2024 · 46 citations
  60. Long-Context Language Modeling with Parallel Context EncodingAnnual Meeting of the Association for Computational Linguistics (ACL)Howard Yen, Tianyu Gao, Danqi Chen2024 · 44 citations
  61. NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning ModelarXiv:2508.14444NVIDIA: Aarti Basant, Abhijit Khairnar, Abhijit Paithankar, et al.2025 · 39 citations
  62. Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer ModelsarXiv:2504.03624NVIDIA: Aaron Blakeman, Aarti Basant, Abhinav Khattar, et al.2025 · 38 citations
  63. Efficient Reasoning Models: A SurveyarXiv:2504.10903Sicheng Feng, Gongfan Fang, Xinyin Ma, et al.2025 · 37 citations
  64. MambaByte: Token-free Selective State Space ModelarXiv:2401.13660Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan, et al.2024 · 35 citations
  65. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormerarXiv:2307.14995Zhen Qin, Dong Li, Weigao Sun, et al.2023 · 33 citations
  66. The Hidden Attention of Mamba ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Ameen Ali Ali, Itamar Zimerman, Lior Wolf2025 · 32 citations
  67. A Survey of Small Language ModelsarXiv:2410.20011Chien Van Nguyen, Xuan Shen, Ryan Aponte, et al.2024 · 31 citations
  68. Scaling Laws and Compute-Optimal Training Beyond Fixed Training DurationsConference on Neural Information Processing Systems (NeurIPS)Alexander Hägele, Elie Bakouch, Atli Kosson, et al.2024 · 31 citations
  69. Jamba-1.5: Hybrid Transformer-Mamba Models at ScalearXiv:2408.12570Jamba Team: Barak Lenz, Alan Arazi, Amir Bergman, et al.2024 · 30 citations
  70. Towards the Law of Capacity Gap in Distilling Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Chen Zhang, Qiuchi Li, Dawei Song, et al.2025 · 27 citations
  71. Baichuan-M1: Pushing the Medical Capability of Large Language ModelsarXiv:2502.12671Bingning Wang, Haizhou Zhao, Huozhi Zhou, et al.2025 · 24 citations
  72. RWKV-7 "Goose" with Expressive Dynamic State EvolutionarXiv:2503.14456Bo Peng, Ruichong Zhang, Daniel Goldstein, et al.2025 · 23 citations
  73. LongReward: Improving Long-context Large Language Models with AI FeedbackAnnual Meeting of the Association for Computational Linguistics (ACL)Jiajie Zhang, Zhongni Hou, Xin Lv, et al.2025 · 21 citations
  74. Block-State TransformersarXiv:2306.09539Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.2023 · 20 citations
  75. Locating and Editing Factual Associations in MambaarXiv:2404.03646Arnab Sen Sharma, David Atkinson, David Bau2024 · 18 citations
  76. Linearizing Large Language ModelsarXiv:2405.06640Jean Mercat, Igor Vasiljevic, Sedrick Keh, et al.2024 · 16 citations
  77. A Survey on Latent ReasoningarXiv:2507.06203Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, et al.2025 · 15 citations
  78. Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and PerformancearXiv:2507.22448Jingwei Zuo, Maksim Velikanov, Ilyas Chahed, et al.2025 · 15 citations
  79. Semantic-Aware Logical Reasoning via a Semiotic FrameworkAnnual Meeting of the Association for Computational Linguistics (ACL)Yunyao Zhang, Xinglang Zhang, Junxi Sheng, et al.2026 · 13 citations
  80. When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM ReasoningarXiv:2504.01005Nishad Singhi, Hritik Bansal, Arian Hosseini, et al.2025 · 12 citations
  81. Base of RoPE Bounds Context LengtharXiv:2405.14591Xin Men, Mingyu Xu, Bingning Wang, et al.2024 · 12 citations
  82. Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase PretrainingarXiv:2412.15285Steven Feng, Shrimai Prabhumoye, Kezhi Kong, et al.2024 · 12 citations
  83. MiniPLM: Knowledge Distillation for Pre-Training Language ModelsarXiv:2410.17215Yuxian Gu, Hao Zhou, Fandong Meng, et al.2024 · 12 citations
  84. Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-ThoughtarXiv:2505.15431Tencent Hunyuan Team, Ao Liu, Botong Zhou, et al.2025 · 10 citations
  85. Associative Recurrent Memory TransformerarXiv:2407.04841Ivan Rodkin, Yuri Kuratov, Aydar Bulatov, et al.2024 · 8 citations
  86. Just read twice: closing the recall gap for recurrent language modelsarXiv:2407.05483Simran Arora; Aman Timalsina; Aaryan Singhal; Benjamin Spector; Sabri Eyuboglu; Xinyi Zhao; Ashish Rao; Atri Rudra;, Christopher Re2024 · 8 citations
  87. The Quest for the Right Mediator: Surveying Mechanistic Interpretability for NLP Through the Lens of Causal Mediation AnalysisComputational LinguisticsAaron Mueller, Jannik Brinkmann, Millicent Li, et al.2026 · 7 citations
  88. MoM: Linear Sequence Modeling with Mixture-of-MemoriesarXiv:2502.13685Jusen Du, Weigao Sun, Disen Lan, et al.2025 · 7 citations
  89. Detecting Training Data of Large Language Models via Expectation MaximizationConference of the European Chapter of the Association for Computational Linguistics (EACL)Gyuwan Kim, Yang Li, Evangelia Spiliopoulou, et al.2026 · 5 citations
  90. Barriers to Discrete Reasoning with Transformers: A Survey Across Depth, Exactness, and BandwidthConference of the European Chapter of the Association for Computational Linguistics (EACL)Michelle Yuan, Weiyi Sun, Amir H. Rezaeian, et al.2026 · 3 citations
  91. MISSION at KSAA-CAD 2024: AraT5 with Arabic Reverse DictionaryArabic Natural Language Processing Conference (ArabicNLP)Thamer Alharbi2024 · 3 citations
  92. Lizard: An Efficient Linearization Framework for Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Chien Van Nguyen, Huy Huu Nguyen, Ruiyi Zhang, et al.2026 · 2 citations
  93. LoPT: Lossless Parallel Tokenization Acceleration for Long Context Inference of Large Language ModelAnnual Meeting of the Association for Computational Linguistics (ACL)Wei Shao, Zheng Lingchao, Pengyu Wang, et al.2026 · 2 citations
  94. MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context ModelingarXiv:2602.11761MiniCPM Team, Wenhao An, Yingfa Chen, et al.2026 · 2 citations
  95. MiniMax Sparse AttentionarXiv:2606.13392Xunhao Lai, Weiqi Xu, Yufeng Yang, et al.2026 · 2 citations
  96. DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context ReasoningFindings of the Association for Computational LinguisticsHossein Entezari Zarch, Lei Gao, Chaoyi Jiang, et al.2026 · 1 citation
  97. StateX: Enhancing RNN Recall via Post-training State ExpansionFindings of the Association for Computational LinguisticsXingyu Shen, Yingfa Chen, Zhen Leng Thai, et al.2026 · 1 citation
  98. VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy DetectionInternational Workshop on Semantic Evaluation (SemEval)Hritav Solanki, Shubham Sharma, Manish Prasad, et al.2026 · 1 citation
  99. A Survey of Retentive NetworkFindings of the Association for Computational LinguisticsHaiqi Yang, Zhiyuan Li, Yi Chang, et al.2026
  100. AraSSM: A bidirectional state-space encoder for Arabic masked language modelingarXiv:2608.08256Ahmed Amine Aliane, Hassina Aliane, Nasredine Semmar2026

1 2

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/albert-gu/cited-by/