Elamur Research⌕
☰
home / papers / 2026 / A Survey of Retentive Network

A Survey of Retentive Network

Paper · 2026 · Haiqi Yang, Zhiyuan Li, Yi Chang et al.

Summary

Transformers scale poorly on long inputs: self-attention costs grow quadratically with sequence length during training, and the per-token KV cache inflates memory use at inference. RetNet (Sun et al., 2023) addresses this with a retention mechanism that merges recurrent-style inductive bias with attention-like parallel training. The authors say no systematic review of the growing RetNet literature existed, so they present what they call the first survey of it.

The paper has no new dataset or experiments. Instead, it does three things:

  • Explains the architecture. It covers the parallel, recurrent and chunkwise recurrent forms of retention, gated Multi-Scale Retention, and the overall block design.
  • Organizes the applications. It groups RetNet-based models into NLP, computer vision, audio, natural science, and spatio-temporal/environmental modeling.
  • Discusses open problems. These include weak associative recall caused by lossy state compression, unclear scaling behavior and training stability, hybrid designs, and robustness of the recurrent state.

The resulting map is useful for anyone weighing efficient Transformer alternatives for long-sequence or resource-constrained settings.

Key points

  • Block structure: RetNet stacks L identical blocks, each pairing Multi-Scale Retention (MSR) with an FFN. Each head gets a fixed decay γ = 1 − 2^(−5−arange(0,h)), followed by a swish gate and per-head GroupNorm.
  • Complexity: The chunkwise form makes training linear in sequence length, O(N) for a fixed chunk size B, while recurrent inference runs in constant time, O(1) per step.
  • Surveyed models: Examples include YOCO, LION and STSR (NLP); RMT, ViR, RetViT and SegRet (CV); RetNet-EEND, LS-EEND and LRetUNet (audio); CellFM, GR-pKa and DeepHapNet (science); and ST-RetNet, H-RetNet and FisH (spatio-temporal).
  • Proposed fixes for recall loss: The authors point to adaptive or input-gated decay (citing Mamba, GLA and YOCO as examples) and to hybrids such as KARMA, which combines retention with KANs.
  • Publication: Findings of ACL 2026, pp. 5200–5216, by authors from Jilin University.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Jilin UniversityChangchun · China30 works · 5,252 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Haiqi Yang, Zhiyuan Li, Yi Chang and Yuan Wu. 2026. A Survey of Retentive Network. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.256

@inproceedings{yang2026survey,
  title = {{A Survey of Retentive Network}},
  author = {Haiqi Yang, Zhiyuan Li, Yi Chang and Yuan Wu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.256},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.256}
}

References

107 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arshia Afzal, Elias Abad Rocamora, Leyla Naz Can-dogan, Pol Puigdemont, Francesco Tonin, Yongtao Wu, Mahsa Shoaran, and Volkan Cevher. Lion: A bidirectional framework that trains like a transformer and infers like an rnn. find
  2. Arshia Afzal, Elias Abad Rocamora, Leyla Naz Cando-gan, Pol Puigdemont, Francesco Tonin, Yongtao Wu, Mahsa Shoaran, and Volkan Cevher. 2025. Linear attention for efficient bidirectional sequence modeling. arXiv preprint arXiv:2502.16249. — [arXiv:2502.16249](https://arxiv.org/abs/2502.16249) find
  3. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
  4. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157–166. find
  5. Yueyang Cang, Pingge Hu, Xiaoteng Zhang, Xingtong Wang, Yuhang Liu, and Li Shi. 2024. Retcompletion: High-speed inference image completion with retentive network. arXiv preprint arXiv:2410.04056. — [arXiv:2410.04056](https://arxiv.org/abs/2410.04056) find
  6. Qian Chang, Xia Li, and Xiufeng Cheng. 2024. Graph retention networks for dynamic graphs. arXiv preprint arXiv:2411.11259. find
  7. Jun Cheng, Tao Meng, Xiao Ao, and Xiaohua Wu. 2024. Pre-training retnet of simulating entities and relations as sentences for knowledge graph reasoning. In 2024 4th Asia Conference on Information Engineering (ACIE), pages 6–10. IEEE. find
  8. Lei Cheng and Siyang Cao. 2025. Transrad: Retentive vision transformer for enhanced radar object detection. IEEE Transactions on Radar Systems. find
  9. Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, and 1 others. 2020. Rethinking attention with performers. arXiv preprint arXiv:2009.14794.
  10. Jinghui Chu, Wangtao Liu, Qi Tian, and Wei Lu. 2024. Pfprnet: A phase-wise feature pyramid with retention network for polyp segmentation. IEEE Journal of Biomedical and Health Informatics. find
  11. Lior Cohen, Kaixin Wang, Bingyi Kang, and Shie Mannor. 2024. Improving token-based world models with parallel observation prediction. arXiv preprint arXiv:2402.05643. find
  12. Soham De, Samuel L Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, and 1 others. 2024. Griffin: Mixing gated linear recurrences with local attention for efficient language models. arXiv preprint arXiv:2402… find
  13. JE Domínguez-Vidal and Alberto Sanfeliu. 2024. Force and velocity prediction in human-robot collaborative transportation tasks through video retentive networks. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9307–9313. IEEE. find
  14. Shreyas Dongre and Shrushti Mehta. 2024. Retvit: Retentive vision transformers. In 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), pages 1–8. IEEE. find
  15. Khaled ELKarazle, Valliappan Raman, Caslon Chua, and Patrick Then. 2023. Retseg: Retention-based colorectal polyps segmentation network. arXiv preprint arXiv:2310.05446. — [arXiv:2310.05446](https://arxiv.org/abs/2310.05446) find
  16. Gopi Krishna Erabati and Helder Araujo. 2024. Retformer: Embracing point cloud transformer with retentive network. IEEE Transactions on Intelligent Vehicles. find
  17. Gopi Krishna Erabati and Helder Araujo. 2025. Retseg3d: Retention-based 3d semantic segmentation for autonomous driving. Computer Vision and Image Understanding, 250:104231. find
  18. Lily Erickson. 2023. Cross-axis transformer with 3d rotary positional embeddings. Preprint, arXiv:2311.07184. — [arXiv:2311.07184](https://arxiv.org/abs/2311.07184) find
  19. Dengxin Fan and Songyan Liu. 2024. Monoretnet: A self-supervised model for monocular depth estimation with bidirectional half-duplex retention. In International Conference on Intelligent Computing, pages 361–372. Springer. find
  20. Qihang Fan, Huaibo Huang, Mingrui Chen, Hongmin Liu, and Ran He. 2024. Rmt: Retentive networks meet vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5641–5651. find
  21. Meng Feiyu. 2024. Cfpsg: Collaborative filtering poi similarity graph enhanced retentive network for next poi recommendation. In 2024 21st International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pages 1–4. IEEE. find
  22. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. find
  23. Albert Gu, Karan Goel, and Christopher Ré. 2021. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396. find
  24. Meisheng Guan, Haiyong Xu, Yeyao Chen, Ting Luo, Yang Song, and Huaping Wang. 2025. Retuie: Retention-based underwater image enhancement. In 2025 International Symposium on Machine Learning and Media Computing (MLMC), pages 1–10. IEEE. find
  25. Chen Guo, Xinran Li, Jiaman Ma, Yimeng Li, Yuefan Liu, Haiying Qi, Li Zhang, and Yuhan Jin. 2024. Vl-mfer: A vision-language multimodal pretrained model with multiway-fuzzy-experts bidirectional retention network. IEEE Transactions on Fuzzy Systems. find
  26. Tianjun Guo, Weixin Zhao, and Jian Peng. 2026. Ret-unet: Enhancing medical image segmentation with self-retention. Array, 29:100653. find
  27. Ayse Asu Guvenli and Bora Isildak. 2024. B-jet tagging with retentive networks: A novel approach and comparative study. arXiv preprint arXiv:2412.08134. — [arXiv:2412.08134](https://arxiv.org/abs/2412.08134) find
  28. Jia Han, Zhiyong Yu, and Jian Yang. 2025. Radio frequency-retentive network for automatic modulation classification. Electronics Letters, 61(1):e70203. find
  29. Zhu Han, Shuyi Xu, Lianru Gao, Zhi Li, and Bing Zhang. 2024. Gretnet: Gaussian retentive network for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters. find
  30. Ali Hatamizadeh, Michael Ranzinger, and Jan Kautz. 2023a. Vir: Vision retention networks. arXiv. org. find
  31. Ali Hatamizadeh, Michael Ranzinger, Shiyi Lan, Jose M Alvarez, Sanja Fidler, and Jan Kautz. 2023b. Vir: Towards efficient vision retention backbones. arXiv preprint arXiv:2310.19731. — [arXiv:2310.19731](https://arxiv.org/abs/2310.19731) find
  32. Wei He, Kai Han, Yehui Tang, Chengcheng Wang, Yujie Yang, Tianyu Guo, and Yunhe Wang. 2024. Densemamba: State space models with dense hidden connection for efficient large language models. arXiv preprint arXiv:2403.00818. — [arXiv:2403.00818](https://arxiv.org/abs/2403.00818) find
  33. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  34. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735– 1780. find
  35. Jia Cheng Hu, Roberto Cavicchioli, and Alessandro Capotondi. 2024a. Shifted window fourier transform and retention for image captioning. arXiv preprint arXiv:2408.13963. — [arXiv:2408.13963](https://arxiv.org/abs/2408.13963) find
  36. Jingjing Hu, Dan Guo, Kun Li, Zhan Si, Xun Yang, and Meng Wang. 2024b. Maskable retentive network for video moment retrieval. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1476–1485. find
  37. Jianan Huang, Xuebing Liu, Qing Zhu, Yaonan Wang, Mingtao Feng, Jiaming Zhou, Zhen Zhou, Lin Chen, and Danwei Wang. 2025a. Rampgrasp: Retentive attention-based multiscale perception grasp detection network. IEEE Transactions on Circuits and Systems for Video Technology. find
  38. Jingjia Huang, Jingyan Tu, Ge Meng, Yingying Wang, Yuhang Dong, Xiaotong Tu, Xinghao Ding, and Yue Huang. 2024a. Efficient perceiving local details via adaptive spatial-frequency information integration for multi-focus image fusion. In Proceedings of the 32nd ACM International Conference on Multimed… find
  39. Kai-Wei Huang and Chia-Ping Chen. 2024. Long audio file speaker diarization with feasible end-to-end models. In 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 1–6. IEEE. find
  40. Qihe Huang, Zhengyang Zhou, Kuo Yang, Gengyu Lin, Zhongchao Yi, and Yang Wang. 2024b. Leret: Language-empowered retentive network for time series forecasting. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24. find
  41. Yuyao Huang, Kai Chen, Wei Tian, and Lu Xiong. 2025b. Boost query-centric network efficiency for multi-agent motion forecasting. IEEE Robotics and Automation Letters. find
  42. Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. 2020. Transformers are rnns: Fast autoregressive transformers with linear attention. In International conference on machine learning, pages 5156–5165. PMLR. find
  43. Sangwon Kim, In-su Jang, and Byoung Chul Ko. 2024. Domain-free fire detection using the spatial–temporal attention transform of the yolo backbone. Pattern Analysis and Applications, 27(2):45. find
  44. Bin Lei, Caiwen Ding, and 1 others. 2023. Flashvideo: A framework for swift inference in text-to-video generation. arXiv preprint arXiv:2401.00869. — [arXiv:2401.00869](https://arxiv.org/abs/2401.00869) find
  45. Sihan Li and Juhua Huang. 2025. Resgdanet: An efficient residual group attention neural network for medical image classification. Applied Sciences, 15(5):2693. find
  46. Xing Li and Yuequan Bao. Adaptive gated meta graph retention network: A model for urban traffic flow prediction. Available at SSRN 5170149. find
  47. Zhiyuan Li, Yi Chang, and Yuan Wu. 2025. Segret: An efficient design for semantic segmentation with retentive network. arXiv preprint arXiv:2502.14014. — [arXiv:2502.14014](https://arxiv.org/abs/2502.14014) find
  48. Zhiyuan Li, Tingyu Xia, Yi Chang, and Yuan Wu. 2024. A survey of rwkv. arXiv preprint arXiv:2412.14847. — [arXiv:2412.14847](https://arxiv.org/abs/2412.14847) find
  49. Di Liang and Xiaofei Li. 2024. Ls-eend: Long-form streaming end-to-end neural diarization with online attractor extraction. arXiv preprint arXiv:2410.06670. — [arXiv:2410.06670](https://arxiv.org/abs/2410.06670) find
  50. Ruixing Lin and Shunmei Piao. 2024. Change detection of building remote sensing images based on rmt-bit. In 2024 4th International Conference on Electronic Information Engineering and Computer Science (EIECS), pages 428–432. IEEE. find
  51. Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu. 2022. A survey of transformers. AI open, 3:111– 132. find
  52. Zhicheng Lin, Rongpu Cui, Limiao Ning, and Jian Peng. 2025a. Temporal features-fused vision retentive network for echocardiography image segmentation. Sensors, 25(6):1909. find
  53. Zhijie Lin, Zilong Zhu, Lingling Guo, Jingjing Chen, and Jiyi Wu. 2025b. Disease detection algorithm for tea health protection based on improved real-time detection transformer. Applied Sciences (2076-3417), 15(4). find
  54. Zhou Linhao, Zhong Shenghua, and Xiao Zhijiao. 2024. Discovering multi-relational integration for knowledge tracing with retentive networks. In Proceedings of the 2024 International Conference on Multimedia Retrieval, pages 960–968. find
  55. Li, and Shiliang Li. 2024. Gr-p k a: a message-passing neural network with retention mechanism for p k a prediction. Briefings in Bioinformatics, 25(5):bbae408. find
  56. Zhaoyi Min, Qianqian Xiao, Muhammad Abbas, and Duanjin Zhang. 2025. Retentive network-based time series anomaly detection in cyber-physical systems. Engineering Applications of Artificial Intelligence, 145:110215. find
  57. SIMONE MOSCO. 2023. Exploiting retentive networks in 3d lidar semantic segmentation. find
  58. Cheng Nian, Weiyi Zhang, Fasih Ud Din Farrukh, Liting Niu, Dapeng Jiang, Fei Chen, and Chun Zhang. 2024. A 77.79 gops/w retentive network fpga inference accelerator with optimized workload. In IECON 2024-50th Annual Conference of the IEEE Industrial Electronics Society, pages 1–7. IEEE. find
  59. Jiayuan Liu, Bo Zhou, Xue Wan, Yan Pan, Zicong Li, and Yuanbin Shao. 2025. Mar-vo: A match-and-refine framework for uav’s monocular visual odometry in planetary environments. IEEE Transactions on Geoscience and Remote Sensing. find
  60. Masashi Okada, Mayumi Komatsu, and Tadahiro Taniguchi. 2024. A contact model based on denoising diffusion to learn variable impedance control for contact-rich manipulation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7286–7293. IEEE. find
  61. Jing Liu, Donglin Jing, Yanyan Cao, Ying Wang, Chaoping Guo, Peijun Shi, and Haijing Zhang. 2024a. Lightweight progressive fusion calibration network for rotated object detection in remote sensing images. Electronics, 13(16):3172. find
  62. Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, and 1 others. 2023. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048. find
  63. Zhe Liu, Jinghua Hou, Xinyu Wang, Xiaoqing Ye, Jingdong Wang, Hengshuang Zhao, and Xiang Bai. 2024b. Lion: Linear group rnn for 3d object detection in point clouds. Advances in Neural Information Processing Systems, 37:13601–13626. find
  64. Lihong Peng, Xin Liu, Min Chen, Wen Liao, Jiale Mao, and Liqian Zhou. 2024. Mgndti: A drug-target interaction prediction framework based on multimodal representation learning and the gating mechanism. Journal of Chemical Information and Modeling, 64(16):6684–6698. find
  65. Ziyu Liu, Yi Shen, Yunliang Jiang, Hancan Zhu, Hailong Hu, Yanlei Kang, Ming Chen, and Zhong Li. 2024c. Variation and evolution analysis of sars-cov-2 using self-game sequence optimization. Frontiers in Microbiology, 15:1485748. find
  66. Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y. Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, and Christopher Ré. 2023. Hyena hierarchy: Towards larger convolutional language models. Preprint, arXiv:2302.10866. find
  67. Baichao Long, Wang Zhu, and Jianli Xiao. 2024. St-retnet: A long-term spatial-temporal traffic flow prediction method. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pages 3– 16. Springer. find
  68. Zhen Qin, Songlin Yang, and Yiran Zhong. 2023. Hierarchically gated recurrent neural network for sequence modeling. Advances in Neural Information Processing Systems, 36:33202–33221. find
  69. Junwei Luo, Jiaojiao Wang, Jingjing Wei, Chaokun Yan, and Huimin Luo. 2025. Deephapnet: a haplotype assembly method based on retnet and deep spectral clustering. Briefings in Bioinformatics, 26(1):bbae656. find
  70. Yufan Qiu, Yaping Liu, and Shuo Zhang. 2024. Rn-ete: A retentive network-based encryption traffic encoder. In Proceedings of the 2024 3rd International Conference on Cryptography, Network Security and Communication Technology, pages 214–219. find
  71. Omayma Mahjoub, Sasha Abramowitz, Ruan de Kock, Wiem Khlifi, Simon du Toit, Jemma Daniel, Louay Ben Nessir, Louise Beyers, Claude Formanek, Liam Clark, and Arnu Pretorius. 2025. Sable: a performant, efficient and scalable sequence model for marl. Preprint, arXiv:2410.01706. — [arXiv:2410.01706](https://arxiv.org/abs/2410.01706) find
  72. Runyu Miao, Danlin Liu, Liyun Mao, Xingyu Chen, Leihao Zhang, Zhen Yuan, Shanshan Shi, Honglin Prajit Ramachandran, Barret Zoph, and Quoc V Le. 2017. Swish: a self-gated activation function. arXiv preprint arXiv:1710.05941, 7(1):5. find
  73. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053. find
  74. Ruilin Su, Wanshan Zhang, and Dunhui Yu. 2024. Sequence-to-sequence multi-hop knowledge reasoning based on retentive network. In 2024 7th International Conference on Computer Information Science and Application Technology (CISAT), pages 360–366. IEEE. find
  75. Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. 2023. Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:2307.08621.
  76. Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, and Furu Wei. 2022. A length-extrapolatable transformer. arXiv preprint arXiv:2212.10554.
  77. Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, and Furu Wei. 2024. You only cache once: Decoder-decoder architectures for language models. Advances in Neural Information Processing Systems, 37:7339– 7361. find
  78. Paweł Tomiło. 2025. Retention mechanism based neural network model for measuring aircraft landing distance. In 2025 IEEE 12th International Workshop on Metrology for AeroSpace (MetroAeroSpace), pages 321–326. IEEE. find
  79. Paweł Tomiło, Jan Laskowski, and Agnieszka Laskowska. 2025. Artificial neural network model based on kolmogorov-arnold representation theorem and retention mechanism for real-time aircraft flight phases classification. Engineering Applications of Artificial Intelligence, 160:112004. find
  80. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  81. Siyu Wang, Xiaocong Chen, and Lina Yao. 2024b. Retentive decision transformer with adaptive masking for reinforcement learning based recommendation systems. arXiv preprint arXiv:2403.17634. — [arXiv:2403.17634](https://arxiv.org/abs/2403.17634) find
  82. Zeyu Wang, Libo Zhao, Jizheng Zhang, Rui Song, Haiyu Song, Jiana Meng, and Shidong Wang. 2025c. Multi-text guidance is important: Multi-modality image fusion via large generative vision-language model. International Journal of Computer Vision, pages 1–23. find
  83. Tengqing Wu. 2024. A diffusion data enhancement retentive model for sequential recommendation. In 2024 7th International Conference on Computer Information Science and Application Technology (CISAT), pages 114–118. IEEE. find
  84. Yuxin Wu and Kaiming He. 2018. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3–19. find
  85. Yunyang Xie, Kai Chen, Shenghui Li, Bingqian Li, and Ning Zhang. 2024a. Uarc: Unsupervised anomalous traffic detection with improved u-shaped autoencoder and retnet based multi-clustering. In International Conference on Information and Communications Security, pages 187–207. Springer. find
  86. Zexun Xie, Min Xu, Shudong Zhang, and Lijuan Zhou. 2024b. Rcat: Retentive clip adapter tuning for improved video recognition. Electronics, 13(5):965. find
  87. Yimo Yan, Songyi Cui, Jiahui Liu, Yaping Zhao, Bodong Zhou, and Yong-Hong Kuo. 2025. Multimodal fusion for large-scale traffic prediction with heterogeneous retentive networks. Information Fusion, 114:102695. find
  88. Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim. 2024a. Gated linear attention transformers with hardware-efficient training. Preprint, arXiv:2312.06635. find
  89. Baoquan Wang, Yan Zeng, and Dongming Feng. 2025a. Deep learning-based damage assessment of hinge joints for multi-girder bridges utilizing vehicle-induced bridge responses. Engineering Structures, 333:120148. find
  90. Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim. 2025. Parallelizing linear transformers with the delta rule over sequence length. Preprint, arXiv:2406.06484. find
  91. Bin Wang, Fei Deng, and Peifan Jiang. 2024a. Eegdir: Electroencephalogram denoising network for temporal information storage and global modeling through retentive network. Computers in Biology and Medicine, 177:108626. find
  92. Wenkui Yang, Zhida Zhang, Xiaoqiang Zhou, Junxian Duan, and Jie Cao. 2024b. Tt-df: A large-scale diffusion-based dataset and benchmark for human body forgery detection. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pages 429–443. Springer. find
  93. Junliang Wang, Wenlong Hang, Shuang Liang, Qiong Wang, Badong Chen, and Jing Qin. 2025b. Convolutional retentive network for eeg decoding. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  94. Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768. find
  95. Xuan Yang, Yunxuan Dong, Lina Yang, and Thomas Wu. 2024c. Short-term photovoltaic forecasting model for qualifying uncertainty during hazy weather. arXiv preprint arXiv:2407.19663. — [arXiv:2407.19663](https://arxiv.org/abs/2407.19663) find
  96. Xuan Yang, Tao Peng, Haijia Bi, and Jiayu Han. 2024d. Span-level bidirectional retention scheme for aspect sentiment triplet extraction. Information Processing & Management, 61(5):103823. find
  97. Yuansong Zeng, Jiancong Xie, Zhuoyi Wei, Yun Su, Ningyuan Shangguan, Shuangyu Yang, Chengyang Zhang, Wenbing Li, Jinbo Zhang, Nan Fang, and 1 others. 2024. Cellfm: a large-scale foundation model pre-trained on transcriptomics of 100 million human cells. bioRxiv, pages 2024–06. find
  98. Fanlong Zhang, Huanming Chen, Quan Chen, and Jianqi Liu. 2025a. Cloud software code generation via knowledge graphs and multi-modal learning. find
  99. Lingzhe Zhang, Zewen Xiao, and Huaiyuan Wang. 2024a. Transient stability assessment of power system based on time-adapative retnet. In 2024 3rd Asia Power and Electrical Technology Conference (APET), pages 391–396. IEEE. find
  100. Tianning Zhang, Feng Liu, Yuming Yuan, Rui Su, Wanli Ouyang, and Lei Bai. 2024b. Fast information streaming handler (fish): A unified seismic neural network for single station real-time earthquake early warning. arXiv preprint arXiv:2408.06629. — [arXiv:2408.06629](https://arxiv.org/abs/2408.06629) find
  101. Yuxuan Zhang, Zipeng Zhang, Weiwei Guo, Wei Chen, Zhaohai Liu, and Houguang Liu. 2025b. Lretunet: A u-net-based retentive network for single-channel speech enhancement. Computer Speech & Language, page 101798. find
  102. Zipeng Zhang, Zhencai Zhu, Bin Meng, Zheng Yang, Mingke Wu, Xinyu Cheng, Binhong Li, and Houguang Liu. 2025c. Intelligent coal gangue identification: A novel amplitude frequency sensitive neural network. Expert Systems with Applications, 274:126880. find
  103. Kaili Zheng, Feixiang Lu, Yihao Lv, Liangjun Zhang, Chenyi Guo, and Ji Wu. 2024. 3d human pose estimation via non-causal retentive networks. In European Conference on Computer Vision, pages 111– 128. Springer. find
  104. Li Zhou, Dayang Wang, Yongshun Xu, Shuo Han, Bahareh Morovati, Shuyi Fan, and Hengyong Yu. 2024. Gradient guided co-retention feature pyramid network for ldct image denoising. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 153–163. Springer. find
  105. Wang Zhu, Baichao Long, and Jianli Xiao. 2024. Spatial-temporal retentive heterogeneous graph convolutional network for traffic flow prediction. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE. find
  106. Zijie Zhu, Feng Ding, Chenglong Chu, and Fangming Zhong. 2025. Retention enhanced cross-modal attention for multi-hop vqa. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  107. Zhongliang Zou, Shuzhen Yang, Mansheng Wang, and Bo Song. 2025. Multilevel retentive networks with edge enhanced attention for infrastructure crack detection. Engineering Structures, 343:121191. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-survey-retentive-network/