Elamur Research⌕
☰
home / papers / RoBERTa: A Robustly Optimized BERT Pretraining… / cited by / page 3

Who cites «RoBERTa: A Robustly Optimized BERT Pretraining Approach»

9,837 papers of our library cite it (of the 149,439 whose bibliographies we read) · 1406 with a page here, the most cited first · page 3 of 15 · 201–300 · ← back to «RoBERTa: A Robustly Optimized BERT Pretraining Approach»

1 2 3 4 … 10 … 15

  1. LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionarXiv:2303.16199Renrui Zhang, Jiaming Han, Chris Liu, et al.2023 · 333 citations
  2. SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment AnalysisAnnual Meeting of the Association for Computational Linguistics (ACL)Hao Tian, Can Gao, Xinyan Xiao, et al.2020 · 333 citations
  3. ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and GenerationarXiv:2107.02137Yu Sun, Shuohuan Wang, Shikun Feng, et al.2021 · 332 citations
  4. GraphCodeBERT: Pre-training Code Representations with Data FlowarXiv:2009.08366Daya Guo, Shuo Ren, Shuai Lu, et al.2021 · 332 citations
  5. TimeLMs: Diachronic Language Models from TwitterAnnual Meeting of the Association for Computational Linguistics (ACL)Daniel Loureiro, Francesco Barbieri, Leonardo Neves, et al.2022 · 331 citations
  6. Language Is Not All You Need: Aligning Perception with Language ModelsConference on Neural Information Processing Systems (NeurIPS)Shaohan Huang, Li Dong, Wenhui Wang, et al.2023 · 330 citations
  7. Unsupervised Domain Clusters in Pretrained Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Roee Aharoni, Yoav Goldberg2020 · 330 citations
  8. Unsupervised Corpus Aware Language Model Pre-training for Dense Passage RetrievalAnnual Meeting of the Association for Computational Linguistics (ACL)Luyu Gao, Jamie Callan2022 · 328 citations
  9. Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate DetectionAnnual Meeting of the Association for Computational Linguistics (ACL)Bertie Vidgen, Tristan Thrush, Zeerak Waseem, et al.2021 · 325 citations
  10. LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative FusionAnnual Meeting of the Association for Computational Linguistics (ACL)Dongfu Jiang, Xiang Ren, Bill Yuchen Lin2023 · 321 citations
  11. How many data points is a prompt worth?Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)Teven Le Scao, Alexander Rush2021 · 320 citations
  12. Probing Pretrained Language Models for Lexical SemanticsConference on Empirical Methods in Natural Language Processing (EMNLP)Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, et al.2020 · 320 citations
  13. WANLI: Worker and AI Collaboration for Natural Language Inference Dataset CreationarXiv:2201.05955Alisa Liu, Swabha Swayamdipta, Noah A. Smith, et al.2022 · 317 citations
  14. Double Graph Based Reasoning for Document-level Relation ExtractionConference on Empirical Methods in Natural Language Processing (EMNLP)Shuang Zeng, Runxin Xu, Baobao Chang, et al.2020 · 314 citations
  15. Explainable Automated Fact-Checking for Public Health ClaimsConference on Empirical Methods in Natural Language Processing (EMNLP)Neema Kotonya, Francesca Toni2020 · 314 citations
  16. A Transformer-based approach to Irony and Sarcasm detectionNeural Computing and ApplicationsRolandos Alexandros Potamias, Georgios Siolas, Andreas - Georgios Stafylopatis2019 · 312 citations
  17. S2ORC: The Semantic Scholar Open Research CorpusarXiv:1911.02782Kyle Lo, Lucy Lu Wang, Mark Neumann, et al.2019 · 312 citations
  18. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Shangbin Feng, Chan Young Park, Yuhan Liu, et al.2023 · 311 citations
  19. Byte Pair Encoding is Suboptimal for Language Model PretrainingFindings of the Association for Computational LinguisticsKaj Bostrom, Greg Durrett2020 · 310 citations
  20. UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive LearningAnnual Meeting of the Association for Computational Linguistics (ACL)Wei Li, Can Gao, Guocheng Niu, et al.2021 · 308 citations
  21. Muppet: Massive Multi-task Representations with Pre-FinetuningConference on Empirical Methods in Natural Language Processing (EMNLP)Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, et al.2021 · 307 citations
  22. AugGPT: Leveraging ChatGPT for Text Data AugmentationIEEE Transactions on Big DataHaixing Dai; Zhengliang Liu; Wenxiong Liao; Xiaoke Huang; Yihan Cao; Zihao Wu; Lin Zhao; Shaochen Xu; Wei Liu; Ninghao Liu; Sheng Li; Dajiang Zhu; Hongmin Cai; Lichao Sun; Quanzheng Li; Dinggang Shen; Tianming Liu;, Xiang Li2023 · 306 citations
  23. Can AI-Generated Text be Reliably Detected?arXiv:2303.11156Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, et al.2023 · 306 citations
  24. SPoT: Better Frozen Model Adaptation through Soft Prompt TransferAnnual Meeting of the Association for Computational Linguistics (ACL)Tu Vu, Brian Lester, Noah Constant, et al.2022 · 306 citations
  25. SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized OptimizationAnnual Meeting of the Association for Computational Linguistics (ACL)Haoming Jiang, Pengcheng He, Weizhu Chen, et al.2020 · 304 citations
  26. Socratic Models: Composing Zero-Shot Multimodal Reasoning with LanguagearXiv:2204.00598Andy Zeng, Maria Attarian, Brian Ichter, et al.2022 · 302 citations
  27. Want To Reduce Labeling Cost? GPT-3 Can HelpFindings of the Association for Computational LinguisticsShuohang Wang, Yang Liu, Yichong Xu, et al.2021 · 300 citations
  28. Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question AnsweringConference on Empirical Methods in Natural Language Processing (EMNLP)Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, et al.2020 · 300 citations
  29. FlauBERT: Unsupervised Language Model Pre-training for FrencharXiv:1912.05372Hang Le, Loïc Vial, Jibril Frej, et al.2019 · 299 citations
  30. Entity-Based Knowledge Conflicts in Question AnsweringConference on Empirical Methods in Natural Language Processing (EMNLP)Shayne Longpre, Kartik Perisetla, Anthony Chen, et al.2021 · 298 citations
  31. Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for LittleConference on Empirical Methods in Natural Language Processing (EMNLP)Koustuv Sinha, Robin Jia, Dieuwke Hupkes, et al.2021 · 297 citations
  32. BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in BanglaFindings of the Association for Computational LinguisticsAbhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, et al.2022 · 296 citations
  33. AdapterDrop: On the Efficiency of Adapters in TransformersConference on Empirical Methods in Natural Language Processing (EMNLP)Andreas Rücklé, Gregor Geigle, Max Glockner, et al.2021 · 296 citations
  34. Text Classification via Large Language ModelsFindings of the Association for Computational LinguisticsXiaofei Sun, Xiaoya Li, Jiwei Li, et al.2023 · 292 citations
  35. QAFactEval: Improved QA-Based Factual Consistency Evaluation for SummarizationConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, et al.2022 · 292 citations
  36. A Review of Large Language Models and Autonomous Agents in ChemistryarXiv:2407.01603Mayk Caldas Ramos, Christopher J. Collison, Andrew D. White2024 · 288 citations
  37. PAQ: 65 Million Probably-Asked Questions and What You Can Do With ThemTransactions of the Association for Computational Linguistics (TACL)Patrick Lewis, Yuxiang Wu, Linqing Liu, et al.2021 · 288 citations
  38. Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence EncodersConference on Empirical Methods in Natural Language Processing (EMNLP)Jue Wang, Wei Lu2020 · 288 citations
  39. An Explanation of In-context Learning as Implicit Bayesian InferencearXiv:2111.02080Sang Michael Xie, Aditi Raghunathan, Percy Liang, et al.2021 · 286 citations
  40. ReClor: A Reading Comprehension Dataset Requiring Logical ReasoningarXiv:2002.04326Weihao Yu, Zihang Jiang, Yanfei Dong, et al.2020 · 286 citations
  41. USR: An Unsupervised and Reference Free Evaluation Metric for Dialog GenerationAnnual Meeting of the Association for Computational Linguistics (ACL)Shikib Mehri, Maxine Eskenazi2020 · 286 citations
  42. AlignScore: Evaluating Factual Consistency with A Unified Alignment FunctionAnnual Meeting of the Association for Computational Linguistics (ACL)Yuheng Zha, Yichi Yang, Ruichen Li, et al.2023 · 284 citations
  43. Spelling Error Correction with Soft-Masked BERTAnnual Meeting of the Association for Computational Linguistics (ACL)Shaohua Zhang, Haoran Huang, Jicong Liu, et al.2020 · 284 citations
  44. Contextualized Perturbation for Textual Adversarial AttackConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Dianqi Li, Yizhe Zhang, Hao Peng, et al.2021 · 283 citations
  45. Fine-grained Fact Verification with Kernel Graph Attention NetworkAnnual Meeting of the Association for Computational Linguistics (ACL)Zhenghao Liu, Chenyan Xiong, Maosong Sun, et al.2020 · 283 citations
  46. XTREME-R: Towards More Challenging and Nuanced Multilingual EvaluationConference on Empirical Methods in Natural Language Processing (EMNLP)Sebastian Ruder, Noah Constant, Jan Botha, et al.2021 · 281 citations
  47. Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?Annual Meeting of the Association for Computational Linguistics (ACL)Yada Pruksachatkun, Jason Phang, Haokun Liu, et al.2020 · 281 citations
  48. Large Language Models for Mathematical Reasoning: Progresses and ChallengesConference of the European Chapter of the Association for Computational Linguistics (EACL)Janice Ahn, Rishu Verma, Renze Lou, et al.2024 · 280 citations
  49. Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question AnsweringarXiv:1911.10470Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, et al.2019 · 280 citations
  50. OpenPrompt: An Open-source Framework for Prompt-learningAnnual Meeting of the Association for Computational Linguistics (ACL)Ning Ding, Shengding Hu, Weilin Zhao, et al.2022 · 279 citations
  51. TRUE: Re-evaluating Factual Consistency EvaluationConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Or Honovich, Roee Aharoni, Jonathan Herzig, et al.2022 · 279 citations
  52. A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future DirectionsComputational LinguisticsJunchao Wu, Shu Yang, Runzhe Zhan, et al.2025 · 277 citations
  53. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLProceedings of the AAAI Conference on Artificial IntelligenceHaoyang Li, Jing Zhang, Cuiping Li, et al.2023 · 277 citations
  54. Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, et al.2021 · 277 citations
  55. Graph Retrieval-Augmented Generation: A SurveyACM Transactions on Information SystemsBoci Peng, Yun Zhu, Yongchao Liu, et al.2024 · 276 citations
  56. Text Classification Using Label Names Only: A Language Model Self-Training ApproachConference on Empirical Methods in Natural Language Processing (EMNLP)Yu Meng, Yunyi Zhang, Jiaxin Huang, et al.2020 · 274 citations
  57. XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and BeyondLanguage Resources and Evaluation Conference (LREC)Francesco Barbieri, Luis Espinosa Anke, Jose Camacho-Collados2022 · 271 citations
  58. Towards Understanding and Mitigating Social Biases in Language ModelsarXiv:2106.13219Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, et al.2021 · 271 citations
  59. Attention is Not Only a Weight: Analyzing Transformers with Vector NormsConference on Empirical Methods in Natural Language Processing (EMNLP)Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, et al.2020 · 270 citations
  60. DiffCSE: Difference-based Contrastive Learning for Sentence EmbeddingsConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, et al.2022 · 269 citations
  61. ConveRT: Efficient and Accurate Conversational Representations from TransformersFindings of the Association for Computational LinguisticsMatthew Henderson, Iñigo Casanueva, Nikola Mrkšić, et al.2020 · 269 citations
  62. Injecting Numerical Reasoning Skills into Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Mor Geva, Ankit Gupta, Jonathan Berant2020 · 269 citations
  63. CUAD: An Expert-Annotated NLP Dataset for Legal Contract ReviewarXiv:2103.06268Dan Hendrycks, Collin Burns, Anya Chen, et al.2021 · 268 citations
  64. IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language UnderstandingConference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)Bryan Wilie, Karissa Vincentio, Genta Indra Winata, et al.2020 · 266 citations
  65. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval ModelsNeurIPS 2021 Datasets and BenchmarksNandan Thakur, Nils Reimers, Andreas Rücklé, et al.2021 · 265 citations
  66. Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic ParsingFindings of the Association for Computational LinguisticsXi Victoria Lin, Richard Socher, Caiming Xiong2020 · 263 citations
  67. How Can We Accelerate Progress Towards Human-like Linguistic Generalization?Annual Meeting of the Association for Computational Linguistics (ACL)Tal Linzen2020 · 263 citations
  68. LLaMA-VID: An Image is Worth 2 Tokens in Large Language ModelsLecture notes in computer scienceYanwei Li, Chengyao Wang, Jiaya Jia2023 · 262 citations
  69. The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language VariantsAnnual Meeting of the Association for Computational Linguistics (ACL)Lucas Bandarkar, Davis Liang, Benjamin Muller, et al.2023 · 262 citations
  70. TLDR: Extreme Summarization of Scientific DocumentsFindings of the Association for Computational LinguisticsIsabel Cachola, Kyle Lo, Arman Cohan, et al.2020 · 262 citations
  71. Towards Debiasing Sentence RepresentationsAnnual Meeting of the Association for Computational Linguistics (ACL)Paul Pu Liang, Irene Mengze Li, Emily Zheng, et al.2020 · 262 citations
  72. DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI ScalearXiv:2201.05596Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.2022 · 261 citations
  73. Condenser: a Pre-training Architecture for Dense RetrievalConference on Empirical Methods in Natural Language Processing (EMNLP)Luyu Gao, Jamie Callan2021 · 261 citations
  74. Fairseq S2T: Fast Speech-to-Text Modeling with FairseqConference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)Changhan Wang, Yun Tang, Xutai Ma, et al.2020 · 259 citations
  75. LexGLUE: A Benchmark Dataset for Legal Language Understanding in EnglisharXiv:2110.00976Ilias Chalkidis; Abhik Jana; Dirk Hartung; Michael Bommarito; Ion Androutsopoulos; Daniel Martin Katz;, Nikolaos Aletras2021 · 258 citations
  76. Parameter-efficient Multi-task Fine-tuning for Transformers via Shared HypernetworksAnnual Meeting of the Association for Computational Linguistics (ACL)Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, et al.2021 · 258 citations
  77. Multilingual is not enough: BERT for FinnisharXiv:1912.07076Antti Virtanen, Jenna Kanerva, Rami Ilo, et al.2019 · 258 citations
  78. Fine-Tuning LLaMA for Multi-Stage Text RetrievalInternational ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)Xueguang Ma, Liang Wang, Nan Yang, et al.2023 · 257 citations
  79. The Curse of Recursion: Training on Generated Data Makes Models ForgetarXiv:2305.17493Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, et al.2023 · 255 citations
  80. Generating Datasets with Pretrained Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Timo Schick, Hinrich Schutze2021 · 254 citations
  81. Analyzing Leakage of Personally Identifiable Information in Language ModelsProceedings - IEEE Symposium on Security and Privacy/Proceedings of the ... IEEE Symposium on Security and PrivacyNils Lukas, Ahmed Salem, Robert Sim, et al.2023 · 252 citations
  82. ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and ExplanationAnnual Meeting of the Association for Computational Linguistics (ACL)Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, et al.2021 · 251 citations
  83. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence FrontiersarXiv:2102.01454Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, et al.2021 · 251 citations
  84. COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted ListConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Luyu Gao, Zhuyun Dai, Jamie Callan2021 · 250 citations
  85. Interpreting Pretrained Contextualized Representations via Reductions to Static EmbeddingsAnnual Meeting of the Association for Computational Linguistics (ACL)Rishi Bommasani, Kelly Davis, Claire Cardie2020 · 250 citations
  86. Automatic Detection of Machine Generated Text: A Critical SurveyInternational Conference on Computational Linguistics (COLING)Ganesh Jawahar, Muhammad Abdul-Mageed, Laks Lakshmanan V.S.2020 · 247 citations
  87. E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTarXiv:1911.03681Nina Poerner, Ulli Waltinger, Hinrich Schutze2019 · 247 citations
  88. Knowledge-Grounded Dialogue Generation with Pre-trained Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Xueliang Zhao, Wei Wu, Can Xu, et al.2020 · 246 citations
  89. ASQA: Factoid Questions Meet Long-Form AnswersConference on Empirical Methods in Natural Language Processing (EMNLP)Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, et al.2022 · 245 citations
  90. ZeroGen: Efficient Zero-shot Learning via Dataset GenerationConference on Empirical Methods in Natural Language Processing (EMNLP)Jiacheng Ye, Jiahui Gao, Qintong Li, et al.2022 · 245 citations
  91. OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of GeneralizationarXiv:2212.12017Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, et al.2022 · 244 citations
  92. CLUE: A Chinese Language Understanding Evaluation BenchmarkarXiv:2004.05986Liang Xu, Hai Hu, Xuanwei Zhang, et al.2020 · 244 citations
  93. In-context Examples Selection for Machine TranslationFindings of the Association for Computational LinguisticsSweta Agrawal, Chunting Zhou, Mike Lewis, et al.2023 · 243 citations
  94. Hierarchical Graph Network for Multi-hop Question AnsweringConference on Empirical Methods in Natural Language Processing (EMNLP)Yuwei Fang, Siqi Sun, Zhe Gan, et al.2020 · 243 citations
  95. Synthetic Data Generation with Large Language Models for Text Classification: Potential and LimitationsConference on Empirical Methods in Natural Language Processing (EMNLP)Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, et al.2023 · 242 citations
  96. BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagearXiv:2208.03188Kurt Shuster, Jing Xu, Mojtaba Komeili, et al.2022 · 241 citations
  97. Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-ArtWorkshop on Clinical Natural Language Processing (Clinical NLP) @ LRECPatrick Lewis, Myle Ott, Jingfei Du, et al.2020 · 241 citations
  98. BERT-of-Theseus: Compressing BERT by Progressive Module ReplacingConference on Empirical Methods in Natural Language Processing (EMNLP)Canwen Xu, Wangchunshu Zhou, Tao Ge, et al.2020 · 240 citations
  99. Curriculum Learning for Natural Language UnderstandingAnnual Meeting of the Association for Computational Linguistics (ACL)Benfeng Xu, Licheng Zhang, Zhendong Mao, et al.2020 · 240 citations
  100. MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue SystemsConference on Empirical Methods in Natural Language Processing (EMNLP)Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, et al.2020 · 240 citations

1 2 3 4 … 10 … 15

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/3/