Elamur Research⌕
☰
home / papers / BERT: Pre-training of Deep Bidirectional… / cited by / page 3

Who cites «BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding»

23,422 papers of our library cite it (of the 140,884 whose bibliographies we read) · 1500 with a page here, the most cited first · page 3 of 15 · 201–300 · ← the paper

1 2 3 4 … 15

  1. Passage Re-ranking with BERTRodrigo Nogueira, Kyunghyun Cho2019 · 667 citations
  2. BAE: BERT-based Adversarial Examples for Text ClassificationSiddhant Garg, Goutham Ramakrishnan2020 · 665 citations
  3. KILT: a Benchmark for Knowledge Intensive Language TasksFabio Petroni, Aleksandra Piktus, Angela Fan, et al.2020 · 663 citations
  4. K-Adapter: Infusing Knowledge into Pre-Trained Models with AdaptersRuize Wang, Duyu Tang, Nan Duan, et al.2020 · 662 citations
  5. Efficient Content-Based Sparse Attention with Routing TransformersAurko Roy, Mohammad Saffar, Ashish Vaswani, et al.2020 · 660 citations
  6. CCNet: Extracting High Quality Monolingual Datasets from Web Crawl DataGuillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, et al.2020 · 650 citations
  7. ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation TransferYuanmeng Yan, Rumei Li, Sirui Wang, et al.2021 · 647 citations
  8. CodeSearchNet Challenge: Evaluating the State of Semantic Code SearchHamel Husain, Ho-Hsiang Wu, Tiferet Gazit, et al.2020 · 647 citations
  9. A Survey on Automated Fact-CheckingZhijiang Guo, Michael Schlichtkrull, Andreas Vlachos2022 · 646 citations
  10. CoCa: Contrastive Captioners are Image-Text Foundation ModelsJiahui Yu, Zirui Wang, Vijay Vasudevan, et al.2022 · 646 citations
  11. FLAIR: An Easy-to-Use Framework for State-of-the-Art NLPAlan Akbik, Tanja Bergmann, Duncan Blythe, et al.2019 · 645 citations
  12. A Corpus for Reasoning About Natural Language Grounded in Photographs (NLVR2)Alane Suhr, Stephanie Zhou, Ally Zhang, et al.2019 · 640 citations
  13. A Frustratingly Easy Approach for Entity and Relation ExtractionZexuan Zhong, Danqi Chen2020 · 629 citations
  14. Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and BeyondMikel Artetxe, Holger Schwenk2018 · 627 citations
  15. LayoutLMv2: Multi-modal Pre-training for Visually-rich Document UnderstandingYang Xu, Yiheng Xu, Tengchao Lv, et al.2021 · 623 citations
  16. Scalable Zero-shot Entity Linking with Dense Entity RetrievalLedell Wu, Fabio Petroni, Martin Josifoski, et al.2020 · 618 citations
  17. BloombergGPT: A Large Language Model for FinanceShijie Wu, Ozan Irsoy, Steven Lu, et al.2023 · 608 citations
  18. Role-Play with Large Language ModelsMurray Shanahan, Kyle McDonell, Laria Reynolds2023 · 606 citations
  19. Towards a Unified View of Parameter-Efficient Transfer LearningJunxian He, Chunting Zhou, Xuezhe Ma, et al.2021 · 604 citations
  20. Event Extraction by Answering (Almost) Natural QuestionsXinya Du, Claire Cardie2020 · 602 citations
  21. A Joint Neural Model for Information Extraction with Global FeaturesYing Lin, Heng Ji, Fei Huang, et al.2020 · 600 citations
  22. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language ModelShaden Smith, Mostofa Patwary, Brandon Norick, et al.2022 · 595 citations
  23. Text Embeddings by Weakly-Supervised Contrastive Pre-trainingLiang Wang, Nan Yang, Xiaolong Huang, et al.2022 · 594 citations
  24. FNet: Mixing Tokens with Fourier TransformsJames Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.2022 · 593 citations
  25. WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning RepresentationsMohammad Taher Pilehvar, Jose Camacho-Collados2019 · 592 citations
  26. The Flan Collection: Designing Data and Methods for Effective Instruction TuningShayne Longpre, Le Hou, Tu Vu, et al.2023 · 590 citations
  27. A Robustly Optimized BERT Pre-training Approach with Post-trainingLiu Zhuang, Lin Wayne, Shi Ya, et al.2021 · 585 citations
  28. Measuring Bias in Contextualized Word RepresentationsKeita Kurita, Nidhi Vyas, Ayush Pareek, et al.2019 · 584 citations
  29. PAWS: Paraphrase Adversaries from Word ScramblingYuan Zhang, Jason Baldridge, Luheng He2019 · 584 citations
  30. Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text ModelsJianmo Ni, Gustavo Hernandez Abrego, Noah Constant, et al.2021 · 582 citations
  31. What do you learn from context? Probing for sentence structure in contextualized word representationsIan Tenney, Patrick Xia, Berlin Chen, et al.2019 · 582 citations
  32. OCR-free Document Understanding TransformerGeewook Kim, Teakgyu Hong, Moonbin Yim, et al.2021 · 580 citations
  33. PhoBERT: Pre-trained language models for VietnameseDat Quoc Nguyen, Anh Tuan Nguyen2020 · 571 citations
  34. AudioLM: a Language Modeling Approach to Audio GenerationZalan Borsos, Raphael Marinier, Damien Vincent, et al.2022 · 570 citations
  35. Extractive Summarization as Text MatchingMing Zhong, Pengfei Liu, Yiran Chen, et al.2020 · 567 citations
  36. Learning How to Ask: Querying LMs with Mixtures of Soft PromptsGuanghui Qin, Jason Eisner2021 · 566 citations
  37. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, et al.2023 · 564 citations
  38. AGIEval: A Human-Centric Benchmark for Evaluating Foundation ModelsWanjun Zhong, Ruixiang Cui, Yiduo Guo, et al.2024 · 563 citations
  39. MIND: A Large-scale Dataset for News RecommendationFangzhao Wu, Ying Qiao, Jiun-Hung Chen, et al.2020 · 562 citations
  40. Leveraging Pre-trained Checkpoints for Sequence Generation TasksSascha Rothe, Shashi Narayan, Aliaksei Severyn2019 · 562 citations
  41. Plug and Play Language Models: A Simple Approach to Controlled Text GenerationSumanth Dathathri, Andrea Madotto, Janice Lan, et al.2019 · 562 citations
  42. Neural Legal Judgment Prediction in EnglishIlias Chalkidis, Ion Androutsopoulos, Nikolaos Aletras2019 · 558 citations
  43. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsLonghui Yu, Weisen Jiang, Han Shi, et al.2023 · 556 citations
  44. Massively Multilingual Neural Machine Translation in the Wild: Findings and ChallengesNaveen Arivazhagan, Ankur Bapna, Orhan Firat, et al.2019 · 555 citations
  45. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual GeneralizationJunjie Hu, Sebastian Ruder, Aditya Siddhant, et al.2020 · 551 citations
  46. Reducing Transformer Depth on Demand with Structured DropoutAngela Fan, Edouard Grave, Armand Joulin2019 · 546 citations
  47. Precise Zero-Shot Dense Retrieval without Relevance LabelsLuyu Gao, Xueguang Ma, Jimmy Lin, et al.2022 · 545 citations
  48. BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment AnalysisHu Xu, Bing Liu, Lei Shu, et al.2019 · 544 citations
  49. What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical ExamsDi Jin, Eileen Pan, Nassim Oufattole, et al.2020 · 540 citations
  50. End-to-End Open-Domain Question Answering with BERTseriniWei Yang, Yuqing Xie, Aileen Lin, et al.2019 · 539 citations
  51. Document Ranking with a Pretrained Sequence-to-Sequence ModelRodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, et al.2020 · 538 citations
  52. Cosmos QA: Machine Reading Comprehension with Contextual Commonsense ReasoningLifu Huang, Ronan Le Bras, Chandra Bhagavatula, et al.2019 · 536 citations
  53. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-TuningArmen Aghajanyan, Sonal Gupta, Luke Zettlemoyer2021 · 530 citations
  54. The Second Conversational Intelligence Challenge (ConvAI2)Emily Dinan, Varvara Logacheva, Valentin Malykh, et al.2019 · 525 citations
  55. Simple, Scalable Adaptation for Neural Machine TranslationAnkur Bapna, Orhan Firat2019 · 524 citations
  56. Evaluating Models' Local Decision Boundaries via Contrast SetsMatt Gardner, Yoav Artzi, Victoria Basmov, et al.2020 · 520 citations
  57. How Does NLP Benefit Legal System: A Summary of Legal Artificial IntelligenceHaoxi Zhong, Chaojun Xiao, Cunchao Tu, et al.2020 · 518 citations
  58. Instruction Tuning for Large Language Models: A SurveyShengyu Zhang, Linfeng Dong, Xiaoya Li, et al.2023 · 515 citations
  59. Probing Classifiers: Promises, Shortcomings, and AdvancesYonatan Belinkov2022 · 513 citations
  60. DeCLUTR: Deep Contrastive Learning for Unsupervised Textual RepresentationsJohn Giorgi, Osvald Nitski, Bo Wang, et al.2021 · 513 citations
  61. FinQA: A Dataset of Numerical Reasoning over Financial DataZhiyu Chen, Wenhu Chen, Charese Smiley, et al.2021 · 510 citations
  62. SPLADE: Sparse Lexical and Expansion Model for First Stage RankingThibault Formal, Benjamin Piwowarski, Stéphane Clinchant2021 · 510 citations
  63. Assessing BERT's Syntactic AbilitiesYoav Goldberg2019 · 509 citations
  64. TabFact: A Large-scale Dataset for Table-based Fact VerificationWenhu Chen, Hongmin Wang, Jianshu Chen, et al.2019 · 508 citations
  65. Gender Bias in Contextualized Word EmbeddingsJieyu Zhao, Tianlu Wang, Mark Yatskar, et al.2019 · 503 citations
  66. Sparse, Dense, and Attentional Representations for Text RetrievalYi Luan, Jacob Eisenstein, Kristina Toutanova, et al.2021 · 500 citations
  67. Few-shot Learning with Multilingual Language ModelsXi Victoria Lin, Todor Mihaylov, Mikel Artetxe, et al.2021 · 496 citations
  68. A-OKVQA: A Benchmark for Visual Question Answering using World KnowledgeDustin Schwenk, Apoorv Khandelwal, Christopher Clark, et al.2022 · 490 citations
  69. Language Models are Multilingual Chain-of-Thought ReasonersFreda Shi, Mirac Suzgun, Markus Freitag, et al.2022 · 490 citations
  70. Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data TasksJason Phang, Thibault Févry, Samuel R. Bowman2018 · 490 citations
  71. BERT for Coreference Resolution: Baselines and AnalysisMandar Joshi, Omer Levy, Luke Zettlemoyer, et al.2019 · 488 citations
  72. DeeBERT: Dynamic Early Exiting for Accelerating BERT InferenceJi Xin, Raphael Tang, Jaejun Lee, et al.2020 · 480 citations
  73. Null It Out: Guarding Protected Attributes by Iterative Nullspace ProjectionShauli Ravfogel, Yanai Elazar, Hila Gonen, et al.2020 · 480 citations
  74. ToTTo: A Controlled Table-To-Text Generation DatasetAnkur Parikh, Xuezhi Wang, Sebastian Gehrmann, et al.2020 · 479 citations
  75. ARBERT & MARBERT: Deep Bidirectional Transformers for ArabicMuhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi2021 · 478 citations
  76. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late InteractionKeshav Santhanam, Omar Khattab, Jon Saad-Falcon, et al.2022 · 477 citations
  77. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability CurvatureEric Mitchell, Yoonho Lee, Alexander Khazatsky, et al.2023 · 474 citations
  78. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timeMitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, et al.2022 · 474 citations
  79. GLaM: Efficient Scaling of Language Models with Mixture-of-ExpertsNan Du, Yanping Huang, Andrew M. Dai, et al.2021 · 474 citations
  80. Data Augmentation using Pre-trained Transformer ModelsVarun Kumar, Ashutosh Choudhary, Eunah Cho2020 · 468 citations
  81. PPT: Pre-trained Prompt Tuning for Few-shot LearningYuxian Gu, Xu Han, Zhiyuan Liu, et al.2022 · 467 citations
  82. Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTShijie Wu, Mark Dredze2019 · 466 citations
  83. XCOPA: A Multilingual Dataset for Causal Commonsense ReasoningEdoardo Maria Ponti, Goran Glavaš, Olga Majewska, et al.2020 · 465 citations
  84. P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and TasksXiao Liu, Kaixuan Ji, Yicheng Fu, et al.2021 · 464 citations
  85. Masked Autoencoders Are Scalable Vision LearnersKaiming He, Xinlei Chen, Saining Xie, et al.2021 · 463 citations
  86. Text-to-SQL Empowered by Large Language Models: A Benchmark EvaluationDawei Gao, Haibin Wang, Yaliang Li, et al.2023 · 461 citations
  87. From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual TransformersAnne Lauscher, Vinit Ravishankar, Ivan Vulić, et al.2020 · 461 citations
  88. Defending Against Neural Fake NewsRowan Zellers, Ari Holtzman, Hannah Rashkin, et al.2019 · 461 citations
  89. SimVLM: Simple Visual Language Model Pretraining with Weak SupervisionZirui Wang, Jiahui Yu, Adams Wei Yu, et al.2021 · 459 citations
  90. Are All Languages Created Equal in Multilingual BERT?Shijie Wu, Mark Dredze2020 · 458 citations
  91. Exploring Pre-trained Language Models for Event Extraction and GenerationSen Yang, Dawei Feng, Linbo Qiao, et al.2019 · 454 citations
  92. LinkBERT: Pretraining Language Models with Document LinksMichihiro Yasunaga, Jure Leskovec, Percy Liang2022 · 452 citations
  93. OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning FrameworkPeng Wang, An Yang, Rui Men, et al.2022 · 452 citations
  94. ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-trainingWeizhen Qi, Yu Yan, Yeyun Gong, et al.2020 · 451 citations
  95. Inherent Disagreements in Human Textual InferencesEllie Pavlick, Tom Kwiatkowski2019 · 451 citations
  96. Large language models are few-shot clinical information extractorsMonica Agrawal, Stefan Hegselmann, Hunter Lang, et al.2022 · 448 citations
  97. Show your work: Scratchpads for intermediate computation with language models2021 · 448 citations
  98. LLM-Pruner: On the Structural Pruning of Large Language ModelsXinyin Ma, Gongfan Fang, Xinchao Wang2023 · 447 citations
  99. Quality at a Glance: An Audit of Web-Crawled Multilingual DatasetsJulia Kreutzer, Isaac Caswell, Lisa Wang, et al.2021 · 447 citations
  100. HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document SummarizationXingxing Zhang, Furu Wei, Ming Zhou2019 · 446 citations

1 2 3 4 … 15

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/cited-by/page/3/