Elamur Research⌕
☰
home / papers / DistilBERT, a distilled version of BERT: smaller… / cited by / page 2

Who cites «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»

2,404 papers of our library cite it (of the 149,439 whose bibliographies we read) · 284 with a page here, the most cited first · page 2 of 3 · 101–200 · ← back to «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»

1 2 3

  1. Aspect-Controllable Opinion SummarizationConference on Empirical Methods in Natural Language Processing (EMNLP)Reinald Kim Amplayo, Stefanos Angelidis, Mirella Lapata2021 · 114 citations
  2. Certifying LLM Safety against Adversarial PromptingarXiv:2309.02705Aounon Kumar, Chirag Agarwal, Suraj Srinivas, et al.2023 · 113 citations
  3. Efficient Large Scale Language Modeling with Mixtures of ExpertsarXiv:2112.10684Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al.2021 · 113 citations
  4. DistiLLM: Towards Streamlined Distillation for Large Language ModelsarXiv:2402.03898Jongwoo Ko, Sungnyun Kim, Tianyi Chen, et al.2024 · 112 citations
  5. Large Language Models Enable Few-Shot ClusteringTransactions of the Association for Computational Linguistics (TACL)Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, et al.2024 · 111 citations
  6. Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to SystemsACM Computing SurveysXupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al.2023 · 110 citations
  7. Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense ReasoningAnnual Meeting of the Association for Computational Linguistics (ACL)Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, et al.2021 · 107 citations
  8. LEDGAR: A Large-Scale Multi-label Corpus for Text Classification of Legal Provisions in ContractsLanguage Resources and Evaluation Conference (LREC)Don Tuggener, Pius von Däniken, Thomas Peetz, et al.2020 · 107 citations
  9. LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale InstructionsarXiv:2304.14402Minghao Wu, Abdul Waheed, Chiyu Zhang, et al.2023 · 105 citations
  10. minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language ModelsarXiv:2203.13112Kanishka Misra2022 · 102 citations
  11. On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language RepresentationsAnnual Meeting of the Association for Computational Linguistics (ACL)Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, et al.2022 · 100 citations
  12. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language ModelsarXiv:2601.18734Siyan Zhao, Zhihui Xie, Mengchen Liu, et al.2026 · 97 citations
  13. CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language TechnologiesFindings of the Association for Computational LinguisticsWeiyan Shi, Ryan Li, Yutong Zhang, et al.2024 · 93 citations
  14. TransQuest: Translation Quality Estimation with Cross-lingual TransformersInternational Conference on Computational Linguistics (COLING)Tharindu Ranasinghe, Constantin Orasan, Ruslan Mitkov2020 · 91 citations
  15. Probing for idiomaticity in vector space modelsConference of the European Chapter of the Association for Computational Linguistics (EACL)Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, et al.2021 · 89 citations
  16. Do Membership Inference Attacks Work on Large Language Models?arXiv:2402.07841Michael Duan, Anshuman Suri, Niloofar Mireshghallah, et al.2024 · 87 citations
  17. Foundational Challenges in Assuring Alignment and Safety of Large Language ModelsarXiv:2404.09932Usman Anwar, Abulhair Saparov, Javier Rando, et al.2024 · 87 citations
  18. REST: Retrieval-Based Speculative DecodingarXiv:2311.08252Zhenyu He; Zexuan Zhong; Tianle Cai; Jason D. Lee;, Di He2023 · 87 citations
  19. Retrieval-Augmented Generation for Natural Language Processing: A SurveyArtificial Intelligence ReviewShangyu Wu, Ying Xiong, Yufei Cui, et al.2024 · 86 citations
  20. ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper ReviewingarXiv:2306.00622Ryan Liu, Nihar B. Shah2023 · 84 citations
  21. ORCA: A Challenging Benchmark for Arabic Language UnderstandingFindings of the Association for Computational LinguisticsAbdelRahim Elmadany, ElMoatez Billah Nagoudi, Muhammad Abdul-Mageed2023 · 83 citations
  22. Knowledge Fusion of Large Language ModelsarXiv:2401.10491Fanqi Wan, Xinting Huang, Deng Cai, et al.2024 · 82 citations
  23. Reinforcement Learning via Self-DistillationarXiv:2601.20802Jonas Hubotter, Frederike Lubeck, Lejs Behric, et al.2026 · 74 citations
  24. QuRating: Selecting High-Quality Data for Training Language ModelsarXiv:2402.09739Alexander Wettig, Aatmik Gupta, Saumya Malik, et al.2024 · 74 citations
  25. Structural Persistence in Language Models: Priming as a Window into Abstract Language RepresentationsTransactions of the Association for Computational Linguistics (TACL)Arabella Sinclair, Jaap Jumelet, Willem Zuidema, et al.2022 · 74 citations
  26. Annotation Error Detection: Analyzing the Past and Present for a More Coherent FutureJan-Christoph Klie, Bonnie Webber, Iryna Gurevych2023 · 73 citations
  27. ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic TextArabic Natural Language Processing Conference (ArabicNLP)Maram Hasanain, Firoj Alam, Hamdy Mubarak, et al.2023 · 67 citations
  28. FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning ModelsFindings of the Association for Computational LinguisticsShramay Palta, Rachel Rudinger2023 · 67 citations
  29. A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, et al.2023 · 65 citations
  30. It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story BooksBingsheng Yao, Dakuo Wang, Tongshuang Wu, et al.2022 · 64 citations
  31. Building a Bridge: A Method for Image-Text Sarcasm Detection Without Pretraining on Image-Text DataProceedings of the First International Workshop on Natural Language Processing Beyond TextXinyu Wang, Xiaowen Sun, Tan Yang, et al.2020 · 62 citations
  32. mmBERT: A Modern Multilingual Encoder with Annealed Language LearningarXiv:2509.06888Marc Marone, Orion Weller, William Fleshman, et al.2025 · 59 citations
  33. Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired SketchingConference on Empirical Methods in Natural Language Processing (EMNLP)Simon A. Aytes, Jinheon Baek, Sung Ju Hwang2025 · 51 citations
  34. From Decoding to Meta-Generation: Inference-time Algorithms for Large Language ModelsarXiv:2406.16838Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.2024 · 51 citations
  35. VariErr NLI: Separating Annotation Error from Human Label VariationAnnual Meeting of the Association for Computational Linguistics (ACL)Leon Weber-Genzel, Siyao Peng, Marie-Catherine De Marneffe, et al.2024 · 51 citations
  36. Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMRJuri Opitz, Anette Frank2021 · 51 citations
  37. Language models scale reliably with over-training and on downstream tasksarXiv:2403.08540Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, et al.2024 · 46 citations
  38. Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport DistillationarXiv:2301.12566Zhiqi Huang, Puxuan Yu, James Allan2023 · 46 citations
  39. F-coref: Fast, Accurate and Easy to Use Coreference ResolutionConference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)Shon Otmazgin, Arie Cattan, Yoav Goldberg2022 · 45 citations
  40. Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training LabelsarXiv:2406.17633Nicholas Pangakis, Samuel Wolken2024 · 42 citations
  41. DeepSeek: Paradigm Shifts and Technical Evolution in Large AI ModelsarXiv:2507.09955Luolin Xiong, Haofen Wang, Xi Chen, et al.2025 · 41 citations
  42. Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved SamplingarXiv:2410.11325Wenda Xu, Rujun Han, Zifeng Wang, et al.2024 · 41 citations
  43. Dual-Space Knowledge Distillation for Large Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Songming Zhang, Xue Zhang, Zengkui Sun, et al.2024 · 39 citations
  44. Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMsarXiv:2402.12030 · Transactions on Machine Learning Research, January 2025Nicolas Boizard, Kevin El Haddad, Celine Hudelot, et al.2024 · 35 citations
  45. Computational Politeness in Natural Language Processing: A SurveyACM Computing SurveysPriyanshu Priya, Mauajama Firdaus, Asif Ekbal2024 · 34 citations
  46. Controllable Summarization with Constrained Markov Decision ProcessTransactions of the Association for Computational Linguistics (TACL)Hou Pong Chan, Lu Wang, Irwin King2021 · 34 citations
  47. A Survey of Small Language ModelsarXiv:2410.20011Chien Van Nguyen, Xuan Shen, Ryan Aponte, et al.2024 · 31 citations
  48. Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective predictionWorkshop on Understanding Implicit and Underspecified LanguageMurathan Kurfalı, Robert Östling2021 · 31 citations
  49. Click: Controllable Text Generation with Sequence Likelihood Contrastive LearningFindings of the Association for Computational LinguisticsChujie Zheng, Pei Ke, Zheng Zhang, et al.2023 · 29 citations
  50. How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument StructureTransactions of the Association for Computational Linguistics (TACL)Michael Wilson, Jackson Petty, Robert Frank2023 · 29 citations
  51. One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial SupportarXiv:2306.09237Ronja Stern, Vishvaksenan Rasiah, Veton Matoshi, et al.2023 · 29 citations
  52. A Survey on Asking Clarification Questions Datasets in Conversational SystemsAnnual Meeting of the Association for Computational Linguistics (ACL)Hossein A. Rahmani, Xi Wang, Yue Feng, et al.2023 · 28 citations
  53. Context Matters: A Pragmatic Study of PLMs' Negation UnderstandingAnnual Meeting of the Association for Computational Linguistics (ACL)Reto Gubelmann, Siegfried Handschuh2022 · 28 citations
  54. Towards the Law of Capacity Gap in Distilling Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Chen Zhang, Qiuchi Li, Dawei Song, et al.2025 · 27 citations
  55. FuseChat: Knowledge Fusion of Chat ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Fanqi Wan, Longguang Zhong, Ziyi Yang, et al.2025 · 25 citations
  56. What is the Role of Small Models in the LLM Era: A SurveyComputational LinguisticsLihu Chen, Gael Varoquaux2024 · 23 citations
  57. Joint Prompt Optimization of Stacked LLMs using Variational InferenceConference on Neural Information Processing Systems (NeurIPS)Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, et al.2023 · 23 citations
  58. Exploring Retrieval Augmented Generation in ArabicProcedia Computer ScienceSamhaa R. El-Beltagy, Mohamed A. Abdallah2024 · 22 citations
  59. A Large-Scale English Multi-Label Twitter Dataset for Cyberbullying and Online Abuse DetectionWorkshop on Online Abuse and HarmsSemiu Salawu, Jo Lumsden, Yulan He2021 · 22 citations
  60. Semantics of Multiword Expressions in Transformer-Based Models: A SurveyTransactions of the Association for Computational Linguistics (TACL)Filip Miletić, Sabine Schulte im Walde2024 · 20 citations
  61. Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context ExpertsFindings of the Association for Computational LinguisticsNghia T. Le, Fan Bai, Alan Ritter2022 · 20 citations
  62. Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale PretrainingInternational Conference on Computational Linguistics (COLING)Grigorii Guz, Patrick Huber, Giuseppe Carenini2020 · 19 citations
  63. Does Commonsense help in detecting Sarcasm?Workshop on Insights from Negative Results in NLPSomnath Basu Roy Chowdhury, Snigdha Chaturvedi2021 · 17 citations
  64. AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection DatasetarXiv:2105.03143Mohamed Seghir Hadj Ameur, Hassina Aliane2021 · 16 citations
  65. When Truth Matters - Addressing Pragmatic Categories in Natural Language Inference (NLI) by Large Language Models (LLMs)Joint Conference on Lexical and Computational Semantics (*SEM)Reto Gubelmann, Aikaterini-lida Kalouli, Christina Niklaus, et al.2023 · 15 citations
  66. Pre-training Distillation for Large Language Models: A Design Space ExplorationAnnual Meeting of the Association for Computational Linguistics (ACL)Hao Peng, Xin Lv, Yushi Bai, et al.2025 · 14 citations
  67. MiniPLM: Knowledge Distillation for Pre-Training Language ModelsarXiv:2410.17215Yuxian Gu, Hao Zhou, Fandong Meng, et al.2024 · 12 citations
  68. Parsing Argumentative Structure in English-as-Foreign-Language EssaysWorkshop on Innovative Use of NLP for Building Educational Applications (BEA)Jan Wira Gotama Putra, Simone Teufel, Takenobu Tokunaga2021 · 12 citations
  69. How Are Idioms Processed Inside Transformer Language Models?Joint Conference on Lexical and Computational Semantics (*SEM)Ye Tian, Isobel James, Hye Son2023 · 11 citations
  70. Speciesist Language and Nonhuman Animal Bias in English Masked Language ModelsInformation Processing & ManagementMasashi Takeshita, Rafal Rzepka, Kenji Araki2022 · 11 citations
  71. T-STAR: Truthful Style Transfer using AMR Graph as Intermediate RepresentationAnubhav Jangra, Preksha Nema, Aravindan Raghuveer2022 · 11 citations
  72. EMO: Embedding Model Distillation via Intra-Model Relation and Optimal Transport AlignmentsConference on Empirical Methods in Natural Language Processing (EMNLP)Minh-Phuc Truong, Hai An Vu, Tu Vu, et al.2025 · 10 citations
  73. Knowledge Distillation of Black-Box Large Language ModelsarXiv:2401.07013Hongzhan Chen, Ruijun Chen, Yuqi Yi, et al.2024 · 10 citations
  74. Investigating Forgetting in Pre-Trained Representations Through Continual LearningarXiv:2305.05968Yun Luo, Zhen Yang, Xuefeng Bai, et al.2023 · 10 citations
  75. RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP)Zhaoyan Gong, Juan Li, Zhiqiang Liu, et al.2025 · 9 citations
  76. To Distill or Not to Distill? On the Robustness of Robust Knowledge DistillationAnnual Meeting of the Association for Computational Linguistics (ACL)Abdul Waheed, Karima Kadaoui, Muhammad Abdul-Mageed2024 · 8 citations
  77. Cross-lingual transfer for low-resource Arabic language understandingArabic Natural Language Processing Workshop (WANLP)Khadige Abboud, Olga Golovneva, Christopher DiPersio2022 · 8 citations
  78. SemEval-2022 Task 3: PreTENS — Evaluating Neural Networks on Presuppositional Semantic KnowledgeRoberto Zamparelli, Shammur Chowdhury, Dominique Brunato, et al.2022 · 8 citations
  79. ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA ObjectiveFindings of the Association for Computational LinguisticsSultan Alrowili, K. Vijay-Shanker2021 · 8 citations
  80. End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective BaselinearXiv:2107.01700Tuan Manh Lai, Trung Bui, Doo Soon Kim2021 · 8 citations
  81. Beyond Logits: Aligning Feature Dynamics for Effective Knowledge DistillationAnnual Meeting of the Association for Computational Linguistics (ACL)Guoqiang Gong, Jiaxing Wang, Jin Xu, et al.2025 · 7 citations
  82. Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL)Chao Xue, Yao Wang, Mengqiao Liu, et al.2026 · 6 citations
  83. Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMsAnnual Meeting of the Association for Computational Linguistics (ACL)Anshumann, Mohd Abbas Zaidi, Akhil Kedia, et al.2025 · 6 citations
  84. Detecting Anti-Semitic Hate Speech using Transformer-based Large Language ModelsarXiv:2405.03794Dengyi Liu, Minghao Wang, Andrew G. Catlin2024 · 6 citations
  85. PTUK-HULAT at ArAIEval Shared Task Fine-tuned Distilbert to Predict Disinformative TweetsArabic Natural Language Processing Conference (ArabicNLP)Areej Jaber, Paloma Martinez2023 · 6 citations
  86. IISERB Brains at SemEval-2022 Task 6: A Deep-learning Framework to Identify Intended Sarcasm in EnglishInternational Workshop on Semantic Evaluation (SemEval)Tanuj Shekhawat, Manoj Kumar, Udaybhan Rathore, et al.2022 · 6 citations
  87. Vulnerability of LLMs to Vertically Aligned Text ManipulationsAnnual Meeting of the Association for Computational Linguistics (ACL)Zhecheng Li, Yiwei Wang, Bryan Hooi, et al.2025 · 5 citations
  88. NovBench: Evaluating Large Language Models on Academic Paper Novelty AssessmentFindings of the Association for Computational LinguisticsWenqing Wu, Yi Zhao, Yuzhuo Wang, et al.2026 · 4 citations
  89. EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in HealthcareLecture notes in computer scienceEman Alamoudi, Ellis Solaiman2025 · 4 citations
  90. MultiCoPIE: A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE DisambiguationWorkshop on Multiword Expressions (MWE)Uliana Sentsova, Debora Ciminari, Josef van Genabith, et al.2025 · 4 citations
  91. ReDASPersuasion at ArAIEval Shared Task: Multilingual and Monolingual Models For Arabic Persuasion DetectionArabic Natural Language Processing Conference (ArabicNLP)Fatima Zahra Qachfar, Rakesh Verma2023 · 4 citations
  92. Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of ExplanationsFindings of the Association for Computational LinguisticsPingjun Hong, Beiduo Chen, Siyao Peng, et al.2026 · 3 citations
  93. Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality PredictionFilip Miletić, Sabine Schulte im Walde2025 · 3 citations
  94. TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in HebrewAnita Gelboim, Elior Sulem2025 · 3 citations
  95. Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using TransformersNiv Fono, Harel Moshayof, Eldar Karol, et al.2024 · 3 citations
  96. Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract ClassificationWorkshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or IndividualAmrita Singh, H. Suhan Karaca, Aditya Joshi, et al.2026 · 2 citations
  97. LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned RepresentationsAnnual Meeting of the Association for Computational Linguistics (ACL)Robin Vujanic, Thomas Rückstieß2026 · 2 citations
  98. MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation DetectionConference of the European Chapter of the Association for Computational Linguistics (EACL)Arkadiusz Modzelewski, Witold Sosnowski, Eleni Papadopulos, et al.2026 · 2 citations
  99. Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM UnlearningAnnual Meeting of the Association for Computational Linguistics (ACL)Naixin Zhai, Pengyang Shao, Binbin Zheng, et al.2026 · 2 citations
  100. Multi-Granularity Semantic Revision for Large Language Model DistillationAnnual Meeting of the Association for Computational Linguistics (ACL)Xiaoyu Liu, Yun Zhang, Wei Li, et al.2026 · 2 citations

1 2 3

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/