Papers of 2021 — page 2263 papers · all years1 2 3GLaM: Efficient Scaling of Language Models with Mixture-of-ExpertsNan Du, Yanping Huang, Andrew M. Dai, et al.arXiv:2112.06905GPT Understands, TooXiao Liu, Yanan Zheng, Zhengxiao Du, et al.arXiv:2103.10385GPT3Mix: Leveraging Large-scale Language Models for Text AugmentationKang Min Yoo, Dongju Park, Jaewook Kang, et al.Findings of the Association for Computational…Grammar-Based Concept Alignment for Domain-Specific Machine TranslationArianna Masciolini, Aarne RantaMachine translationGraph Rewriting for Enhanced Universal DependenciesBruno Guillaume, Guy PerrierEnglishGraphCodeBERT: Pre-training Code Representations with Data FlowDaya Guo, Shuo Ren, Shuai Lu, et al.arXiv:2009.08366GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoningZilong Zheng, Shuwen Qiu, Lifeng Fan, et al.EnglishGSum: A General Framework for Guided Neural Abstractive SummarizationZi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, et al.Proceedings of the 2021 Conference of the North…HateCheck: Functional Tests for Hate Speech Detection ModelsPaul Röttger, Bertie Vidgen, Dong Nguyen, et al.Proceedings of the 59th Annual Meeting of the…He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on PragmaticsNanjiang Jiang, Marie-Catherine de MarneffeEnglishHeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion RecognitionAvihay Chriqui, Inbal YahavModern HebrewHebrew Psychological LexiconsNatalie Shapira, Dana Atzil-Slonim, Daniel Juravski, et al.Modern HebrewHighly accurate protein structure prediction with AlphaFoldJohn Jumper, Richard Evans, Alexander Pritzel, et al.NatureHow focus particles and accents affect attachmentKaty Carlson, David PotterEnglishHow Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language ModelsPhillip Rust, Jonas Pfeiffer, Ivan Vulić, et al.Proceedings of the 59th Annual Meeting of the…How many data points is a prompt worth?Teven Le Scao, Alexander RushProceedings of the 2021 Conference of the North…How useful are Enhanced Universal Dependencies for semantic interpretation?Jamie Y. Findlay, Dag T. T. HaugEnglishHuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden UnitsWei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, et al.arXiv:2106.07447Hurdles to Progress in Long-form Question AnsweringKalpesh Krishna, Aurko Roy, Mohit IyyerarXiv:2103.06332Improving Arabic Diacritization with Regularized Decoding and Adversarial TrainingHan Qin, Guimin Chen, Yuanhe Tian, et al.Semitic languagesInter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic ReasoningPan Lu, Ran Gong, Shibiao Jiang, et al.arXiv:2105.04165Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-TuningArmen Aghajanyan, Sonal Gupta, Luke ZettlemoyerProceedings of the 59th Annual Meeting of the…It's Not Just Size That Matters: Small Language Models Are Also Few-Shot LearnersTimo Schick, Hinrich SchützeProceedings of the 2021 Conference of the North…Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-trainingOshin Agarwal, Heming Ge, Siamak Shakeri, et al.Proceedings of the 2021 Conference of the North…LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text PairsChristoph Schuhmann, Richard Vencu, Romain Beaumont, et al.arXiv:2111.02114Large Dual Encoders Are Generalizable RetrieversJianmo Ni, Chen Qu, Jing Lu, et al.arXiv:2112.07899Large-scale text pre-training helps with dialogue act recognition, but not without fine-tuningBill Noble, Vladislav MaraevEnglishLatent Hatred: A Benchmark for Understanding Implicit Hate SpeechMai ElSherief, Caleb Ziems, David Muchlinski, et al.Proceedings of the 2021 Conference on Empirical…LayoutLMv2: Multi-modal Pre-training for Visually-rich Document UnderstandingYang Xu, Yiheng Xu, Tengchao Lv, et al.Proceedings of the 59th Annual Meeting of the…Learning Compact Metrics for MTAmy Pu, Hyung Won Chung, Ankur Parikh, et al.Proceedings of the 2021 Conference on Empirical…Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate DetectionBertie Vidgen, Tristan Thrush, Zeerak Waseem, et al.Proceedings of the 59th Annual Meeting of the…Learning How to Ask: Querying LMs with Mixtures of Soft PromptsGuanghui Qin, Jason EisnerarXiv:2104.06599 · NAACL-HLT 2021Learning To Retrieve Prompts for In-Context LearningOhad Rubin, Jonathan Herzig, Jonathan BerantarXiv:2112.08633Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, et al.arXiv:2103.00020Lemmatisation, etymology and information overload on English and Swedish editions of WiktionaryAllahverdi VerdizadeEnglishLeveraging Passage Retrieval with Generative Models for Open Domain Question AnsweringGautier Izacard, Edouard GraveProceedings of the 16th Conference of the…LexGLUE: A Benchmark Dataset for Legal Language Understanding in EnglishIlias Chalkidis; Abhik Jana; Dirk Hartung; Michael Bommarito; Ion Androutsopoulos; Daniel Martin Katz;, Nikolaos AletrasarXiv:2110.00976MacBERTh: Development and Evaluation of a Historically Pre-trained Language Model for English (1450-1950)Enrique Manjavacas Arevalo, Lauren FonteynEnglishMaking Pre-trained Language Models Better Few-shot LearnersTianyu Gao, Adam Fisch, Danqi ChenProceedings of the 59th Annual Meeting of the…MasakhaNER: Named Entity Recognition for African LanguagesDavid Ifeoluwa Adelani, Jade Abbott, Graham Neubig, et al.Transactions of the Association for…Masked Autoencoders Are Scalable Vision LearnersKaiming He, Xinlei Chen, Saining Xie, et al.arXiv:2111.06377Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for LittleKoustuv Sinha, Robin Jia, Dieuwke Hupkes, et al.Proceedings of the 2021 Conference on Empirical…MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence FrontiersKrishna Pillutla, Swabha Swayamdipta, Rowan Zellers, et al.arXiv:2102.01454Measuring Coding Challenge Competence With APPSDan Hendrycks, Steven Basart, Saurav Kadavath, et al.arXiv:2105.09938Measuring Mathematical Problem Solving With the MATH DatasetDan Hendrycks, Collin Burns, Saurav Kadavath, et al.arXiv:2103.03874Merging Models with Fisher-Weighted AveragingMichael Matena, Colin RaffelarXiv:2111.09832Metacommunicative-why fragments as probes into the grammar of the speech act layerRebecca Woods, Luis VicenteEnglishMIDAS: A Dialog Act Annotation Scheme for Open Domain Human-Machine Spoken ConversationsDian Yu, Zhou YuEnglishMind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style TransferFanchao Qi, Yangyi Chen, Xurui Zhang, et al.Proceedings of the 2021 Conference on Empirical…MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicsKunhao Zheng, Jesse Michael Han, Stanislas PoluarXiv:2109.00110Minor changes make a difference: a case study on the consistency of UD-based dependency parsersDmytro Kalpakchi, Johan BoyeMova's core listMischievous nominal constructions in Universal DependenciesNathan Schneider, Amir ZeldesMova's core listMorphyNet: a Large Multilingual Database of Derivational and Inflectional MorphologyKhuyagbaatar Batsuren, Gabor Bella, Fausto GiunchigliaEnglishMTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing BenchmarkHaoran Li, Abhinav Arora, Shuohui Chen, et al.Proceedings of the 16th Conference of the…Multilingual Translation from Denoising Pre-TrainingYuqing Tang, Chau Tran, Xian Li, et al.Findings of the Association for Computational…Multimodal Few-Shot Learning with Frozen Language ModelsMaria Tsimpoukelli, Jacob Menick, Serkan Cabi, et al.arXiv:2106.13884Multitask Prompted Training Enables Zero-Shot Task GeneralizationVictor Sanh, Albert Webson, Colin Raffel, et al.arXiv:2110.08207Multiword expressions as discourse markers in Hebrew and LithuanianGiedre Valunaite Oleskeviciene, Chaya LiebeskindModern HebrewMuppet: Massive Multi-task Representations with Pre-FinetuningArmen Aghajanyan, Anchit Gupta, Akshat Shrivastava, et al.Proceedings of the 2021 Conference on Empirical…MuRIL: Multilingual Representations for Indian LanguagesSimran Khanuja, Diksha Bansal, Sarvesh Mehtani, et al.arXiv:2103.10730Neural Modeling for Named Entities and Morphology (NEMO2)Dan Bareket, Reut TsarfatyModern HebrewNew Perspectives in Biblical and Rabbinic HebrewAaron D. Hornkohl, Geoffrey KhanAncient HebrewNLP in the DH pipeline: Transfer-learning to a ChronolectAynat Rubinstein, Avi ShmidmanModern HebrewNoisy Channel Language Model Prompting for Few-Shot Text ClassificationSewon Min, Mike Lewis, Hannaneh Hajishirzi, et al.arXiv:2108.04106NOPE: A Corpus of Naturally-Occurring Presuppositions in EnglishAlicia Parrish, Sebastian Schuster, Alex Warstadt, et al.EnglishOCR-free Document Understanding TransformerGeewook Kim, Teakgyu Hong, Moonbin Yim, et al.arXiv:2111.15664On Generalization in Coreference ResolutionShubham Toshniwal, Patrick Xia, Sam Wiseman, et al.EnglishOn the Opportunities and Risks of Foundation ModelsRishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.arXiv:2108.07258Overview of the WANLP 2021 Shared Task on Sarcasm and Sentiment Detection in ArabicIbrahim Abu Farha, Wajdi Zaghouani, Walid MagdyProceedings of the Sixth Arabic Natural…P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and TasksXiao Liu, Kaixuan Ji, Yicheng Fu, et al.arXiv:2110.07602PAQ: 65 Million Probably-Asked Questions and What You Can Do With ThemPatrick Lewis, Yuxiang Wu, Linqing Liu, et al.Transactions of the Association for…Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights CasesIlias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, et al.Proceedings of the 2021 Conference of the North…Parameter-Efficient Transfer Learning with Diff PruningDemi Guo, Alexander Rush, Yoon KimProceedings of the 59th Annual Meeting of the…ParaShoot: A Hebrew Question Answering DatasetOmri Keren, Omer LevyModern HebrewPerceiver: General Perception with Iterative AttentionAndrew Jaegle, Felix Gimeno, Andrew Brock, et al.arXiv:2103.03206Persistent Anti-Muslim Bias in Large Language ModelsAbubakar Abid, Maheen Farooqi, James ZouarXiv:2101.05783Points of Contact: The Shared Intellectual History of Vocalisation in Syriac, Arabic, and HebrewNick PosegayAncient HebrewPre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingPengfei Liu, Weizhe Yuan, Jinlan Fu, et al.arXiv:2107.13586Predicting scalar inferences from "or" to "not both" using neural sentence encodersElissa Li, Sebastian Schuster, Judith DegenEnglishProgram Synthesis with Large Language ModelsJacob Austin, Augustus Odena, Maxwell Nye, et al.arXiv:2108.07732Prompt Programming for Large Language Models: Beyond the Few-Shot ParadigmLaria Reynolds, Kyle McDonellarXiv:2102.07350ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural LanguageOyvind Tafjord, Bhavana Dalvi, Peter ClarkFindings of the Association for Computational…QMSum: A New Benchmark for Query-based Multi-domain Meeting SummarizationMing Zhong, Da Yin, Tao Yu, et al.Proceedings of the 2021 Conference of the North…Quality at a Glance: An Audit of Web-Crawled Multilingual DatasetsJulia Kreutzer, Isaac Caswell, Lisa Wang, et al.arXiv:2103.12028 · Transactions of the…QuestEval: Summarization Asks for Fact-based EvaluationThomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, et al.Proceedings of the 2021 Conference on Empirical…Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A SurveyBonan Min, Hayley Ross, Elior Sulem, et al.arXiv:2111.01243Recipes for Building an Open-Domain ChatbotStephen Roller, Emily Dinan, Naman Goyal, et al.Proceedings of the 16th Conference of the…Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative RefinementAlireza Mohammadshahi, James HendersonEnglishRecursively Summarizing Books with Human FeedbackJeff Wu, Long Ouyang, Daniel M. Ziegler, et al.arXiv:2109.10862Referenceless Parsing-Based Evaluation of AMR-to-English GenerationEmma Manning, Nathan SchneiderEnglishResolving Prepositional Phrase Attachment Ambiguities with Contextualized Word EmbeddingsAdwait Ratnaparkhi, Atul KumarEnglishResults of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News DomainMarkus Freitag, Ricardo Rei, Nitika Mathur, et al.Proceedings of the Sixth Conference on Machine…Retrieval Augmentation Reduces Hallucination in ConversationKurt Shuster, Spencer Poff, Moya Chen, et al.arXiv:2104.07567Revisiting Shallow Discourse Parsing in the PDTB-3: Handling Intra-sentential ImplicitsZheng Zhao, Bonnie WebberEnglishRocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question AnsweringYingqi Qu, Yuchen Ding, Jing Liu, et al.Proceedings of the 2021 Conference of the North…Scaling Up Visual and Vision-Language Representation Learning With Noisy Text SupervisionChao Jia, Yinfei Yang, Ye Xia, et al.arXiv:2102.05918 · International Conference on…Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPTimo Schick, Sahana Udupa, Hinrich SchutzearXiv:2103.00453Semantic Ambiguity of So-Called "Stative Verbs" in Biblical Hebrew: Beyond Active vs. Stative Binary Opposition for the Qatal and Qatil Forms in the Qal StemJun SatoAncient HebrewSemLink 2.0: Chasing Lexical ResourcesKevin Stowe, Jenette Preciado, Kathryn Conger, et al.EnglishSentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text ModelsJianmo Ni, Gustavo Hernandez Abrego, Noah Constant, et al.arXiv:2108.088771 2 3