Noah A. Smith · page 2
Papers
- WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022 · 317 citations)
Crowdsourced NLP datasets tend to be repetitive. Annotators writing many examples quickly fall back on a small set of habits, and models then learn shortcuts that fail outside the training distribution. The authors propose a pipeline in which a language model drafts examples and humans judge and polish them. They… - A Dependency Parser for Tweets (2014 · 298 citations)
Tweets break the assumptions of newswire parsers: many tokens (retweet markers, usernames, hashtags, URLs, punctuation) have no syntactic role, a tweet can hold several utterances with their own roots, and multiword names and idioms are common. - A Bayesian Mixed Effects Model of Literary Character (2014 · 285 citations)
The paper addresses unsupervised discovery of character types ("personas") in fiction. Earlier models treated a persona as the sole source of all words linked to a character, so learned types tended to group characters by author or period instead of by role. Bamman, Underwood and Smith propose a hierarchical Bayesian… - Paraphrase Identification as Probabilistic Quasi-Synchronous Recognition (2009 · 278 citations)
The paper addresses paraphrase identification: deciding whether two sentences mean essentially the same thing. - Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning (2019 · 259 citations)
Quoref has over 24K span-answer questions on 4.7K Wikipedia paragraphs (about 40% film plot summaries, which mention many characters), written by crowd workers so that answering needs linking two mentions of one entity. - The Multilingual Amazon Reviews Corpus (2020 · 257 citations)
Multilingual text classification lacks large, openly available benchmarks. Existing options such as RCV1/RCV2, MLDoc, XNLI, Yelp and earlier Amazon review releases are small, hard to obtain, missing non-English training data, unreliable about language labels, or lacking fixed splits. The authors address this gap with… - Neural Cross-Lingual Named Entity Recognition with Minimal Resources (2018 · 249 citations)
The paper tackles named entity recognition for languages with no labeled data, transferring knowledge from English annotations. - Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters (2013 · 247 citations)
Off-the-shelf part-of-speech taggers struggle with chat-like online writing such as tweets and IRC messages. - Sparse Overcomplete Word Vector Representations (2015 · 228 citations)
Dense word embeddings work well in practice, but they look nothing like the discrete classes and relations of lexical semantic theory, and their dimensions are hard to interpret. - How Language Model Hallucinations Can Snowball (2023 · 226 citations)
Hallucinations in large language models are usually blamed on gaps in what the model knows. This paper argues that some false statements have a different cause. Once a model commits to a wrong answer, it makes up supporting claims to stay consistent with it, even though it rejects those same claims when asked about… - Fine-Grained Human Feedback Gives Better Rewards for Language Model Training (2023 · 223 citations)
Standard RLHF turns a human's overall preference between whole outputs into one scalar reward at the end of the sequence. - Measuring Association Between Labels and Free-Text Rationales (2021 · 220 citations)
Explanations in interpretable NLP are only useful if they faithfully reflect how a model reached its decision. - Deep Multitask Learning for Semantic Dependency Parsing (2017 · 211 citations)
Semantic dependency graphs are costly to annotate, and the available annotation is split across competing formalisms, so each one has relatively little training data. - Sentence Mover's Similarity: Automatic Evaluation for Multi-Sentence Texts (2019 · 209 citations)
Popular metrics such as ROUGE and BLEU score generated text by exact word overlap. As a result, they miss paraphrased or reordered content and correlate weakly with human judgments. The embedding-based word mover's distance (WMD) is more flexible, but it treats a document as an unordered bag of words and ignores how… - Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2 (2023 · 207 citations)
After the first TÜLU release, open instruction tuning moved quickly. Stronger base models, better distilled datasets and simpler adaptation methods all appeared. This work from AI2 and the University of Washington tests these advances one by one and keeps the useful ones in TÜLU 2, a suite of LLAMA-2 models at 7B… - The Right Tool for the Job: Matching Model and Instance Complexities (2020 · 196 citations)
Running large pretrained encoders such as BERT-large in production is costly, because every incoming instance pays the full inference price. - One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022 · 194 citations)
Most text embedding models are built for one kind of job. A retriever such as DPR handles semantic similarity poorly, SimCSE has the reverse weakness, and both lose quality when moved to new domains like finance or medicine. The authors propose INSTRUCTOR, a single encoder that takes a natural-language description of… - Challenges in Automated Debiasing for Toxic Language Detection (2021 · 191 citations)
Toxic language classifiers often learn shortcuts that harm minority groups. They flag tweets simply for mentioning minority identities, for containing profanity, or for using features of African American English (AAE). This paper asks whether debiasing techniques developed for NLU tasks such as NLI can fix these… - RealTime QA: What's the Answer Right Now? (2022 · 181 citations)
Most open-domain QA benchmarks assume an answer stays fixed regardless of when the question is asked. RealTime QA drops that assumption. It is a living benchmark that publishes fresh questions about current events every week and scores systems on them before the next batch appears. Questions come from the weekly news… - Neural Discourse Structure for Text Categorization (2017 · 181 citations)
The paper asks whether the rhetorical organization of a document can guide a text classifier toward its most informative parts. - Shared Logistic Normal Distributions for Soft Parameter Tying in Unsupervised Grammar Induction (2009 · 177 citations)
Unsupervised induction of dependency grammars usually puts Dirichlet priors on grammar weights. Dirichlets cannot express beliefs about which rule probabilities should rise or fall together. An earlier logistic normal (LN) prior (Cohen et al., 2008) allowed such correlations, but only inside a single multinomial.… - Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models (2023 · 174 citations)
Commercial LM APIs bill per token, but how many tokens a text needs depends on the tokenizer and its training data. - Binding Language Models in Symbolic Languages (2022 · 170 citations)
End-to-end neural systems perform well on NLP tasks but are hard to interpret and fragile. Symbolic programs are transparent but limited to what their grammar can express. Earlier neural-symbolic hybrids needed custom languages, task-specific modules and large training sets. The authors introduce BINDER, a framework… - Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold (2017 · 160 citations)
The paper addresses frame-semantic argument identification: finding and labeling the text spans that fill the roles of FrameNet frames. - Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut (2014 · 159 citations)
The paper addresses the automatic identification of multiword expressions (MWEs) in running English text. It covers the full range of MWE types, from proper names and compounds to phrasal verbs, support-verb constructions and fixed phrases. Earlier work typically targeted one MWE subclass at a time and handled only…
This page in Markdown → https://md.research.elamur.ai/free/en/authors/noah-a-smith/page/2/