Luke Zettlemoyer · page 2
Papers
- BERT for Coreference Resolution: Baselines and Analysis (2019 · 490 citations)
The paper asks how much a pretrained transformer helps coreference resolution, the task of grouping mentions that refer to the same entity. - Paraphrase-Driven Learning for Open Question Answering (2013 · 444 citations)
The paper tackles open-domain question answering over a noisy knowledge base built automatically by information extraction from web text. - Neural AMR: Sequence-to-Sequence Models for Parsing and Generation (2017 · 425 citations)
The paper applies sequence-to-sequence neural models to both AMR parsing and AMR generation. The main obstacle is data sparsity, so the authors pretrain on millions of unlabelled Gigaword sentences: a parser labels them, a generator is pretrained on those pairs, and then both are fine-tuned on the 16,833 gold pairs.… - Emerging Cross-lingual Structure in Pretrained Language Models (2020 · 382 citations)
Multilingual masked language models such as mBERT and XLM transfer well across languages. Earlier work proposed explanations, such as a shared subword vocabulary, but tested them only on a single mBERT and never measured them systematically. The authors train bilingual MLMs on Wikipedia, pairing English with French… - Scaling Semantic Parsers with On-the-Fly Ontology Matching (2013 · 363 citations)
The authors address the problem of scaling semantic parsers to large open-domain knowledge bases such as Freebase. - Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language (2015 · 327 citations)
The paper introduces question-answer driven semantic role labeling (QA-SRL): each verbal predicate is labeled with wh-questions, such as "Who finished something?", and each question is answered by phrases of the sentence. - Multi-hop Reading Comprehension through Question Decomposition and Rescoring (2019 · 312 citations)
Multi-hop reading comprehension asks a system to combine evidence from several paragraphs. The paper presents DecompRC, which splits a compositional question into simpler single-hop sub-questions. An ordinary off-the-shelf single-hop RC model then answers these sub-questions. - Muppet: Massive Multi-task Representations with Pre-Finetuning (2021 · 307 citations)
Pre-trained language models rely on self-supervision, yet labeled data for related tasks is often available. - Globally Coherent Text Generation with Neural Checklist Models (2016 · 304 citations)
RNN generators produce fluent local text but tend to forget which inputs they have already covered. In long outputs this leads to omitted, repeated or invented content. Kiddon, Zettlemoyer and Choi address this with the neural checklist model. It is a GRU encoder-decoder that keeps a soft record of which agenda items… - Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (2018 · 285 citations)
Most strong neural semantic role labeling (SRL) systems are BIO taggers. They handle one predicate at a time, assume gold predicates are given, and cannot use span-level features. In practice they need a separate, error-prone predicate identification step. He, Lee, Levy and Zettlemoyer propose an end-to-end model… - Lexical Generalization in CCG Grammar Induction for Semantic Parsing (2011 · 285 citations)
CCG-based semantic parsers learn lexicons that map words to syntactic categories paired with lambda-calculus meanings. - Surface Form Competition: Why the Highest Probability Answer Isn't Always Right (2021 · 277 citations)
Zero-shot multiple-choice evaluation often scores each option by the probability a language model gives it. - Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023 · 272 citations)
Chain-of-Thought (CoT) prompting helps large language models solve multi-step problems, but it is unclear which properties of the demonstrated rationales cause the gains. - The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2023 · 262 citations)
Most multilingual NLU benchmarks together span only about 30 languages, which makes it hard to tell how well language models actually understand text beyond high-resource languages. - Contrastive Decoding: Open-ended Text Generation as Optimization (2022 · 253 citations)
Open-ended text generation with large language models faces a tradeoff. Picking the most probable continuation tends to yield short, looping text, while sampling often wanders off topic or loses coherence. The authors introduce contrastive decoding (CD), a training-free, search-based method that pairs two frozen… - Trusting Your Evidence: Hallucinate Less with Context-aware Decoding (2024 · 249 citations)
Language models often underweight the input context and fall back on knowledge memorized during pretraining. - Learning Context-Dependent Mappings from Sentences to Logical Form (2009 · 246 citations)
The paper tackles semantic parsing for dialogue-like interactions, where a sentence's meaning often relies on earlier turns. - In-context Examples Selection for Machine Translation (2023 · 243 citations)
The paper asks how the choice and ordering of in-context demonstrations shape the output of a large generative model when it is used for machine translation. - Noisy Channel Language Model Prompting for Few-Shot Text Classification (2021 · 243 citations)
Few-shot text classification with large causal language models usually relies on "direct" prompting: the model scores each label's verbalizer given the input. - A Discrete Hard EM Approach for Weakly Supervised Question Answering (2019 · 237 citations)
Many QA datasets supply only the final answer string, not how it was derived. The same TriviaQA answer may appear in several document spans, and a DROP number can come from many different arithmetic expressions, most of which are spurious. Min et al. cast these weakly supervised tasks as discrete latent-variable… - Compositional Questions Do Not Necessitate Multi-hop Reasoning (2019 · 235 citations)
The paper asks whether questions that appear compositional actually force a reading comprehension system to combine evidence from several paragraphs. - Detecting Hallucinated Content in Conditional Neural Sequence Generation (2021 · 232 citations)
Neural generators for translation and summarization can produce fluent output that contains material the input does not support. - Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders (2020 · 232 citations)
Word Sense Disambiguation (WSD) systems tend to fail on rare senses. Sense frequencies follow a Zipfian distribution, so models learn to favor the most frequent sense (MFS) and do poorly on less frequent senses (LFS) or senses never seen in training. Blevins and Zettlemoyer address this with a bi-encoder model (BEM).… - Broad-coverage CCG Semantic Parsing with AMR (2015 · 229 citations)
The authors study how to automatically convert sentences from news texts into AMR. AMR is a general-purpose representation of sentence meaning that is not tied to any particular downstream task. Previously, semantic parsers had to be retrained for each application, and they worked only with narrow types of sentences.… - A Generative Model for Parsing Natural Language to Meaning Representations (2008 · 210 citations)
This EMNLP 2008 paper by Wei Lu, Hwee Tou Ng, Wee Sun Lee and Luke S. Zettlemoyer addresses semantic parsing: mapping a sentence to a variable-free, tree-shaped meaning representation (MR). Earlier systems learn one explicit grammar with a fixed set of lexical entries and rules, so they can fail on test sentences…
This page in Markdown → https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/page/2/