Noah A. Smith · page 4
Papers
- Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback (2024 · 50 citations)
Preference-based training is now a standard final stage for language models. Published setups, however, combine different datasets, algorithms and evaluation suites, so it is unclear which ingredient actually drives the gains. The authors divide the pipeline into four components: preference data, the learning… - Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models (2024 · 42 citations)
Multilingual language models fit many languages into one fixed set of parameters, so each language usually ends up weaker than it would be in a dedicated monolingual model. - Recall-Oriented Learning of Named Entities in Arabic Wikipedia (2012 · 42 citations)
The paper tackles named entity recognition in Arabic Wikipedia, where no labeled target-domain training data exists. - Semi-Supervised Frame-Semantic Parsing for Unknown Predicates (2011 · 41 citations)
Frame-semantic parsers trained on FrameNet struggle with predicates that never appear in the lexicon or its annotated sentences. - An Empirical Comparison of Parsing Methods for Stanford Dependencies (2014 · 39 citations)
Stanford dependencies are usually obtained by parsing to constituents and then applying head rules (c-parsing), which is slow. - Infusing Finetuning with Semantic Dependencies (2021 · 35 citations)
Pretrained language models clearly encode a lot of syntax. Whether they also capture meaning, specifically predicate-argument structure, is less clear. The authors test this with a new full-graph probe for semantic dependency parsing on the DM formalism (CoNLL 2015 data). They compare it with a syntactic probe on the… - Dynamic Language Models for Streaming Text (2014 · 32 citations)
Most language models assume word distributions stay fixed over time. That assumption fits poorly with fast-changing streams like news and social media. Yogatama et al. propose a probabilistic language model whose parameters evolve daily and can depend on observable non-text signals, here stock market movements. Each… - Feature-Rich Translation by Quasi-Synchronous Lattice Parsing (2009 · 31 citations)
The paper presents a log-linear translation model that puts phrase features, target n-gram and dependency-syntax features, and tree-to-tree configuration features into one score. - Morphosyntactic probing of multilingual BERT models (2023 · 26 citations)
The paper asks how much morphological knowledge multilingual masked language models hold and where in a sentence the cues for a word's morphological tag are found. - Coarse Lexical Semantic Annotation with Supersenses: An Arabic Case Study (2012 · 24 citations)
The authors reuse WordNet's supersense labels (the coarse lexicographer classes) as a light scheme for lexical semantics, and apply it to Arabic. - MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization (2024 · 19 citations)
Byte-level and subword language models usually split non-Latin-script text into many more pieces than English. - Supersense Tagging for Arabic: the MT-in-the-Middle Attack (2013 · 10 citations)
The paper tackles nominal supersense tagging for Arabic: finding noun chunks and assigning each one of the 25 coarse WordNet noun categories. - MameLoshnLM: Yiddish Language Model and Evaluation Benchmark (2026)
Yiddish has a long written tradition but a small and unreliable presence on the web, so multilingual models handle it poorly and there are few trustworthy ways to measure them. - Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior (2026)
Researchers often say that pretraining data shapes what a language model knows and how it is biased. Most of the evidence, however, is correlational and comes from finished models and whole corpora. Nadkarni and colleagues test this causally. Their recipe needs a model with intermediate checkpoints, optimizer states…
This page in Markdown → https://md.research.elamur.ai/free/en/authors/noah-a-smith/page/4/