Elamur Research⌕
☰
home / authors / Colin Raffel / all works

Colin Raffel — all works

107 works · page 1 of 2 · 1–100 · ← Colin Raffel

1 2

  1. Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerarXiv:1910.10683Colin Raffel, Noam Shazeer, Adam Roberts, et al.2019 · 11,325 citations
  2. mT5: A Massively Multilingual Pre-trained Text-to-Text TransformerConference of the North American Chapter of the Association for Computational Linguistics (NAACL)Linting Xue, Noah Constant, Adam Roberts, et al.2021 · 3,226 citations
  3. FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence ↗arXiv (Cornell University) · no summary yetKihyuk Sohn et al.2020 · 2,292 citations
  4. Emergent Abilities of Large Language ModelsarXiv:2206.07682Jason Wei, Yi Tay, Rishi Bommasani, et al.2022 · 2,269 citations
  5. Multitask Prompted Training Enables Zero-Shot Task GeneralizationarXiv:2110.08207Victor Sanh, Albert Webson, Colin Raffel, et al.2021 · 1,612 citations
  6. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model ↗arXiv:2211.05100 · no summary yetBigScience Workshop et al.2022 · 1,606 citations
  7. How Much Knowledge Can You Pack Into the Parameters of a Language Model? ↗Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · no summary yetAdam Roberts, Colin Raffel, Noam Shazeer2020 · 1,100 citations
  8. Extracting Training Data from Large Language ModelsarXiv:2012.07805Nicholas Carlini, Florian Tramer, Eric Wallace, et al.2020 · 955 citations
  9. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context LearningConference on Neural Information Processing Systems (NeurIPS)Haokun Liu, Derek Tam, Mohammed Muqeeth, et al.2022 · 754 citations
  10. ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models ↗Transactions of the Association for Computational Linguistics · no summary yetLinting Xue et al.2022 · 676 citations
  11. MixMatch: A Holistic Approach to Semi-Supervised Learning ↗arXiv (Cornell University) · no summary yetDavid Berthelot et al.2019 · 666 citations
  12. The FineWeb Datasets: Decanting the Web for the Finest Text Data at ScaleConference on Neural Information Processing Systems (NeurIPS)Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, et al.2024 · 434 citations
  13. Thermometer Encoding: One Hot Way To Resist Adversarial Examples ↗International Conference on Learning Representations · no summary yetJacob Buckman et al.2018 · 433 citations
  14. Large Language Models Struggle to Learn Long-Tail KnowledgearXiv:2211.08411Nikhil Kandpal, Haikang Deng, Adam Roberts, et al.2022 · 366 citations
  15. TIES-Merging: Resolving Interference When Merging ModelsConference on Neural Information Processing Systems (NeurIPS)Prateek Yadav, Derek Tam, Leshem Choshen, et al.2023 · 324 citations
  16. A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music ↗arXiv (Cornell University) · no summary yetRoberts, Adam et al.2018 · 296 citations
  17. Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems ↗arXiv (Cornell University) · no summary yetColin Raffel, Daniel P. W. Ellis2015 · 286 citations
  18. Mir_Eval: A Transparent Implementation Of Common Mir Metrics. ↗no summary yetColin Raffel et al.2014 · 282 citations
  19. Online and Linear-Time Attention by Enforcing Monotonic Alignments ↗arXiv (Cornell University) · no summary yetColin Raffel et al.2017 · 267 citations
  20. Monotonic Infinite Lookback Attention for Simultaneous Machine TranslationAnnual Meeting of the Association for Computational Linguistics (ACL)Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, et al.2019 · 263 citations
  21. An Empirical Survey of Data Augmentation for Limited Data Learning in NLPTransactions of the Association for Computational Linguistics (TACL)Jiaao Chen, Derek Tam, Colin Raffel, et al.2023 · 254 citations
  22. Scaling Data-Constrained Language Models ↗arXiv:2305.16264 · no summary yetNiklas Muennighoff et al.2023 · 249 citations
  23. Realistic Evaluation of Deep Semi-Supervised Learning Algorithms ↗arXiv (Cornell University) · no summary yetAvital Oliver et al.2018 · 241 citations
  24. Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition ↗arXiv (Cornell University) · no summary yetYao Qin et al.2019 · 227 citations
  25. Merging Models with Fisher-Weighted AveragingConference on Neural Information Processing Systems (NeurIPS)Michael Matena, Colin Raffel2021 · 208 citations
  26. WT5?! Training Text-to-Text Models to Explain their Predictions ↗arXiv:2004.14546 · no summary yetSharan Narang et al.2020 · 199 citations
  27. Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLParXiv:2112.10508Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky, et al.2021 · 191 citations
  28. Improving and Simplifying Pattern Exploiting TrainingConference on Empirical Methods in Natural Language Processing (EMNLP)Derek Tam, Rakesh R. Menon, Mohit Bansal, et al.2021 · 189 citations
  29. PromptSource: An Integrated Development Environment and Repository for Natural Language PromptsarXiv:2202.01279Stephen H. Bach, Victor Sanh, Zheng-Xin Yong, et al.2022 · 174 citations
  30. Deduplicating Training Data Mitigates Privacy Risks in Language ModelsarXiv:2202.06539Nikhil Kandpal, Eric Wallace, Colin Raffel2022 · 172 citations
  31. ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring ↗arXiv (Cornell University) · no summary yetDavid Berthelot et al.2020 · 171 citations
  32. Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching ↗no summary yetColin Raffel2016 · 154 citations
  33. Efficient Methods for Natural Language Processing: A SurveyTransactions of the Association for Computational Linguistics (TACL)Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.2023 · 145 citations
  34. Scaling Up Models and Data with t5x and seqio ↗arXiv:2203.17189 · no summary yetAdam Roberts et al.2022 · 130 citations
  35. Do Transformer Modifications Transfer Across Implementations and Applications? ↗Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · no summary yetSharan Narang et al.2021 · 123 citations
  36. Evaluating the Factual Consistency of Large Language Models Through News Summarization ↗Findings of the Association for Computational Linguistics: ACL 2023 · no summary yetDerek Tam et al.2023 · 118 citations
  37. NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned ↗arXiv (Cornell University) · no summary yetSewon Min et al.2021 · 108 citations
  38. FineWeb2: One Pipeline to Scale Them All — Adapting Pre-Training Data Processing to Every LanguagearXiv:2506.20920Guilherme Penedo, Hynek Kydliček, Vinko Sabolčec, et al.2025 · 99 citations
  39. A Survey on Data Selection for Language ModelsarXiv:2402.16827Alon Albalak, Yanai Elazar, Sang Michael Xie, et al.2024 · 95 citations
  40. Optimizing DTW-based audio-to-MIDI alignment and matching ↗IEEE International Conference on Acoustics Speech and Signal Processing · no summary yetColin Raffel, Daniel P. W. Ellis2016 · 92 citations
  41. What Language Model to Train if You Have One Million GPU Hours? ↗arXiv:2210.15424 · no summary yetTeven Le Scao et al.2022 · 73 citations
  42. Training Neural Networks with Fixed Sparse Masks ↗arXiv (Cornell University) · no summary yetYi-Lin Sung, Varun Nair, Colin Raffel2021 · 72 citations
  43. What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? ↗arXiv:2204.05832 · no summary yetThomas Wang et al.2022 · 62 citations
  44. Learning-based Memory Allocation for C++ Server Workloads ↗International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · no summary yetMartin Maas et al.2020 · 60 citations
  45. Monotonic Chunkwise Attention ↗arXiv (Cornell University) · no summary yetChung‐Cheng Chiu, Colin Raffel2017 · 60 citations
  46. Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer ↗arXiv (Cornell University) · no summary yetDavid Berthelot et al.2018 · 59 citations
  47. Realistic Evaluation of Semi-Supervised Learning Algorithms. ↗International Conference on Learning Representations · no summary yetAvital Oliver et al.2018 · 56 citations
  48. Distributed Inference and Fine-tuning of Large Language Models Over The Internet ↗NeurIPS 2023 · no summary yetAlexander Borzunov et al.2023 · 52 citations
  49. Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions ↗arXiv (Cornell University) · no summary yetYao Qin et al.2019 · 52 citations
  50. Is Generator Conditioning Causally Related to GAN Performance? ↗arXiv (Cornell University) · no summary yetAugustus Odena et al.2018 · 48 citations
  51. Petals: Collaborative Inference and Fine-tuning of Large Models ↗ACL 2023 (demo) · no summary yetAlexander Borzunov et al.2022 · 47 citations
  52. Onsets and Frames: Dual-Objective Piano Transcription ↗arXiv (Cornell University) · no summary yetCurtis Hawthorne et al.2017 · 47 citations
  53. DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows ↗Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yetAjay Patel, Colin Raffel, Chris Callison-Burch2024 · 43 citations
  54. Bidirectional Language Models Are Also Few-shot Learners ↗arXiv (Cornell University) · no summary yetAjay Patel et al.2022 · 42 citations
  55. Towards GAN Benchmarks Which Require Generalization ↗arXiv (Cornell University) · no summary yetIshaan Gulrajani, Colin Raffel, Luke Metz2020 · 42 citations
  56. Knowledge is a Region in Weight Space for Fine-tuned Language Models ↗Findings of the Association for Computational Linguistics: EMNLP 2023 · no summary yetAlmog Gueta et al.2023 · 40 citations
  57. ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning ↗Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yetShachar Don-Yehiya et al.2023 · 37 citations
  58. Efficient Online Data Mixing For Language Model Pre-TrainingarXiv:2312.02406Alon Albalak, Liangming Pan, Colin Raffel, et al.2023 · 35 citations
  59. Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model ↗Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · no summary yetHaikang Deng, Colin Raffel2023 · 34 citations
  60. On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition ↗arXiv:2106.03062 · no summary yetChing-Yuan Bai et al.2021 · 30 citations
  61. Learning a Latent Space of Multitrack Measures ↗arXiv (Cornell University) · no summary yetIan Simon et al.2018 · 29 citations
  62. Learning Hard Alignments with Variational Inference ↗IEEE International Conference on Acoustics Speech and Signal Processing · no summary yetDieterich Lawson et al.2018 · 26 citations
  63. Soft Merging of Experts with Adaptive Routing ↗arXiv:2306.03745 · no summary yetMohammed Muqeeth, Haokun Liu, Colin Raffel2023 · 24 citations
  64. Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks ↗Proceedings of the AAAI Conference on Artificial Intelligence · no summary yetNikolai Yakovenko et al.2016 · 22 citations
  65. The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text ↗neural information processing systems · no summary yetNikhil Kandpal et al.2025 · 21 citations
  66. Learning to Route Among Specialized Experts for Zero-Shot Generalization ↗arXiv (Cornell University) · no summary yetMuqeeth, Mohammed et al.2024 · 21 citations
  67. Robust and Generalizable Visual Representation Learning via Random Convolutions ↗arXiv (Cornell University) · no summary yetXu, Zhenlin et al.2020 · 21 citations
  68. Merging by Matching Models in Task Parameter Subspaces ↗arXiv (Cornell University) · no summary yetDerek Tam, Mohit Bansal, Colin Raffel2023 · 20 citations
  69. Large-Scale Content-Based Matching Of Midi And Audio Files. ↗Zenodo (CERN European Organization for Nuclear Research) · no summary yetColin Raffel, Daniel P. W. Ellis2015 · 20 citations
  70. Extracting Ground-Truth Information From Midi Files: A Midifesto. ↗Zenodo (CERN European Organization for Nuclear Research) · no summary yetColin Raffel, Daniel P. W. Ellis2016 · 17 citations
  71. A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning ↗arXiv (Cornell University) · no summary yetPrateek Yadav et al.2024 · 16 citations
  72. Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models ↗arXiv:2404.05567 · no summary yetBowen Pan et al.2024 · 14 citations
  73. Deflecting Adversarial Attacks ↗arXiv (Cornell University) · no summary yetYao Qin et al.2020 · 13 citations
  74. Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples ↗arXiv (Cornell University) · no summary yetSamarth Sinha et al.2020 · 13 citations
  75. ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization ↗arXiv:2311.13171 · no summary yetPrateek Yadav et al.2023 · 11 citations
  76. Hickle: A HDF5-based python pickle replacement ↗The Journal of Open Source Software · no summary yetDanny C. Price et al.2018 · 11 citations
  77. Learning with Limited Text Data ↗Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts · no summary yetDiyi Yang, Ankur Parikh, Colin Raffel2022 · 10 citations
  78. Pruning subsequence search with attention-based embedding ↗IEEE International Conference on Acoustics Speech and Signal Processing · no summary yetColin Raffel, Daniel P. W. Ellis2016 · 10 citations
  79. Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data ↗arXiv:2302.00674 · no summary yetAlon Albalak, Colin Raffel, William Yang Wang2023 · 9 citations
  80. Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models ↗arXiv (Cornell University) · no summary yetNikhil Kandpal et al.2023 · 9 citations
  81. Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language ↗neural information processing systems · no summary yetZhenlin Xu, Marc Niethammer, Colin Raffel2022 · 8 citations
  82. Explaining the Learning Dynamics of Direct Feedback Alignment ↗International Conference on Learning Representations · no summary yetJustin Gilmer et al.2017 · 7 citations
  83. Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games ↗arXiv (Cornell University) · no summary yetNikolai Yakovenko et al.2015 · 5 citations
  84. Position: The Most Expensive Part of an LLM should be its Training Data ↗arXiv (Cornell University) · no summary yetNikhil Kandpal, Colin Raffel2025 · 4 citations
  85. Realistic Evaluation of Model Merging for Compositional Generalization ↗arXiv (Cornell University) · no summary yetDerek Tam et al.2024 · 4 citations
  86. TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior ↗arXiv (Cornell University) · no summary yetGül Sena Altıntaş et al.2025 · 3 citations
  87. The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions ↗arXiv (Cornell University) · no summary yetDevin Kwok et al.2025 · 2 citations
  88. Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator ↗arXiv (Cornell University) · no summary yetLi, YuXin et al.2025 · 2 citations
  89. AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution ↗arXiv (Cornell University) · no summary yetFengyuan Liu, Nikhil Kandpal, Colin Raffel2024 · 2 citations
  90. Estimating timing and channel distortion across related signals ↗IEEE International Conference on Acoustics Speech and Signal Processing · no summary yetColin Raffel, Daniel P. W. Ellis2014 · 2 citations
  91. How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data ↗arXiv (Cornell University) · no summary yetJoel Niklaus et al.2026 · 1 citation
  92. Efficiently Estimating Data Efficiency for Language Model Fine-tuning ↗arXiv (Cornell University) · no summary yetGyung Hyun Je, Colin Raffel2025 · 1 citation
  93. A Large-scale Study on Training Sample Memorization in Generative Modeling ↗no summary yetChing-Yuan Bai et al.2021 · 1 citation
  94. Improving Interpolation in Autoencoders ↗no summary yetDavid Berthelot et al.2019 · 1 citation
  95. Evaluating Implicit Generative Models With Large Samples ↗no summary yetIshaan Gulrajani, Colin Raffel, Luke Metz2018 · 1 citation
  96. Training a Subsampling Mechanism in Expectation ↗arXiv (Cornell University) · no summary yetColin Raffel, Dieterich Lawson2017 · 1 citation
  97. Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models ↗arXiv (Cornell University) · no summary yetMalikeh Ehghaghi et al.2026
  98. Model Merging via Data-Free Covariance Estimation ↗arXiv (Cornell University) · no summary yetMarawan Gamal Abdel Hameed et al.2026
  99. The Appeal and Reality of Recycling LoRAs with Adaptive Merging ↗arXiv (Cornell University) · no summary yetHaokun Liu et al.2026
  100. FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale ↗arXiv (Cornell University) · no summary yetAjay Patel, Colin Raffel, Chris Callison-Burch2026

1 2

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/colin-raffel/works/