🧑‍🔬 Steven Y. Feng

Steven Y. Feng works in natural language processing, with a focus on the training data that language models learn from: how that data can be expanded or varied, and whether its character matters as much as its sheer volume. Their papers here, from 2021 to 2024, fall under Language models & pretraining and Psycholinguistics & language acquisition. They combine broad reviews of existing techniques with controlled experiments in which models are trained from scratch.

The most cited of these is A Survey of Data Augmentation Approaches for NLP (2021). It asks why augmentation is routine in computer vision yet less developed for text, where inputs are discrete and a transformation can easily change an example's label. The survey weighs how simple a method is to apply against how much it improves results, and it points to the limited theory behind why augmentation works. It sorts techniques into three groups: rule-based edits, MixUp-style blending of examples, and generation by models, for instance through backtranslation.

The more recent work turns toward language acquisition. Is Child-Directed Speech Effective Training Data for Language Models? (2024) asks whether the speech addressed to children explains why they learn language from far less input than models need. GPT-2 and RoBERTa are trained on English CHILDES, and the results are compared with a synthetic dialogue corpus written by GPT-4 and with subtitle, Wikipedia and BabyLM text. Syntactic knowledge is tested with the Zorro benchmark and meaning with word-similarity tasks. The study also looks separately at the effect of presenting conversations in order of the child's age.

Feng is based at Stanford University and has earlier papers from Carnegie Mellon University and the University of Waterloo. Co-authors include Eduard Hovy, Varun Gangal, Teruko Mitamura, Soroush Vosoughi and Jason Wei, as well as Michael C. Frank and Noah D. Goodman.

Steven Y. Feng has 2 papers in our library, in 2021–2024, most of them on Language models & pretraining, Psycholinguistics & language acquisition; often with Eduard Hovy, Jason Wei, Michael C. Frank.