🧑‍🔬 Tom Brown

Tom Brown's papers here study how large language models absorb their training text, and what that means for privacy and for model quality. The recurring question is what happens to sequences a model has seen, whether it can reproduce them verbatim or is harmed when they recur too often. The methods range from black-box attacks that probe a trained model from the outside to controlled training runs analysed with scaling laws and mechanistic tools. The work sits between Language models & pretraining and Interpretability.

The most cited paper, Extracting Training Data from Large Language Models (2020), challenged the assumption that models trained on vast, deduplicated corpora for few epochs keep their training text private. It showed that someone with only query access can recover exact training sequences. The method first samples many plausible generations and then ranks them by comparing the target model's likelihood with that of a reference model, so that passages the target finds suspiciously probable rise to the top. GPT-2 served as the test case.

The newer paper, Scaling Laws and Interpretability of Learning from Repeated Data (2022), moves from extraction to the effects of repetition itself. It trains transformer models on corpora that are mostly unique but contain a small, heavily repeated subset, and it varies model size, the share of repeated tokens and the number of repeats. Read through both scaling laws and mechanistic interpretability, the results show a marked double descent in how much the repetition hurts.

The extraction study was written with Nicholas Carlini, Florian Tramer, Eric Wallace, Colin Raffel, Dawn Song and others. The repeated-data work was done with Danny Hernandez, Ben Mann and Dawn Drain, among others.

Tom Brown has 2 papers in our library, in 2020–2022, most of them on Language models & pretraining, Interpretability; often with Adam Roberts, Ariel Herbert-Voss, Ben Mann.