🧑‍🔬 Aurélien Rodriguez

Aurélien Rodriguez's papers here, from 2022–2023, sit where large neural language models meet automated reasoning. The work asks how transformer models should be trained and put to use. One strand builds general-purpose language models from openly available text while weighing the cost of running them. The other uses learned models to steer search through formal mathematical proofs. These strands correspond to the topics Language models & pretraining and Reasoning.

The most cited paper is LLaMA: Open and Efficient Foundation Language Models. It introduced a family of foundation models from 7B to 65B parameters. The models were trained on many more tokens than compute-optimal scaling rules would suggest, so that smaller models stay cheaper at inference time. The training text came only from public sources, including filtered web crawls, code, encyclopedic articles, books and scientific preprints.

The second paper, HyperTree Proof Search for Neural Theorem Proving, describes Evariste, a prover that adapts AlphaZero-style search to formal mathematics. It explores proof hypertrees rather than single lines of reasoning. A transformer proposes tactics, and a critic estimates which goals can be proved. Both models keep improving online as the search runs, and the experiments include Metamath and Lean.

Taken in order, the papers move from neural theorem proving in 2022 to open language-model pretraining in 2023. Both came from teams at Meta AI. Marie-Anne Lachaux, Thibaut Lavril, Timothée Lacroix and Xavier Martinet are co-authors on both. Other collaborators include Guillaume Lample and Hugo Touvron.

Aurélien Rodriguez has 2 papers in our library, in 2022–2023, most of them on Language models & pretraining, Reasoning; often with Marie-Anne Lachaux, Thibaut Lavril, Timothée Lacroix.