Elamur Research⌕
☰
home / papers / 2023 / Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs

Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs

Paper · 2023 · Kelvin Guu, Albert Webson, Ellie Pavlick et al.

Summary

Training data attribution (TDA) tries to explain a model's behavior on a test example by identifying the training examples responsible for it. Standard tools such as influence functions and TracIn assign each training example a single score and implicitly treat these scores as summable. This ignores redundancy among examples, ordering effects and curriculum-style dependencies. The authors reframe TDA as learning a fast simulator. Given a sequence of training batches and a test example's starting loss, the simulator forecasts that loss after every step. It is fit on logs of earlier training runs and checked against real runs it has not seen. Their concrete model, Simfluence-Linear, is a Markov process in which each training example has learned multiplicative and additive effects on the test loss. It stays interpretable and runs in milliseconds. TracIn and influence functions turn out to be restricted versions of it, so all methods can be scored on the same footing. In LLM fine-tuning experiments, it roughly doubles Spearman correlation and cuts MSE by about 75% relative to earlier TDA methods. This gives attribution a clear ground truth and supports counterfactual curriculum questions.

Key points

  • Simulators are scored on MSE between predicted and actual per-step losses and on Spearman correlation of final losses. Evaluation uses 10 held-out training runs, and leave-one-out retraining is treated as a special case of this setup.
  • Simfluence-Linear updates the loss at each step from the previous loss using two learned parameters per training example (A_i, B_i). A purely additive variant, Simfluence-Additive, is shown to be equivalent to TracIn-Ideal.
  • Experiments cover both full-model tuning (FMT) and parameter-efficient IA3 tuning, on several tasks (e.g., COPA, RTE) and models. Predicted trajectories often reproduce the spiky ups and downs of real per-example losses.
  • Limitations:
    • The simulator needs at least 2n observed training steps to cover n examples.
    • It cannot handle examples absent from past runs.
    • It misses supermodular combinations, such as two separate facts that jointly enable a third.
    • It ignores optimizer state and learning rate.
  • Proposed future direction: featurized simulators that use neural encoders to produce example embeddings, aiming at better sample efficiency and generalization to unseen examples.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States533 works · 443,885 citations🎓Brown UniversityProvidence · United States85 works · 26,322 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 works

Cite

Cite as Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney and Tolga Bolukbasi. 2023. Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs. arXiv:2303.08114.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.08114 · arXiv 2303.08114

@misc{guu2023simfluence,
  title = {{Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs}},
  author = {Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney and Tolga Bolukbasi},
  year = {2023},
  doi = {10.48550/arxiv.2303.08114},
  eprint = {2303.08114},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.08114}
}

References

35 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41–48, 2009.
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  3. R Dennis Cook and Sanford Weisberg. Characterizations of an empirical influence function for detecting influential cases in regression. Technometrics, 22(4):495–508, 1980. find
  4. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment: First PASCAL Machine Learning Challenges Workshop, MLCW 2005, Sout… find
  5. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. find
  6. Yang Fan, Fei Tian, Tao Qin, Jiang Bian, and Tie-Yan Liu. Learning what data to learn. arXiv preprint arXiv:1702.08635, 2017. — [arXiv:1702.08635](https://arxiv.org/abs/1702.08635) find
  7. Satoru Fujishige. Submodular functions and optimization. Elsevier, 2005. find
  8. Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. Fastif: Scalable influence functions for efficient model interpretation and debugging. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10333–10350, 2021. find
  9. Alon Halevy, Peter Norvig, and Fernando Pereira. The unreasonable effectiveness of data. IEEE intelligent systems, 24(2):8–12, 2009. find
  10. Frank R Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69(346):383–393, 1974. find
  11. Xiaochuang Han and Yulia Tsvetkov. Influence tuning: Demoting spurious correlations via instance attribution and instance-driven updates. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4398–4409, Punta Cana, Dominican Republic, November 2021. Association for Computatio… find
  12. Xiaochuang Han and Yulia Tsvetkov. Orca: Interpreting prompted language models via locating supporting data evidence in the ocean of pretraining data. arXiv preprint arXiv:2205.12600, 2022. — [arXiv:2205.12600](https://arxiv.org/abs/2205.12600) find
  13. Xiaochuang Han, Byron C Wallace, and Yulia Tsvetkov. Explaining black box predictions and unveiling data artifacts through influence functions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5553–5563, 2020.
  14. Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67, 1970. find
  15. Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International conference on machine learning, pp. 2304–2313. PMLR, 2018. find
  16. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  17. Tae-Hoon Kim and Jonghyun Choi. Screenernet: Learning self-paced curriculum for deep neural networks. arXiv preprint arXiv:1801.00904, 2018. — [arXiv:1801.00904](https://arxiv.org/abs/1801.00904) find
  18. Diederik P Kingma and Jimmy Ba. arXiv:1412.6980, 2014.
  19. Adam: A method for stochastic optimization. arXiv preprint Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885–1894. find
  20. Pang Wei W Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influence functions for measuring group effects. Advances in neural information processing systems, 32, 2019. find
  21. Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational L…
  22. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. arXiv preprint arXiv:2205.05638, 2022.
  23. Pouya Pezeshkpour, Sarthak Jain, Sameer Singh, and Byron C Wallace. Combining feature and instance attribution to detect artifacts. In Findings of the Association for Computational Linguistics: ACL 2022, pp. 1934–1946, 2022. find
  24. Garima Pruthi, Frederick Liu, Mukund Sundararajan, and Satyen Kale. Estimating training data influence by tracking gradient descent. CoRR, abs/2002.08484, 2020. URL https://arxiv.org/abs/2002.08484. ↗
  25. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In AAAI spring symposium: logical formalizations of commonsense reasoning, pp. 90–95, 2011. find
  26. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021. find
  27. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Da… — Multitask Prompted Training Enables Zero-Shot Task Generalization
  28. Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8179–8186, 2022. find
  29. Lloyd S Shapley et al. A value for n-person games. 1953. find
  30. Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pp. 4596–4604. PMLR, 2018. find
  31. Anders Søgaard et al. Revisiting methods for finding influential examples. arXiv preprint arXiv:2111.04683, 2021. — [arXiv:2111.04683](https://arxiv.org/abs/2111.04683) find
  32. Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 927… find
  33. Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):4555–4576, 2021. find
  34. Jinghan Yang, Sarthak Jain, and Byron C. Wallace. How many and which training points would need to be removed to flip this prediction? 2023. doi: 10.48550/ARXIV.2302.02169. URL https://arxiv.org/ abs/2302.02169. ↗
  35. Chih-Kuan Yeh, Joon Sik Kim, Ian En-Hsu Yen, and Pradeep Ravikumar. Representer point selection for explaining deep neural networks. In Proc. NeurIPS, 2018. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guu-2023-simfluence-modeling-influence-individual-training/