Elamur Research⌕
☰
home / papers / 2025 / Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models

Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models

Paper · 2025 · Philipp Mondorf, Sondre Wold and Barbara Plank

Summary

Mechanistic interpretability usually locates circuits (minimal subgraphs that drive a model's behaviour) one task at a time. This leaves open whether related tasks rely on shared, reusable components. Mondorf, Wold and Plank study this with a small transformer encoder-decoder trained on PCFG SET (Hupkes et al., 2020), a synthetic, grammar-generated dataset of compositional string transformations. They search for circuits behind ten edit operations: six act on one string (e.g., copy, reverse, swap) and four combine two strings (e.g., append, remove_first). Because the outputs span several tokens, they introduce activation pruning through continuous sparsification. This method learns a deterministically reparameterized mask over the activations of the frozen model. Training keeps the masked model's predictions close to the full model's while penalizing circuit size. The circuits found are both faithful and sparse. Circuits for functionally similar operations share many nodes and partly work on each other's tasks. Merging circuits with set operations produces subnetworks that handle capabilities beyond the originals, which suggests a degree of modularity in the network.

Key points

  • Data: PCFG SET with 10 operations. The unary ones are copy, echo, repeat, reverse, swap and shift. The binary ones are append, prepend, remove_first and remove_second.
  • Method: the circuit is a binary mask over activations, relaxed to a sigmoid with a temperature β that grows exponentially per epoch up to β_max. The objective combines cross-entropy against the full model's output (equivalent to minimizing KL divergence) with an l1 penalty weighted by λ.
  • Findings: the copy circuit is the smallest and is largely contained in other circuits. The echo, repeat and swap circuits can perform copy, but the copy circuit cannot do their tasks. The union of circuits yields new functional subnetworks.
  • Limitations: the method does not guarantee completeness. The circuits found depend on the ablation choice, with mean and zero ablation giving notably different nodes. The study covers only one small model and one dataset.
  • Venue: ACL 2025 (Long Papers), pp. 14934–14955; authors are from LMU Munich/MCML and the University of Oslo.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations🎓University of OsloOslo · Norway51 works · 4,155 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Philipp Mondorf, Sondre Wold and Barbara Plank. 2025. Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.727

@inproceedings{mondorf2025circuit,
  title = {{Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models}},
  author = {Philipp Mondorf, Sondre Wold and Barbara Plank},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.727},
  url = {https://doi.org/10.18653/v1/2025.acl-long.727}
}

References

40 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vulić. 2022. Composable sparse fine-tuning for cross-lingual transfer. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1778–1796, Dublin, Ireland. Association for Computatio… find
  2. Adithya Bhaskar, Alexander Wettig, Dan Friedman, and Danqi Chen. 2024. Finding transformer circuits with edge pruning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  3. Steven Cao, Victor Sanh, and Alexander Rush. 2021. Low-complexity probing via finding subnetworks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 960–966, Online. Association for Computational L… find
  4. Stephen Casper, Shlomi Hod, Daniel Filan, Cody Wild, Andrew Critch, and Stuart Russell. 2022. Graphical clusterability and local specialization in deep neural networks. In ICLR 2022 Workshop on PAIR2Struct: Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data. find
  5. Hang Chen, Jiaying Zhu, Xinyu Yang, and Wenya Wang. 2025. Rethinking circuit completeness in language models: And, or, and adder gates. arXiv preprint arXiv:2505.10039. — [arXiv:2505.10039](https://arxiv.org/abs/2505.10039) find
  6. Abhijith Chintam, Rahel Beloch, Willem Zuidema, Michael Hanna, and Oskar van der Wal. 2023. Identifying and adapting transformer-components responsible for gender bias in an English language model. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pa…
  7. Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36:16318–16352. find
  8. Róbert Csordás, Sjoerd van Steenkiste, and Jürgen Schmidhuber. 2020. Are neural nets modular? inspecting functional modularity through differentiable weight masks. In International Conference on Learning Representations. find
  9. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. 2022. Toy models of superposition. arXiv preprint arXiv:2209.10652. find
  10. Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R Costa-jussà. 2024. A primer on the inner workings of transformer-based language models. arXiv preprint arXiv:2405.00208. find
  11. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, Singapore. Association for Computational… find
  12. Michael Hanna, Ollie Liu, and Alexandre Variengien. 2024a. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. Advances in Neural Information Processing Systems, 36.
  13. Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. 2024b. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretability. find
  14. Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. Compositionality decomposed: How do neural networks generalise? Journal of Artificial Intelligence Research, 67:757–795.
  15. Sullam Jeoung and Jana Diesner. 2022. What changed? investigating debiasing methods using causal mediation analysis. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 255–265, Seattle, Washington. Association for Computational Linguistics. find
  16. Michael Lepori, Thomas Serre, and Ellie Pavlick. 2023. Break it down: Evidence for structural compositionality in neural networks. Advances in Neural Information Processing Systems, 36:42623–42660. find
  17. David Lindner, János Kramár, Sebastian Farquhar, Matthew Rahtz, Tom McGrath, and Vladimir Mikulik. 2024. Tracr: Compiled transformers as a laboratory for interpretability. Advances in Neural Information Processing Systems, 36. find
  18. Christos Louizos, Max Welling, and Diederik P. Kingma. 2018. Learning sparse neural networks through l_0 regularization. In International Conference on Learning Representations. find
  19. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359–17372. find
  20. Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2024. Circuit component reuse across tasks in transformer language models. In The Twelfth International Conference on Learning Representations. find
  21. Joseph Miller, Bilal Chughtai, and William Saunders. 2024. Transformer circuit evaluation metrics are not robust. In First Conference on Language Modeling. find
  22. Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. 2024. The quest for the right mediator: A history, survey, and theoretical grounding of causal interp… — [arXiv:2408.01416](https://arxiv.org/abs/2408.01416) find
  23. Neel Nanda. 2023. Attribution patching: Activation patching at industrial scale. Accessed: 2024-09-19. find
  24. Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations.
  25. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895.
  26. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  27. Judea Pearl. 2001. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty and Artificial Intelligence, 2001, pages 411– 420. Morgan Kaufman. find
  28. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. Adapterfusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503…
  29. Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In The Twelfth International Conference on Learning Representations. find
  30. Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2021. Adapterdrop: On the efficiency of adapters in transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7930–7946.
  31. Pedro Savarese, Hugo Silva, and Michael Maire. 2020. Winning the lottery with continuous sparsification. Advances in neural information processing systems, 33:11380–11390. find
  32. Noam Shazeer. 2020. Glu variants improve transformer. arXiv preprint arXiv:2002.05202. find
  33. Suraj Srinivas, Akshayvarun Subramanya, and R. Venkatesh Babu. 2017. Training sparse neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 455–462. find
  34. Aaquib Syed, Can Rager, and Arthur Conmy. 2024. Attribution patching outperforms automated circuit discovery. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 407–416, Miami, Florida, US. Association for Computational Linguistics. find
  35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  36. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. Curran Associates, Inc. find
  37. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations. find
  38. Chihiro Watanabe. 2019. Interpreting layered neural networks via hierarchical modular representation. In Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12–15, 2019, Proceedings, Part V 26, pages 376–388. Springer. find
  39. Gail Weiss, Yoav Goldberg, and Eran Yahav. 2021. Thinking like transformers. In International Conference on Machine Learning, pages 11080–11090. PMLR. find
  40. Lei Yu, Jingcheng Niu, Zining Zhu, and Gerald Penn. 2024. Functional faithfulness in the wild: Circuit discovery with differentiable computation graph pruning. arXiv preprint arXiv:2407.03779. — [arXiv:2407.03779](https://arxiv.org/abs/2407.03779) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mondorf-2025-circuit-compositions-exploring-modular-structures/