Elamur Research⌕
☰
home / papers / 2019 / The emergence of number and syntax units in LSTM language models

The emergence of number and syntax units in LSTM language models

Paper · 2019 · Yair Lakretz, German Kruszewski, Theo Desbordes et al.

Summary

LSTM language models handle long-distance subject–verb number agreement well, but it was unclear whether they rely on real syntactic structure or on shallow cues. Lakretz et al. study the internal workings of a pretrained Wikipedia LSTM (from Gulordava et al., 2018) at the level of individual neurons. They use the Linzen et al. (2016) agreement test set, a family of synthetic number-agreement tasks with controlled structures and congruent or incongruent intervening nouns, and a dataset annotated with syntactic depth. Their tools are single-unit ablation, analysis of unit activity, and inspection of recurrent weights.

They find that two dedicated units carry the subject's number over long distances, one for singular and one for plural. A separate unit that tracks syntactic structure gates these number units, signalling when to store number and when to release it. A more distributed code handles agreement only when subject and verb are close. These results suggest that an LSTM trained on raw text can learn structure-sensitive processing. The study also yields predictions that can be tested in neuroscience.

Key points

  • Model: the Gulordava et al. (2018) LSTM, with a 650-dimensional embedding layer, two 650-unit hidden layers, and a 50,000-word vocabulary. It was trained on Wikipedia with no fine-tuning for agreement.
  • Synthetic tasks: the NA-tasks are Simple, Adv, 2Adv, CoAdv, NamePP, NounPP and NounPPAdv, with 600 sentences per condition. The syntactic depth set contains 4,033 positions drawn from 1,303 sentences. The data are released on GitHub (FAIRNS).
  • Number units: ablating unit 776 (plural) or unit 988 (singular) brought accuracy close to chance in incongruent conditions, but only for subjects of the matching number.
  • Syntax unit: unit 1150 has unusually strong weights to the number units' gates. These are positive to the forget gates (z = 8.1 and 11.2) and negative to the input gates (z = −16.2 and −7.2), which acts as a "remember" signal.
  • Two mechanisms: a distributed short-range encoding of number also exists, but it fails to carry number across embedded phrases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MetaMenlo Park · United States158 works · 211,459 citations🎓University of AmsterdamAmsterdam · Netherlands89 works · 81,337 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 2 Language models & pretraining · 1,343 works Syntax & grammar formalisms · 161 works

Cite

Cite as Yair Lakretz, German Kruszewski, Theo Desbordes, Dieuwke Hupkes, Stanislas Dehaene and Marco Baroni. 2019. The emergence of number and syntax units in LSTM language models. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1002 · arXiv 1903.07435

@inproceedings{lakretz2019emergence,
  title = {{The emergence of number and syntax units in LSTM language models}},
  author = {Yair Lakretz, German Kruszewski, Theo Desbordes, Dieuwke Hupkes, Stanislas Dehaene and Marco Baroni},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1002},
  eprint = {1903.07435},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1002}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In Proceedings of ICLR Conference Track, Toulon, France. Published online: https://openreview.net/group? id=ICLR.cc/2017/conference. ↗ our card
  2. Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. In Proceedings of ICLR Conference Track, Toulon, France. find
  3. Jean-Philippe Bernardy and Shalom Lappin. 2017. Using deep neural networks to learn syntactic agreement. Linguistic Issues in Language Technology, 15(2):1–15. find
  4. Jeffrey Bowers. 2009. On the biological plausibility of grandmother cells: Implications for neural network theories in psychology and neuroscience. Psychological Review, 116(1):220–251. agreement. Poster presented at the Learning Language in Humans and in Machines conference, online at: https://osf.… ↗
  5. Shammur Chowdhury and Roberto Zamparelli. 2018. RNN simulations of grammaticality judgments on long-distance dependencies. In Proceedings of COLING, pages 133–144, Santa Fe, NM. find
  6. Adhiguna Kuncoro, Chris Dyer, John Hale, Dani Yogatama, Stephen Clark, and Phil Blunsom. 2018b. LSTMs can learn syntax-sensitive dependencies well, but modeling structure makes them better. In Proceedings of ACL, pages 1426–1436, Melbourne, Australia. our card
  7. Lieke Gelderloos and Grzegorz Chrupała. 2016. From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learning. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1309–1319. find
  8. Esther Kutter, Jan Bostroem, Christian Elger, Florian Mormann, and Andreas Nieder. 2018. Single neurons in the human brain encode numbers. Neuron, 100(3):753–761. find
  9. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the EMNLP BlackboxNLP Workshop, pages 240–248, Brussels, Belgium. find
  10. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in NLP. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 681–691. our card
  11. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of NAACL, pages 1195–1205, New Orleans, LA. our card
  12. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521– 535. our card
  13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  14. Tal Linzen and Brian Leonard. 2018. Distinct patterns of syntactic agreement errors in recurrent networks and humans. In Proceedings of CogSci, pages 692– 697, Austin, TX. find
  15. Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018. Visualisation and ’diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure. Journal of Artificial Intelligence Research, 61:907–926. find
  16. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202. find
  17. Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? on the ability of lstms to understand negative polarity items. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 222–231. find
  18. Matthew Nelson, Imen El Karoui, Kristof Giber, Xiaofang Yang, Laurent Cohen, Hilda Koopman, Sydney Cash, Lionel Naccache, John Hale, Christophe Pallier, and Stanislas Dehaene. 2017. Neurophysiological dynamics of phrase-structure building during sentence processing. Proceedings of the National Acade… find
  19. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. 2016. Visualizing and understanding recurrent networks. In Proceedings of ICLR Workshop Track, San Juan, Puerto Rico. Published online: https://openreview.net/group? id=ICLR.cc/2016/workshop. ↗ our card
  20. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. https://arxiv.org/abs/1704. 01444. ↗ our card
  21. Yova Kementchedjhieva and Adam Lopez. 2018. ‘In-dicatements’ that character language models learn English morpho-syntactic units and regularities. In Proceedings of the EMNLP Workshop on analyzing and interpreting neural networks for NLP, Brussels, Belgium. In press. find
  22. Zhiyuan Tang, Ying Shi, Dong Wang, Yang Feng, and Shiyue Zhang. 2017. Memory visualization for gated recurrent neural networks in speech recognition. In Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on, pages 2736–2740. IEEE. find
  23. Jean-Rémi King and Stanislas Dehaene. 2014. Characterizing the dynamics of mental representations: The temporal generalization method. Trends in Cognitive Sciences, 18(4):203–210. find
  24. Ethan Wilcox, Roger Levy, Takashi Morita, and Richard Futrell. 2018. What do RNN language models learn about filler–gap dependencies? In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 211–221. our card
  25. Adhiguna Kuncoro, Chris Dyer, John Hale, and Phil Blunsom. 2018a. The perils of natural behavioral tests for unnatural models: The case of number find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lakretz-2019-emergence-number-syntax-units-lstm/