📄 Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

Summary

Kirov and Cotterell revisit a landmark dispute in cognitive science: can neural networks model how English speakers form past tense verbs? Rumelhart and McClelland's 1986 network encoded words as Wickelphone trigrams inside a fixed-size feed-forward design. Pinker and Prince (1988) criticised it sharply, mainly for low accuracy and implausible errors, and their critique left many linguists skeptical of connectionism ever since.

The authors test whether a modern Encoder-Decoder RNN, borrowed from machine translation, answers those objections. They train it on IPA strings of verb stems and inflected forms without supplying phonological features. They then check how it generalizes to unseen verbs, what errors it makes and how it learns over time, including a multi-task setting with several inflections.

The model handles both regular and irregular past tense almost perfectly and avoids blend errors. It also learns irregulars in an oscillating, verb-specific way. The authors conclude that current neural models deserve renewed consideration in linguistic and cognitive modeling, while open questions about learning biases and child-like development remain.

Key points

• The original R&M model reached roughly 67% accuracy, while the Encoder-Decoder (ED) network learns the past tense nearly perfectly. This covers both regular and irregular verbs and generalizes to held-out verbs.
• The ED network has no built-in phonological or morphological knowledge and learns its own phoneme representations. It maps strings of any length to strings of any length without Wickelphones, and it can assign an exact probability to any output.
• The degenerate errors Pinker and Prince noted in R&M, such as regular/irregular blends of the ate → ated type, do not appear in the ED model's output.
• The input distribution was held fixed during training. Under these conditions the model shows the micro U-shaped oscillations for irregulars described by Plunkett and Marchman. It shows no macro U-shape, however, and the effect is not uniform across all irregulars.
• Training jointly on several mappings (e.g., adding the gerund) makes learning curves steeper, although the gain in final accuracy is not significant. Unresolved issues include homophones (write → wrote vs. right → righted) and the model's capacity to learn unattested patterns like mirror-image reversal.