Elamur Research⌕
☰
home / papers / 2016 / Machine Learning for Metrical Analysis of English Poetry

Machine Learning for Metrical Analysis of English Poetry

Paper · 2016 · Manex Agirrezabal, Iñaki Alegria and Mans Hulden

Summary

The paper treats automatic scansion of English poetry as sequence labeling: each syllable is marked stressed or unstressed. The task looks easy because most lines follow a fixed meter, but poets often break the pattern, so whole-line correctness is hard to reach. The authors train and test on For Better For Verse (4B4V), a hand-scanned corpus from the University of Virginia. They compare two rule-based systems, ZeuScansion (their baseline) and Scandroid, with several learned models: greedy classifiers such as a Linear SVM, an HMM, and CRFs with growing feature sets. They use 10-fold cross-validation and report accuracy per syllable and per complete line. The best system is a CRF with 64 features, at about 91.4% per syllable and 55.3% per line. This beats both rule-based tools, especially on whole lines. The work sets a clear supervised baseline for scansion. It also shows that high syllable scores hide weak line-level performance on irregular verse.

Key points

  • Data: the 4B4V corpus (For Better For Verse), built for an online scansion tutorial and available on GitHub. In each fold, roughly 900 lines are used for training and 100 for testing.

  • Using syllables only, the systems score as follows:

    SystemPer syllablePer line
    ZeuScansion86.78%26.21%
    Scandroid89.78%42.95%
    HMM90.43%49.88%
    CRF88.13%43.93%

  • With richer features, the CRF improves:

    • 10 basic features: 89.66% per syllable, 50.16% per line.
    • All 64 features: 91.41% per syllable, 55.30% per line.
    • Feature selection gave only a small gain that was not statistically significant.

  • Output diversity differs by model:

    • The CRF produced about 30 distinct line patterns.
    • The Linear SVM produced around 50 patterns that occurred only once.
    • Iambic pentameter (x/x/x/x/x/) was the most frequent output for both.

  • Planned next steps are multi-level or regression-style stress (as in Hayes et al. 2012), LSTMs (preliminary results are comparable or better), other languages, and unsupervised learning.

Interesting ideas

  • I1. Reporting accuracy both per unit and per whole sequence shows that around 90% token accuracy can coexist with only half of the sequences being fully correct.
  • I2. Counting how many distinct whole-sequence outputs a model produces reveals whether it over-regularizes (CRF) or scatters (greedy SVM), which is a cheap diagnostic of model bias.
  • I3. Global sequence models enforce coherence across a line, which helps on regular input but hurts the genuine outliers; the right choice depends on how often exceptions occur.
  • I4. The same word (with, in the Milton example) can be stressed or unstressed depending on its position, so a label cannot come from the lexical item alone.
  • I5. Some lines have several acceptable scansions, and binary labels force artificial conflicts; the authors propose graded stress levels instead.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova's evaluation could report the share of sentences whose entire UD tree or frame set is exactly right, next to per-token scores, so that local accuracy cannot hide broken outputs.
  • I2 (tree): Comparing the variety of tree shapes produced by the perceptron parser and gparse against the treebank could flag a parser that collapses rare constructions into the dominant pattern.
  • I3 (tree): When choosing between greedy and globally scored decoding, Mova could measure gains and losses separately on frequent and rare constructions instead of using a single average.
  • I4 (tokens): Taggers and morphology for English function words should rely on context features rather than per-word defaults, and test sets should include such context-dependent cases.
  • I5 (learn): Silver data built from several witness parsers could keep multiple accepted analyses or graded agreement instead of a single forced label, so that true ambiguity is not counted as error.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of the Basque CountryLeioa · Spain57 works · 16,520 citations🎓University of Colorado BoulderBoulder · United States69 works · 51,069 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Manex Agirrezabal, Iñaki Alegria and Mans Hulden. 2016. Machine Learning for Metrical Analysis of English Poetry. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{agirrezabal2016machine,
  title = {{Machine Learning for Metrical Analysis of English Poetry}},
  author = {Manex Agirrezabal, Iñaki Alegria and Mans Hulden},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1074/}
}

References

38 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Manex Agirrezabal, Jeffrey Heinz, Mans Hulden, and Bertol Arrieta. 2014. Assigning stress to out-of-vocabulary words: three approaches. International Conference on Artificial Intelligence, Las Vegas, NV, 27:105–110. find
  2. Manex Agirrezabal, Aitzol Astigarraga, Bertol Arrieta, and Mans Hulden. 2016. ZeuScansion: a tool for scansion of English poetry. Journal of Language Modelling, 4(1):3–28. ↗
  3. Terry VF Brogan. 1981. English versification, 1570-1980: a reference guide with a global appendix. Johns Hopkins University Press. find
  4. Noam Chomsky and Morris Halle. 1968. The sound pattern of English. Harper & Row, New York. find
  5. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20(3):273–297. ↗
  6. Alex Estes and Christopher Hench. 2016. Supervised machine learning for hybrid meter. In Proceedings of the Fifth Workshop on Computational Linguistics for Literature, pages 1–8. Association for Computational Linguistics. find
  7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. The Journal of Machine Learning Research, 9:1871–1874. Software available at http://www.csie.ntu.edu.tw/˜cjlin/liblinear. ↗
  8. Yoav Freund and Robert E. Schapire. 1999. Large margin classification using the perceptron algorithm. Machine learning, 37(3):277–296. ↗
  9. Stephen R. Garner. 1995. Weka: The Waikato environment for knowledge analysis. In Proceedings of the New Zealand computer science research students conference, pages 57–64. Citeseer. find
  10. Pablo Gervás. 2000. A logic programming application for the analysis of Spanish verse. In Computational Logic—CL 2000, pages 1330–1344. Springer. find
  11. Erica Greene, Tugba Bodrumlu, and Kevin Knight. 2010. Automatic analysis of rhythmic poetry with applications to generation and translation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 524–533. Association for Computational Linguistics. find
  12. Peter L Groves. 1998. Strange music: the metre of the English heroic line, volume 74. English Literary Studies. find
  13. Péter Halácsy, András Kornai, and Csaba Oravecz. 2007. HunPos: an open source trigram tagger. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 209–212. Association for Computational Linguistics. find
  14. Morris Halle and Samuel Jay Keyser. 1971. English stress: Its form, its growth, and its role in verse. Harper and Row. find
  15. Charles O Hartman. 1996. Virtual muse: experiments in computer poetry. Wesleyan University Press. find
  16. Charles O. Hartman. 2005. The Scandroid 1.1. Software available at http://oak.conncoll.edu/cohar/ Programs.htm. ↗
  17. Bruce Hayes and Colin Wilson. 2008. A maximum entropy model of phonotactics and phonotactic learning. Linguistic inquiry, 39(3):379–440. ↗
  18. Bruce Hayes, Colin Wilson, and Anne Shisko. 2012. Maxent grammars for the metrics of Shakespeare and Milton. Language, 88(4):691–731. ↗
  19. Bruce Hayes. 1995. Metrical stress theory: Principles and case studies. University of Chicago Press. find
  20. Bruce Hayes. 2011. Introductory Phonology. John Wiley & Sons. find
  21. Malcolm Hayward. 1996. Analysis of a corpus of poetry by a connectionist model of poetic meter. Poetics, 24(1):1–11. ↗
  22. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the eighteenth international conference on machine learning, ICML, volume 1, pages 282–289. find
  23. Harry M Logan. 1988. Computer Analysis of Sound and Meter in Poetry. College Literature, pages 19–24. find
  24. Gareth McAleese. 2007. Improving Scansion with Syntax: an Investigation into the Effectiveness of a Syntactic Analysis of Poetry by Computer using Phonological Scansion Theory. Ph.D. thesis, Open University. find
  25. Andrew McCallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, pages 188–191. Association for Computational Linguistics.
  26. Borja Navarro-Colorado. 2015. A computational linguistic approach to Spanish Golden Age Sonnets: metrical and semantic aspects. Computational Linguistics for Literature, page 105. find
  27. Naoaki Okazaki. 2007. CRFsuite: a fast implementation of Conditional Random Fields (CRFs). Software available at http://www.chokkan.org/software/crfsuite/. ↗
  28. William Pickering. 1832. The Poetical Works of John Milton, volume 2. William Pickering. find
  29. Marc R Plamondon. 2006. Virtual verse analysis: Analysing patterns in poetry. Literary and Linguistic Computing, 21(suppl 1):127–141. ↗
  30. Alex Preminger, Frank J Warnke, and Osborne Bennett Hardison Jr. 2015. Princeton encyclopedia of poetry and poetics. Princeton University Press. find
  31. Lawrence R Rabiner. 1989. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286. ↗
  32. Theodore Roethke. 2011. The collected poems of Theodore Roethke. Anchor. find
  33. Frank Rosenblatt. 1958. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6):386. ↗
  34. Dante Gabriel Rossetti. 1881. Poems: A New Edition. London: Ellis & White. find
  35. David E Rumelhart, James L. McClelland, PDP Research Group, et al. 1988. Parallel distributed processing, volume 1. IEEE. find
  36. Terrence J. Sejnowski and Charles R. Rosenberg. 1987. Parallel networks that learn to pronounce English text. Complex systems, 1(1):145–168. William Shakespeare. 1609. Shakespeare’s sonnets. Thomas Thorpe. find
  37. Timothy Steele. 1999. All the fun’s in how you say a thing: an explanation of meter and versification. Ohio University Press Athens. find
  38. Herbert F Tucker. 2011. Poetic data and the news from poems: A for better for verse memoir. Victorian Poetry, 49(2):267–281. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/agirrezabal-2016-machine-learning-metrical-analysis-english/