Elamur Research⌕
☰
home / papers / 2023 / Fractality of informativity in 300 years of English scientific writing

Fractality of informativity in 300 years of English scientific writing

Paper · 2023 · Yuri Bizzoni and Stefania Degaetano-ortlieb

Summary

The paper asks whether English scientific prose, as it became more information-dense over three centuries, also developed a more regular rhythm in how information is spread across a text. The authors use the Royal Society Corpus (RSC), which holds Royal Society Proceedings and Transactions from 1665 to 1996. They measure each word's surprisal with a model conditioned on the three preceding words, trained separately for each decade. They then average surprisal per sentence, so each article becomes a sequence of sentence values. On that sequence they estimate the Hurst exponent with Adaptive Fractal Analysis (AFA), a measure of long-range persistence.

Values above 0.5 mean informativity rises and falls gradually rather than jumping around. The authors read this as support for a uniform-information account at the level of whole texts. Almost all years score above 0.5. Persistence grows through the 18th century, levels off in the 19th, and drops slightly in the 20th, which they attribute to increasing specialization. The spread of Hurst values across articles shrinks steadily over time, which suggests stylistic convergence.

Key points

  • Data: The RSC contains 47,837 texts and grows from about 2.6M tokens (1665–1699) to about 172M tokens (1950–1996).
    • Version 6.0 adds a Noisy-Channel Spell Checker to clean up OCR errors.
    • Articles with fewer than 200 sentences were excluded.
  • Surprisal: Probabilities are computed per decade from the previous three words. Period averages are normalized by vocabulary size so decades of different sizes can be compared.
  • Hurst estimation: AFA splits the series into overlapping windows of length 2n+1, fits a polynomial in each window, and stitches the fits into one smooth trend. H is the slope of log F(n) against log n. Example articles reach H = 0.92 at the high end and 0.27 at the low end.
  • Trend over time: Spearman correlation between yearly mean H and year is:
    • 0.43 up to the late 18th century;
    • −0.79 from the early 20th century;
    • not significant in between.
  • Convergence (decades from 1700): The decade mean of H shows no significant trend. Its variance and standard deviation fall significantly:
    • Spearman −0.664 and −0.619;
    • Kendall −0.517 and −0.476.

Interesting ideas

  • I1. Treating a document as a sequence of sentence-level surprisal values and measuring its persistence gives a single number for how smoothly information flows through the whole text, beyond any one sentence.
  • I2. Training the probability model separately for each time slice makes surprisal reflect the conventions of that slice rather than modern usage.
  • I3. Single words with very high surprisal tend to be domain terms (a compound chemistry term scores above 23 bits), so surprisal works as a cheap detector of terminology.
  • I4. The authors track the variance and standard deviation of a per-document statistic over time, not only its mean, and the convergence shows up only there.
  • I5. Normalizing averages by vocabulary size and dropping sequences too short for the estimator are simple guards against size artifacts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Generated or compressed-and-regenerated texts could be scored by their Hurst exponent and compared with human texts, flagging output whose information flow jumps abruptly.
  • I2 (tokens): Keeping per-period or per-genre statistics would let the tagger and lemmatizer judge oddness against the right norm, for example when handling older English in tales and fables.
  • I3 (world): High-surprisal noun spans could be proposed as candidate new entities or concepts to look up in WordNet or Wikidata before being added to the world of the text.
  • I4 (learn): Parser or reader evaluation across genres should report the spread of per-document scores, since a stable mean can hide growing or shrinking inconsistency.
  • I5 (learn): Silver-data metrics compared across corpora of very different sizes need similar normalization and minimum-length thresholds, so that size effects are not mistaken for quality changes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Saarland UniversitySaarbrücken · Germany148 works · 18,287 citations🎓Aarhus UniversityAarhus · Denmark6 works · 12 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksDecision Sciences · 209 worksLiterature and Literary Theory · 519 worksStatistics, Probability and Uncertainty · 61 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksPsycholinguistics & language acquisition · 102 works

Cite

Cite as Yuri Bizzoni and Stefania Degaetano-ortlieb. 2023. Fractality of informativity in 300 years of English scientific writing. Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.latechclfl-1.5

@inproceedings{bizzoni2023fractality,
  title = {{Fractality of informativity in 300 years of English scientific writing}},
  author = {Yuri Bizzoni and Stefania Degaetano-ortlieb},
  booktitle = {Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature},
  year = {2023},
  doi = {10.18653/v1/2023.latechclfl-1.5},
  url = {https://doi.org/10.18653/v1/2023.latechclfl-1.5}
}

References

31 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dwight Atkinson. 1999. Scientific discourse in socio-historical context: The Philosophical Transactions of the Royal Society of London, 1675-1975. Erlbaum, New York. find
  2. Douglas Biber and Susan Conrad. 2014. Variation in English: Multi-dimensional studies. Routledge. find
  3. Jianbo Gao, Jing Hu, and Wen-wen Tung. 2011. Facilitating joint chaos and fractal analysis of biosignals through nonlinear adaptive filtering. PLOS ONE, 6(9):1–8. ↗
  4. Douglas Biber and Bethany Gray. 2016. Grammatical complexity in academic English: Linguistic change in writing. Studies in English Language. Cambridge University Press, Cambridge, UK. find
  5. Jianbo Gao, Matthew L Jockers, John Laudun, and Timothy Tangherlini. 2016. A multiscale theory for the dynamical evolution of sentiment in novels. In 2016 International Conference on Behavioral, Economic and Socio-cultural Computing (BESC), pages 1–4. IEEE. find
  6. Yuri Bizzoni, Telma Peura, Mads Thomsen, and Kristoffer Nielbo. 2022. Fractal Sentiments and fairy tales - Fractal scaling of narrative arcs as predictor of the perceived quality of Andersen’s fairy tales. Journal of Data Mining & Digital Humanities, NLP4DH. ↗
  7. M.A.K. Halliday. 1988. On the language of physical science. In Mohsen Ghadessy, editor, Registers of Written English: Situational Factors and Linguistic Features, pages 162–177. Pinter, London. find
  8. Alistair Cockburn, editor. 2001. Agile Software Development. Addison-Wesley Professional, Boston, USA. find
  9. M.A.K. Halliday. 2007. On the concept of ‘Educational linguistics’. In Jonathan J. Webster, editor, The collected works of M. A. K. Halliday, Volume 9: Language and education, pages 354–367. Continuum, London. Originally published in: Giblett, R., & O’Carroll, J. (Eds.). (1990). Discipline, dialogue, difference: Proceedings of the Language in Education conference, Murdoch University, December 1989. Murdoch, Australia: Duration, pp. 23–42. find
  10. Joao Cordeiro, Pedro RM Inácio, and Diogo AB Fernandes. 2015. Fractal beauty in text. In Progress in Artificial Intelligence: 17th Portuguese Conference on Artificial Intelligence, EPIA 2015, Coimbra, Portugal, September 8-11, 2015. Proceedings 17, pages 796–802. Springer. find
  11. Nandan Kumar Das, Rajib Dey, Semanti Chakraborty, PK Panigrahi, and Nirmalya Ghosh. 2016. Probing multifractality in depth-resolved refractive index fluctuations in biological tissues using backscattering spectral interferometry. Journal of Optics, 18(12):125301. ↗
  12. Michael Cyril William Hunter et al. 1989. Establishing the new science: The experience of the early Royal Society. Boydell & Brewer Ltd. find
  13. T. Florian Jaeger. 2010. Redundancy and reduction: Speakers manage syntactic information density. Cognitive Psychology, 61(1):23–62. ↗
  14. Stefania Degaetano-Ortlieb. 2021. Measuring informativity: The rise of compounds as informationally dense structures in 20th century scientific english. In Elena Soave and Douglas Biber, editors, Corpus Approaches to Register Variation, chapter 11, pages 291–312. John Benjamins Publishing Company. find
  15. T. Florian Jaeger and Roger P. Levy. 2007. Speakers optimize information density through syntactic reduction. In Bernhard Schölkopf, John C. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 849–856. MIT Press. find
  16. Hannah Kermes, Stefania Degaetano-Ortlieb, Ashraf Khamis, Jörg Knappen, and Elke Teich. 2016. The Royal Society Corpus: From uncharted data to corpus. In Proceedings of the 10th LREC, Portorož, Slovenia. find
  17. Stefania Degaetano-Ortlieb and Andrew Piper. 2019. The scientization of literary study. In Proceedings of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 18–28, Minneapolis, USA. Association for Computational Linguistics. find
  18. Carsten Klaus, Dietrich Klakow, and Peter Fankhauser. 2019. OCR post-correction of the Royal Society Corpus based on the noisy channel model. In Proceedings of the 41. Jahrestagung der Deutschen Gesellschaft fuer Sprachwissenschaft (DGfS2019), University of Bremen, Germany. find
  19. Stefania Degaetano-Ortlieb and Elke Teich. 2019. Toward an optimal code for communication: The case of scientific English. Corpus Linguistics and Linguistic Theory, 0(0):1–33. Online print. ↗
  20. Katrin Menzel, Jörg Knappen, and Elke Teich. 2021. Generating linguistically relevant metadata for the Royal Society Corpus. Research in Corpus Linguistics, 9(1):1–18. ↗
  21. Quentin Feltgen, Benjamin Fagard, and Jean-Pierre Nadal. 2017. Frequency patterns of semantic change: Corpus-based evidence of a near-critical dynamics in language change. Royal Society Open Science, 4(11):170830. ↗
  22. Lilo Moessner. 2009. The influence of the Royal Society on 17th-century scientific writing. ICAME journal, 33:65–87. find
  23. Stefan Fischer, Jörg Knappen, Katrin Menzel, and Elke Teich. 2020. The Royal Society Corpus 6.0. providing 300+ years of scientific writing for humanistic study. In Proceedings of the International Conference on Language Resources and Evaluation (LREC) 2020, Marseille, France, May 2020, pages 794–802. find
  24. Mahdi Mohseni, Volker Gast, and Christoph Redies. 2021. Fractality and variability in canonical and non-canonical English fiction and in non-fictional texts. Frontiers in Psychology, 12. ↗
  25. Noah Moxham and Aileen Fyfe. 2018. The Royal Society and the prehistory of peer review, 1665–1965. The Historical Journal, 61(4):863–889. ↗
  26. Margery Purver. 2013. The Royal Society: Concept and Creation. Routledge. ↗
  27. Michael A. Riley, Scott Bonnette, Nikita Kuznetsov, Sebastian Wallot, and Jianbo Gao. 2012. A tutorial introduction to adaptive fractal analysis. Frontiers in Physiology, 3:371. ↗
  28. Shankha Sanyal, Archi Banerjee, Anirban Patranabis, Kaushik Banerjee, Ranjan Sengupta, and Dipak Ghosh. 2016. A study on improvisation in a musical performance using multifractal detrended cross correlation analysis. Physica A: Statistical Mechanics and its Applications, 462:67–83. ↗
  29. Claude E. Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379–423, 623–656. ↗
  30. Holger Voormann and Ulrike Gut. 2008. Agile corpus creation. Corpus Linguistics and Linguistic Theory, 4:235–251. ↗
  31. Mark Wilkinson, Michel Dumontier, and IJsbrand Aalbersberg. 2016. The FAIR Guiding Principles for scientific data management and stewardship. Nature – Scientific Data, 3(160018). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bizzoni-2023-fractality-informativity-300-years-english/