Elamur Research⌕
☰
home / papers / 1997 / Automatic Detection of Text Genre

Automatic Detection of Text Genre

Paper · 1997 · Brett Kessler, Geoffrey Nunberg and Hinrich Schutze

Summary

Kessler, Nunberg and Schutze argue that genre, meaning the functional kind of a text, matters to computational linguistics alongside topic and structure. This becomes more pressing as collections such as the Web grow more heterogeneous. They model a genre as a bundle of facets, which are properties tied to computable signals. The signals they use are simple surface cues: lexical items, punctuation and other character-level features, and derived ratios and variation measures. None of these needs tagging or parsing. The authors relabelled the Brown Corpus with their own scheme of three facets (BROW, NARRATIVE, GENRE) and trained logistic regression and 2- and 3-layer perceptrons. They then compared their surface cues with the structural cues of Karlgren and Cutting. The two cue types performed about equally well, and the cheap surface cues often beat a majority baseline. This suggests that genre detection can be done at low cost and used in tagging, parsing, word-sense disambiguation, and sorting search results.

Key points

  • Data: 499 texts selected from the 802 in the Brown Corpus. Ambiguous texts were dropped, and the rest were split into 402 training and 97 evaluation texts.
  • Facets annotated:
    • BROW, with levels POPULAR, MIDDLE, UPPER-MIDDLE and HIGH.
    • NARRATIVE, a binary facet.
    • GENRE, with levels REPORTAGE, EDITORIAL, SCITECH, LEGAL, NONFICTION and FICTION.
  • 55 surface cues were used. Counts are log-transformed so that almost 3000 possible ratios are represented implicitly.
  • Results by genre level:
    • FICTION was recognised best, with 94% of fiction texts labelled correctly.
    • REPORTAGE and SCITECH each reached 83%.
    • LEGAL was weak at 20%, though only 5 evaluation texts belonged to it.
    • NONFICTION was often confused with other classes.
  • Structural cues gave at best a marginal gain over surface cues. A sign-test-style binomial check gave p ≈ 0.41. Neural networks were better on average but less stable.

Interesting ideas

  • I1. Decomposing labels into facets. Representing a class as a combination of attributes lets a system place unseen categories and serve applications that care about only one dimension, such as opinion versus narrative. This could suit Ukrainian corpus metadata schemes.
  • I2. Log-transforming counts to get ratios for free. A linear model over logged counts can learn any ratio between them, such as sentence length or type/token ratio, without enumerating thousands of features. This is a cheap trick for any feature-based classifier.
  • I3. Variation measures as features. The authors use the spread of a quantity, for example the standard deviation of sentence length, and not only its average. This is a simple stylistic signal that transfers to authorship, register, or quality detection.
  • I4. Punctuation and character-level cues as a substitute for structural analysis. Such cues need no tagger or parser, which is attractive for low-resource languages or noisy web text.
  • I5. Evaluating per level against a "always No" baseline with a binomial significance test. This guards against inflated accuracy on rare classes and is a useful habit for small, imbalanced evaluation sets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Palo Alto Research CenterPalo Alto · United States11 works · 5,515 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as Brett Kessler, Geoffrey Nunberg and Hinrich Schutze. 1997. Automatic Detection of Text Genre. 35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/976909.979622

@inproceedings{kessler1997automatic,
  title = {{Automatic Detection of Text Genre}},
  author = {Brett Kessler, Geoffrey Nunberg and Hinrich Schutze},
  booktitle = {35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics},
  year = {1997},
  doi = {10.3115/976909.979622},
  url = {https://doi.org/10.3115/976909.979622}
}

References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Biber, Douglas. 1986. Spoken and written textual dimensions in English: Resolving the contradictory findings. Language, 62(2):384-413. 1988. Variation across Speech and Writing. Cambridge University Press. Cam-Biber. Douglas. bridge. England. find
  2. Biber. Douglas. 1992. The multidimensional approach to linguistic analyses of genre variation: An overview of methodology and finding. Computers in the Humanities, 26(5-6):331-347. find
  3. Biber. Douglas. 1995. Dimensions of Register Variation: A Cross-Linguistic Comparison. Cambridge University Press. Cambridge. England. find
  4. Butcher, S. H.. editor. 1932. Aristotle's Theory of Poetry and Fine Arts. with The Poetics. Macmillan, London. 4th edition. find
  5. Dubrow, Heather. 1982, Genre. Methuen. London and New York. find
  6. Fowler, Alistair. 1982. Kinds of Literature. Harvard University Press. Cambridge. Massachusetts. find
  7. Frye. Northrop. 1957. The Anatomy of Criticism, Princeton University Press. Princeton, New Jersey. find
  8. Hernadi, Paul. 1972. Beyond Genre. Cornell University Press. Ithaca, New York. find
  9. Hobbes, Thomas. 1908. The answer of mr Hobbes to Sir William Davenant's preface before Gondib-ert. In J.E. Spigarn, editor. Critical Essays of the Seventeenth Century. The Clarendon Press, Oxford. find
  10. Karlgren, Jussi and Douglass Cutting. 1994. Recognizing text genres with simple metrics using discriminant analysis. In Proceedings of Coling 94, Kyoto. find
  11. McCullagh, P. and J.A. Nelder. 1989. Generalized Linear Models. chapter 4, pages 101-123. Chapman and Hall, 2nd edition. find
  12. Nunberg, Geoffrey. 1990. The Linguistics of Punctuation. CSLI Publications. Stanford. California. find
  13. Rumelhart. David E.. Richard Durbin. Richard Golden. and Yves Chauvin. 1995. Backpropagation: The basic theory. In Yves Chauvin and David E. Rumelhart, editors, Back. propagation: Theory, Architectures, and Applications. Lawrence Erlbaum. Hillsdale, New Jersey, pages 1-34. find
  14. Staiger, Emil. 1959. Grundbegriffe der Poetik. Atlantis, Zurich. find
  15. Statistical Sciences. 1991. find
  16. S-PLUS Reference Manual. Statistical Sciences. Seattle, Washington. 1978. Les genres du discours. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kessler-1997-automatic-detection-text-genre/