Elamur Research⌕
☰
home / papers / 2001 / The Form is the Substance: Classification of Genres in Text

The Form is the Substance: Classification of Genres in Text

Summary

The paper treats genre, meaning the conventions of form and language style that a text follows regardless of its topic, as a classification dimension that complements topic-based categorization in information retrieval and e-mail prioritization. A small illustrative query on a tagged corpus shows that restricting results to a genre such as advertisements can sharply raise precision. The authors compare two document representations. The first is a bag-of-words vector reduced with Information Gain. The second is a set of presentation features covering crude part-of-speech proportions, verb tense and tense transitions, closed-class word sets, sentence and word length statistics, readability measures, punctuation and letter case. They test each representation, and both combined, with Naïve Bayes, C4.5 and SVM classifiers on a corpus of about ten thousand documents in seven genres. Presentation features alone match or beat word features, except with Naïve Bayes. Concatenating both sets gives the best results with every classifier, and SVM reaches roughly 92% accuracy. This suggests that cheap stylistic cues can support practical genre filtering in search and mail systems.

Key points

  • The corpus has 9,705 documents, each tagged with one of seven genres. Examples include Reuters newswire, radio/TV transcripts, message boards, bulletin boards, FAQs and ads. Evaluation uses 10-fold cross-validation.
  • The feature sets are 323 word features selected via Information Gain (Yang's formulation) and 89 presentation features. A rough POS tagger built on the Porter stemming algorithm, plus the Flesch metric, supplies some of the presentation features.
  • Average recall when each classifier must always pick a genre:
    • SVM: 85.4% with words, 87.1% with presentation features, 92.1% combined.
    • Naïve Bayes: drops to 64.0% with presentation features alone.
  • Some documents can be marked "unknown" (Naïve Bayes score below 0.5, or a negative SVM score). Under this threshold, SVM with combined features gets 94.9% precision, 84.0% recall and 89.1% F1.
  • In the illustrative query, adding a genre restriction to "house" raised precision from 2% to 43%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Part-of-speech tagging

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{dewdney2001form,
  title = {{The Form is the Substance: Classification of Genres in Text}},
  author = {Dewdney, Nigel and VanEss-Dykema, Carol and MacMillan, Richard},
  booktitle = {Proceedings of the ACL 2001 Workshop on Human Language Technology and Knowledge Management},
  year = {2001},
  doi = {10.21236/ada460898},
  url = {https://doi.org/10.21236/ada460898}
}

References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Biber, D. (1995) " Variation Across Speech and Writing", Cambridge University Press, New York. find
  2. Berger, A., S.Della Pietra, V.Della Pietra, ˆ(1996) "A Maximum Entropy Approach to Natural Language Processing", Computational Linguistics, vol. 22 No. 1, pp39−71. find our card
  3. Burges, C. (1998) "A Tutorial on Support Vector Machines for Pattern Recognition", Data Mining and Knowledge Discovery, No. 2. find
  4. Cherney, L. (1999) "Conversation and Community: Chat in a Virtual World", CSLI Publications, Stanford University. find
  5. Cohen, W.W. (1996) "Learning Rules that Classify E− Mail", AAAI Spring Symposium on Machine Learning in Information Access. find
  6. Cristianini, N., J.Shaw−Taylor, (2000) "Introduction to Support Vector Machines", Cambridge University Press, . find
  7. Flesch, R. (1974) "The Art of Readable Writing", Harper and Row, New York. find
  8. Illouz, G., B.Habert, H.Folch, S.Heiden, S.Fleury, P.Lafon, S.Prevost (2000) "TyPTex: Genric features for Text Profiler", Proc. find
  9. RIAO 2000, Paris, France. find
  10. Joachims, T. (1998) "Text Categorization with Support Vector Machines: Learning with Many Relevant Features", European Conference on Machine Learning. find our card
  11. Karlgren, J., and D Cutting (1994) "Recognizing Text Genres with Simple Metrics Using Discriminant Analysis". In Proc. of the 15 th International conference on Computational Linguistings (COLING’94) find
  12. Kessler, B., G Nunberg, H. Schutze (1997) "Automatic Detection of Text Genre". In Proc. if 35 th Annual Meeting of the ACL and 8 th Conference of ECACL. find
  13. May, A. (1997) "Automatic Classification of E−Mail Messages by Message Type", JASIS, vol 48 No.1, pp32−39. find
  14. Mitchell, T.M. (1997) "Machine Learning", McGraw− Hill, Boston, Massachusetts. find
  15. Oard, D.W., N. De Claris, B.J.Dorr, C.Faloutsos, (1994) "On Automatic Filtering of Multilingual Texts", Proceedings of IEEE International Conference on Sytems, Man, and Cybernetics, pp1645−1650, Vsan Antonio Texas. find
  16. Osuna, E. R.Freund, and F.Girosi, (1997) "Training support vector machines: an application to face recognition", IEEE Conference on Computer Vision and Pattern Recognition. find
  17. Pannu, Anandeep, Sycara. (1996) "Learning Text Filtering Preferences", Symposium on Machine Learning and Information Processing, AAAI Symposium Series, Stanford, Ca., March’96. find
  18. Porter, M.E. (1980) "An Algorithm for Suffix Stripping", Program Vol. 14, No. 3, pp318−327, July’80. find
  19. Quinlan, J.R. (1993) "C4.5: Programs for Machine Learning", Morgan Kaufmann, California. find our card
  20. Sahami, M., S.Dumais, D.Heckerman, E.Horvitz, (1998) "A Bayesian Approach to Filtering Junk E−Mail" AAAI Workshop Technical Report WS−98−05. find
  21. Stamatatos, E., N. Fakotakis, and G. Kokkinakis (2000) "Text Genre Detection Using Common Word Frequencies". In the Proc. of the 18 th International Confernece on Computational Linguistics (COLING2000) find
  22. SVM−light webpages URL: http://ais.gmd.de/~thorsten/svm_light/TREC webpages. URL: http://trec.nist.gov ↗
  23. Yang, Y., J.Pedersen, (1997) "A Comparative Study on Feature Selection in Text Categorization", Proceedings of the 1997 International Conference on Machine Learning (ICML), pp.412−420. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dewdney-2001-form-is-substance-classification-genres/