📄 Adjective Density as a Text Formality Characteristic for Automatic Text Classification: A Study Based on the British National Corpus

Summary

The paper asks whether a single lexical statistic, adjective density (the share of adjectives among all word tokens), can track how formal a text category is and help classify texts automatically. The authors built a balanced sub-corpus of the British National Corpus (BNC) with about 3 million tokens from each of eight categories, split 80/20 into training and test data. Seven linguists ranked the categories by formality, and this human ordering was compared with the ordering produced by adjective density using Spearman correlation and linear regression. The two orderings agreed strongly. Spoken material had the fewest adjectives and academic prose the most. A regression model fitted on training data carried over to the unseen portion. A Weka Naïve Bayes classifier using only adjective density reached a weighted F-measure of 0.606 on five well-separated categories. Nouns scored higher, while adverbs and verbs scored lower. The work matters because one cheap, interpretable feature captures much of the speech/writing and academic/non-academic contrast that earlier studies modelled with dozens of features.

Key points

• Data: 8 BNC categories, two spoken (Conversation, Other spoken) and six written (Fiction, Newspapers, Academic prose, Non-academic prose, Other published writing, Unpublished writing), about 24.3M tokens in total, with an 80/20 train/test split.
• Human ranking came from 7 raters with high agreement (ICC = 0.857, p < 0.001). Against the density-based ranking, Spearman r_s = 0.833 (significant at 0.02), and the linear fit gave r = 0.988 and r² = 0.977.
• Adjective density ranged from 3.49% in Conversation to 9.63% in Academic prose. Fiction was the least adjective-rich written category, sitting next to the spoken ones.
• Naïve Bayes over 1,180 texts with density as the only feature did poorly on all 8 classes (weighted F = 0.267). For example, every Conversation file was labelled Other spoken.
• After the vaguer and overlapping classes were removed, five categories gave a weighted F of 0.606, with Conversation at F = 0.880. Under the same setup, nouns reached 0.756, adverbs 0.511 and verbs 0.448.

Interesting ideas

• I1. Check an automatic scalar measure by comparing its ordering of categories with a human ordering via rank correlation, and verify rater agreement (ICC) first so the human reference is trustworthy.
• I2. Measure how far a single interpretable feature gets before building a rich model, and run the same test for each part of speech to see which word class carries the signal.
• I3. Read the confusion matrix to find categories that are internally mixed or underpopulated (here Other spoken also contains dialogue) and treat them as a labelling problem rather than a model failure.
• I4. Sample an equal token volume per category so that relative frequencies are comparable, while noting that document length can still bias per-file statistics.
• I5. A text's formality level, estimated from word-class densities, can serve as context for interpreting how something is said.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Any graded quantity Mova produces could be sanity-checked this way against a small, agreement-tested human ranking before being trusted as a signal.
• I2 (tokens): Proportions of tagger outputs per document (ADJ, NOUN, ADV, VERB) give a cheap, explainable register estimate that downstream stages could consult.
• I3 (learn): When silver-data evaluation shows systematic confusions, first ask whether the target classes or genre labels are well defined before adding repair rules.
• I4 (learn): Treebank and silver-corpus mixes across genres should be size-balanced, and per-document metrics should be logged with length so length effects can be separated out.
• I5 (frames): A register estimate could inform pragmatic judgments, for example flagging casual phrasing in an otherwise academic-density text as a possible marked or ironic choice.