📄 Positivity of the English language

Summary

The paper asks whether English, taken as a whole, leans emotionally positive, negative or neutral. Earlier studies gave mixed answers, partly because they picked words in ways that could bias the outcome. The authors chose words only by how often they are used. The 5000 most frequent words were drawn from each of four large corpora: Twitter, the English portion of Google Books, song lyrics, and The New York Times. Merging these lists gave 10,222 unique words. Workers on Amazon's Mechanical Turk rated each word 50 times for the happiness it evokes, on a 1–9 scale where 5 is neutral. The result is the labMT 1.0 resource. In every corpus the distribution of average word scores has a single peak and is skewed toward the positive side. Positive words clearly outnumber negative ones. This positive lean barely changes between very common and rarer words. The finding gives quantitative support to the Pollyanna Hypothesis. labMT also serves as a reusable valence lexicon for measuring sentiment in text.

Key points

• The labMT 1.0 lexicon covers 10,222 words, built from 501,110 human ratings (50 per word). Scores range from 'laughter' at 8.50 down to 'terrorist' at 1.30, while function words such as 'the' (4.98) sit near 5.
• The share of words scoring above neutral is 72.00% for TW, 78.80% for GB, 78.38% for NYT and 64.14% for ML. The most common score is above 5 in three corpora and exactly neutral for lyrics.
• The authors compared 1000-word frequency bands using Kolmogorov–Smirnov tests. Of 40 pairwise comparisons, 29 could plausibly come from the same distribution (p > 10⁻²).
• Happiness and frequency rank are only weakly linked. Spearman ρ ranges from −0.013 for GB (not significant, p = 0.35) to −0.103 for TW. Linear slopes are of order −10⁻⁵.
• The words with the highest rating spread are mostly profanity, alcohol, religion and politics terms (e.g., 'fucking', 'whiskey', 'capitalism', 'churches'). The authors list these separately from the clearly positive and negative words.

Interesting ideas

• I1. Choosing items purely by usage frequency, instead of by hand-curated "emotional" lists, avoids building the expected answer into the sample.
• I2. Storing the spread of ratings alongside the mean shows which words people disagree on, a signal of ambiguity or contested meaning.
• I3. Splitting the vocabulary into frequency bands and testing whether the bands share one distribution (KS tests) checks whether a property holds across scale.
• I4. The authors stress that aggregate word-count metrics should always be shown with concrete example words as a commonsense check.
• I5. Merging top-word lists from very different genres (tweets, books, news, lyrics) gives a resource that covers both noisy and polished language.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Build lexicons of values and attitudes by starting from frequency-ranked vocabulary, so the compiled knowledge layer has no blind spots for ordinary words.
• I2 (frames): When inferring speaker attitude or irony, treat high-variance words as unreliable evidence and require support from frame context before assigning polarity.
• I3 (learn): Evaluate parsers and taggers separately on frequent versus rare lemma bands and test whether the error distributions differ, exposing frequency-dependent weaknesses.
• I4 (learn): Attach sampled concrete cases to every automated silver-data or repair-rule metric, so a human can spot when a good-looking number hides nonsense.
• I5 (tokens): Assemble lemma and morphology test vocabularies from several registers, including informal ones, so the tokenizer and tagger are checked on slang and nonstandard spelling too.