📄 Making Computers Laugh: Investigations in Automatic Humor Recognition

Summary

The paper tests whether standard text classification can tell humorous sentences from serious ones. It focuses on one-liners: very short jokes whose comic effect sits entirely in a single sentence. The authors built about 16,000 one-liners by bootstrapping from the Web, starting with only ten seed jokes. For negative examples they used three sets of short sentences chosen to resemble jokes in length and style: Reuters news titles, proverbs, and BNC sentences retrieved for their content similarity to each joke. They compared three kinds of features. The first were humor-oriented stylistic cues: alliteration, antonymy and slang. The second were content-based features in an ordinary text-classification setup, and the third combined both in a stacked learner. On balanced data, accuracy ranged from 79.15% (One-liners/BNC) to 96.95% (One-liners/Reuters), always clearly above the baselines. Learning curves level off after roughly 10,000 positive examples. The authors read this as a sign that better features, not more data, are the way forward. The work is presented as the first evidence that automatic classification can handle humor recognition.

Key points

• Data collection: two bootstrapping rounds from 10 seed one-liners yielded about 24,000 candidates.
  • Removing near-duplicates with a longest-common-subsequence similarity measure left about 16,000.
  • A manual check of 200 random items found roughly 9% noise.
• Negative sets: Reuters titles (one year of newswire, 1996–1997), an online proverb collection, and BNC sentences.
  • The BNC sentences were retrieved with tf.idf and cosine similarity in the Smart system.
  • All negatives were limited to single sentences of about 10–15 words.
• Stylistic features: these were alliteration, antonymy and slang, with alliteration detected through an index over the CMU pronunciation dictionary.
  • Content features were used with classifiers including Naïve Bayes.
  • The two feature types were also merged via stacking.
• Skewed data (75% negative / 25% humorous): all methods still beat the 75% baseline.
  • Content features with Naïve Bayes reached 96.19% (Reuters), 81.56% (BNC) and 87.86% (Proverbs).
  • Stylistic features alone reached 87.49%, 77.62% and 76.20%.
• Learning curves: gains stop around 60% of the data. With BNC negatives the curve even dips slightly at the end, which the authors attribute to noise.

Interesting ideas

• I1. Bootstrapping with two constraints. The collector accepts a page only if its URL contains a theme keyword. It then harvests only sibling items in the same HTML list as a known example. This cheap, genre-aware scraping pattern could gather proverbs, slogans or headlines for low-resource languages such as Ukrainian.
• I2. Hard negatives by design. The negatives match the positives in length, register and even topic, as with the BNC sentences retrieved per joke. This stops the classifier from relying on trivial surface cues, and the same approach suits any style or genre detection task.
• I3. Several negative sets as a difficulty scale. Evaluating against easy (news titles) and hard (topically similar plain sentences) negatives shows how much of the result depends on the contrast class, not only on the target phenomenon.
• I4. Learning curves as a planning tool. Plotting accuracy against training size helps decide whether to invest in more data or in richer features. A plateau points toward features, and a late drop can expose label noise.
• I5. Linguistic devices as explicit features. Simple, interpretable signals, such as alliteration from a pronunciation dictionary or antonym pairs from a lexical resource, provide a transparent baseline. They can also be stacked with bag-of-words models.