📄 News Headlines Dataset For Sarcasm Detection

Summary

Sarcasm detection work has mostly used tweets labelled via hashtags. That approach produces noisy labels and informal spelling, which leaves many words missing from pretrained embeddings. It also includes replies that cannot be judged without their thread. Hand-annotated sets like the SemEval Twitter data fix the labelling problem but are too small for deep models. Misra instead lets the publisher supply the label: headlines from TheOnion's satirical "News in Brief" and "News in Photos" sections count as sarcastic, and HuffPost headlines count as real news. Pages were scraped with BeautifulSoup and Selenium, and the HuffPost side was downsampled to balance the classes. The corpus has 28,619 headlines (13,635 sarcastic), each with a label and a link to the article. Because headlines are professionally edited and self-contained, their word2vec coverage is better than that of the SemEval tweets. A brief exploration finds no visible class difference in vocabulary, length or sentiment, which suggests that sarcasm sits in subtler cues. The resource offers a cleaner, larger benchmark that also suits fake-news and humour research.

Key points

• Size: 28,619 records, of which 13,635 are sarcastic and 14,984 are non-sarcastic. The SemEval Twitter set used for comparison has 3,000 records, 2,396 of them sarcastic.
• Share of words without a word2vec embedding: 23.35% for the headlines versus 35.53% for SemEval.
• Each JSON-lines record has three fields: is_sarcastic (1/0), headline, and article_link, which points to the full article for extra text.
• Curation used BeautifulSoup for extraction and Selenium with Chrome Driver to simulate clicks to the next page. The two TheOnion categories were merged, HuffPost was downsampled, records were shuffled, and no further preprocessing was applied.
• Exploratory analysis (word clouds, headline length, sentiment) shows no clear separation between classes. Suggested further uses are fake-news detection and humour/irony studies, with a PyTorch example linked.

Interesting ideas

• I1. Take labels from the nature of the source (a satire outlet versus a regular newsroom) instead of from hashtags or annotators, which yields large, fairly clean labels cheaply.
• I2. Check simple surface statistics (length, sentiment, frequent words) per class to confirm that a shallow shortcut cannot solve the task.
• I3. Store a link to the full document next to each short text so more context can be pulled when a task needs it.
• I4. Prefer self-contained, edited texts so the phenomenon under study is not tangled with missing conversational context or spelling noise.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): a source-labelled headline set could serve as a held-out test for the irony component of prag, checking whether its explanations fire on satirical items and stay quiet on real news.
• I2 (learn): the same check works as a negative control in evaluation, showing that an irony or stance score is not just tracking length or polarity.
• I3 (world): Mova could build the world of the linked article and test whether the headline's claim clashes with it, giving an explainable incongruity signal for irony.
• I4 (tree): such headlines are a clean stress test for headline-style syntax, such as dropped articles and present tense for past events, in the perceptron parser and gparse without typo noise.