📄 Stance Detection in Fake News A Combined Feature Representation

Summary

The paper tackles stance detection for fake news. Given a news headline and an article body, the system decides whether the article agrees with the headline, disagrees with it, discusses it neutrally, or is unrelated. The authors use the Fake News Challenge (FNC-1) dataset and avoid heavy deep architectures. Instead, they join three fairly simple feature groups:

• the n-gram and overlap features from the official FNC-1 baseline;
• seven categories of stance cue words, enlarged using Google News word2vec neighbours;
• averaged word2vec embeddings plus title–article cosine similarity, built only from verbs, nouns and adjectives.

These features feed a small feed-forward neural network. The system reaches 0.596 Macro F1, about 0.013 below the best published result (stackLSTM), and well above the official baseline. The authors also measure how much each cue-word category contributes, both overall and per stance class. The work matters because it shows that a light, interpretable feature set can come close to state-of-the-art results on a heavily imbalanced fact-checking benchmark.

Key points

• Data: FNC-1 covers 300 topics, with 49,972 training and 25,413 test title–article pairs. The classes are highly skewed: Unrelated 73.13%, Discuss 17.82%, Agree 7.36%, Disagree 1.68%. Following Hanselowski et al. (2018), Macro F1 is used instead of the official weighted score.
• Cue words: Drawn from Bahuleyan and Vechtomova (2017), the cue-word groups cover Fake, Negation, Knowledge, Report, Doubt, Denial and Belief. Each word was expanded with its 5 nearest word2vec neighbours, which added 2.5% to the final score.
• Classifier: The model is a neural network with two ReLU hidden layers, a Softmax output, the Adam optimizer and a batch size of 200. It beat SVM, Gradient Boost, Random Forest and Naive Bayes; a linear SVM came close at about 58%.
• Scores (Macro F1): Full system 0.596; embeddings alone 0.488; cue words alone 0.25; FNC-1 baseline 0.454; Athene 0.604; stackLSTM 0.609.
• Feature importance: Information Gain from Gradient Boost ranked Report as the most useful category and Denial and Knowledge as the least useful. Linear SVM coefficients showed that disagreement-related categories weigh more for the Disagree class than for Agree.