📄 Question Identification in Arabic Language Using Emotional Based Features

Summary

The paper tackles a binary task for Arabic social media: deciding whether a post, tweet or comment contains a question that actually expects an answer. Some texts look like questions but are not, such as advertisements, quoted questions, or questions that already come with a reply. The motivation is customer service: real questions can be auto-answered or routed to the right staff.

The authors reuse the feature set of Hasanain et al. as a baseline. It has two parts:

• n-gram lexical features with IDF weights;
• surface features such as text length, URLs, hashtags, mentions, punctuation and question keywords.

On top of this they add a new group of emotion-based features, drawn from a sentiment lexicon of positive and negative words, multiword phrases, emojis and emoticons that they built by hand. The reasoning is that emotionally loaded text usually voices an opinion rather than asking for information.

They test five scikit-learn classifiers on four datasets: generic tweets plus banking, telecom and retail data. Every classifier improves on every dataset when the emotional features are added. Naive Bayes performs best overall.

Key points

• The datasets were annotated manually and split 80/20 for training and testing:
  • Generic: 5000 tweets, taken from Hasanain et al.
  • Banking: 2500 records from Twitter and Facebook.
  • Telecom: 1500 records from Twitter and Facebook.
  • Retail: 1000 Facebook records.
• Preprocessing includes:
  • cleaning and stop-word removal;
  • normalization of Arabic letter variants (alef forms, hamza carriers, ya, ta marbuta) and removal of diacritics;
  • collapsing any run of three or more identical characters into a single character.
• There are ten emotional features:
  • counts of positive and negative lexicon matches;
  • flags for whether the text starts or ends with a positive or negative term;
  • the normalized share of positive and of negative terms;
  • separate counts of positive and negative emojis and emoticons.
• Naive Bayes results, before → after adding the emotional features:
  • Generic: F 0.89 → 0.93.
  • Banking: F 0.83 → 0.87.
  • Telecom: F 0.71 → 0.75.
  • Retail: F 0.80 → 0.83.
• Gains for the other classifiers are smaller. For example, on the generic set Logistic Regression goes from F 0.71 to 0.72 and GaussianNB from 0.67 to 0.69. For comparison, the earlier Arabic work by Hasanain et al. reported F 0.72.

Interesting ideas

• I1. Treat sentiment as counter-evidence for a real information request: a sentence with a question mark that is full of evaluative words is more likely venting or rhetoric than a genuine query. — For Mova (frames): the speech-act layer in prag could use polarity density as one explainable signal for labelling an interrogative as rhetorical rather than information-seeking.
• I2. Separate "has the form of a question" from "wants an answer", and treat quoted questions, ads and already-answered questions as non-seeking. — For Mova (answer): readers working on tales and fables could first check whether a question in the text is addressed to the reader at all, and skip ones a character asks and answers inside the story.
• I3. Record where sentiment appears (opening or closing position), not only how much there is, since edge position carries pragmatic weight. — For Mova (frames): irony and implicature detection could note whether evaluative words sit at clause or utterance boundaries, as a traceable positional cue.
• I4. Measure a feature group's value by an ablation repeated across several different learners, so the gain is not tied to one model. — For Mova (learn): when adding a new feature or rule, report before/after deltas under several parsers or classifiers as a robustness check, alongside negative controls.
• I5. Normalize expressive letter stretching by collapsing runs of three or more identical characters before lookup. — For Mova (tokens): the en tokenizer could map stretched forms to a lemma candidate, but should check the result against a lexicon, because collapsing to one letter can wrongly turn English words with legitimate double letters (e.g. "goooood" → "god").