📄 Effects of term weighting approach with and without stop words removing on Arabic text classification

Summary

The paper asks how two simple ways of encoding words as features affect supervised classification of Arabic documents, and whether stripping stop words changes the outcome. The two encodings are binary presence/absence and raw term frequency (TF). Arabic is a hard case because of its rich morphology, homonymy, part-of-speech ambiguity, flexible word order and case-dependent word forms. The authors cite light stemming as one way to address the word-form problem. They use a small Arabic corpus covering six topics. Two Naive Bayes variants are applied: a Multivariate Bernoulli model for the presence-based features and a Multinomial model for the count-based ones. A 2×2 design crosses the encoding with stop-word removal on or off, and each run is scored on accuracy, recall, precision and F-measure. TF with stop-word removal gives the best results (about 0.95 accuracy). Without removal, the ranking flips and binary features come out ahead, while precision is roughly tied. For a fixed encoding, removing stop words helps. The point is that preprocessing and representation choices interact, so they should not be tuned separately.

Key points

• Corpus: six categories (agriculture, economy, health, politics, science, sport), with 50 documents each and 61 for health.
  • The stated total of 322 documents does not match these per-class counts, which add up to 311.
• Classifiers: Multivariate Bernoulli Naive Bayes (MBNB) for binary weighting and the Multinomial Naive Bayes model for TF weighting.
• Results table (Accuracy / Recall / Precision / F-measure):
  • Exp.1: 0.953 / 0.953 / 0.960 / 0.954
  • Exp.2: 0.922 / 0.922 / 0.940 / 0.923
  • Exp.3: 0.882 / 0.882 / 0.914 / 0.877
  • Exp.4: 0.913 / 0.937 / 0.913 / 0.913
• Labeling problem: the paper's descriptions of Exp.3 and Exp.4 are inconsistent with each other and with its own claims, so the mapping of runs to conditions should be read with caution.
• Reported margins: with stop-word removal, TF beats binary by about 3% accuracy. Without removal, binary beats TF by about 4%.
• Related work is surveyed, including:
  • OSAC (Saad, 2010), with 22,429 documents in ten categories.
  • Al-Salemi & Aziz (2011), who found MBNB better than the NB and MNB models using the TREC-2002 Light Stemmer.
  • PATC, a MapReduce-based parallel Arabic classifier.

Interesting ideas

• I1. Crossing a preprocessing switch with a representation choice in a full 2×2 grid shows interaction effects, here a ranking that flips depending on stop-word removal, which one-factor-at-a-time tests would miss. — For Mova (learn): Run ablations of the tokenizer or normalization options crossed with parser variants, so that a component is never declared "better" from tests under only one preprocessing setting.
• I2. Each feature encoding is paired with the model whose assumptions fit it: a presence/absence model for binary features and a count model for frequencies. This avoids mismatches between what the data says and what the model expects. — For Mova (world): When compiling knowledge such as verb-class or frame statistics, choose the scoring model to match whether a fact is a yes/no attestation or a graded count.
• I3. Reporting several metrics side by side reveals cases where configurations differ in accuracy and recall but are practically tied on precision, which a single headline number would hide. — For Mova (answer): Report precision and recall separately for school-test and fable-QA readers, so that a change which only trades abstentions for errors is visible.
• I4. The paper's internal inconsistencies (corpus total, experiment labels) illustrate why experiment configurations should be recorded automatically rather than described by hand afterwards. — For Mova (learn): Store each evaluation run with a machine-generated manifest of its settings and per-class counts, and check that the totals add up before results are written up.
• I5. Whether a token class like stop words counts as noise depends on the representation consuming it: removal helped count-based features more than presence-based ones. — For Mova (tokens): Function words are essential for UD parsing, but any filtering for downstream lexical statistics should be made per consumer and measured, not applied globally in the tagger.