📄 Unsupervised Learning-based Anomalous Arabic Text Detection

Summary

The paper asks whether a text that does not fit a document collection can be found automatically when the language is Arabic, with no labelled examples. It extends earlier English work (Guthrie et al., 2007). Each paragraph-sized segment becomes a vector of stylistic features. Some are generic, such as sentence length, vocabulary richness and shares of pronouns or conjunctions. Others are Arabic-specific, such as diacritics, nunation, elongation, numeral style and calendar type. Features are z-score normalised. Each segment's anomaly score is its summed dissimilarity to every other segment, computed with cosine dissimilarity, city block distance or Chebychev distance.

For testing, a single religious, social or novel segment was mixed into 50 Aljazeera.net news segments. This was repeated 30 times per type, and success was measured by whether the intruder ranked in the Top 1, 3 or 10. All three measures placed the intruder in the Top 10 more than 75% of the time, and religious text reached about 90%, helped by its diacritics. City block and Chebychev performed alike, and both beat cosine. This matches the English findings and shows that shallow, language-aware style features can flag out-of-place text cheaply.

Key points

• Normal data: 50 Aljazeera.net news segments. Anomalies: one inserted segment that is religious, social or from a novel, with 30 runs per anomaly type.
• Method: fully unsupervised, assuming no model of what "normal" language looks like.
  • Each segment's anomaly score is the sum of its distances to all other segments.
  • Three distance measures were used: cosine dissimilarity (1 − cos), city block, and Chebychev.
  • Features are converted to z-scores first.
• Features:
  • General stylistic ones include average word and sentence length, question sentences, frequent words, vocabulary richness, punctuation, prepositions, conjunctions, several pronoun types, and positive/negative words.
  • Arabic-specific ones include diacritics, diphthongs, nunation, elongation, Arabic vs Indian numerals, and lunar/Hijri calendar.
• Results:
  • All models detected the intruder in the Top 10 more than 75% of the time, compared against a random baseline.
  • Religious intruders reached about 90% in the Top 10.
  • City block ≈ Chebychev > cosine, consistent with the earlier English study.
• Ambiguity rule: an attached "wa" (and) or an abridged one-letter "we" pronoun is not counted when it cannot be told apart from an ordinary letter.

Interesting ideas

• I1. Scoring each item by its summed distance to all peers finds outliers with no training data and no explicit model of the norm.
• I2. Injecting one known foreign item into a homogeneous pool and checking whether it rises to the Top-n gives a cheap, controlled test of a detector.
• I3. When a feature cannot be decided, skip counting it rather than guess. This trades recall for clean, precise statistics.
• I4. Normalising each feature to z-scores before measuring distance keeps any single high-variance feature from dominating.
• I5. Script-specific surface cues (here, diacritic density in religious text) can carry strong genre signal next to generic style features.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver parses or frame sets could be ranked by how far their feature profile sits from the rest of the batch, flagging suspicious outputs for teacher review.
• I2 (learn): This works as a negative-control harness. Plant deliberately corrupted trees or texts among good ones and verify that quality checks surface them above a random baseline.
• I3 (tokens): The tokenizer and morphology can mark undecidable splits as "unknown" and keep them out of feature counts, so explanations never rest on guessed segmentations.
• I4 (world): When comparing entity or text profiles built from mixed measures, per-feature z-scoring makes the distances fair and explainable feature by feature.
• I5 (frames): Register cues such as archaic forms, quotation density or punctuation habits could help pragmatics detect genre shifts, irony or quoted speech inside a text.