📄 OBSIDIAN: An OSINT-Driven NLP Framework for Detecting Cyber-Physical Threats in Arabic Social Media

Summary

The paper targets a gap in Arabic social-media NLP. Prior datasets mostly cover hate speech, offensive language, cyberbullying and harassment, and they treat harm as linguistic toxicity. Security-relevant signals that analysts need for early warning, such as violent intent, explicit threats or people asking for help, are largely missing.

The authors gathered more than 200,000 Arabic tweets from 2024–2025, mostly from the Gulf. They used the Twitter API v2, Apify scraping and public Hugging Face datasets, seeded with keyword lists for security, complaints and help requests. From this pool they manually curated and labelled OBSIDIAN-AR, about 15,000 posts in five classes: Violence, Threat, Distress, Complaint and Neutral. Ambiguous and multi-label items were removed during annotation.

They fine-tuned AraBERT into a classifier called OBSIDIAN and compared it with a TF-IDF Logistic Regression baseline. OBSIDIAN beat the baseline on every metric and reached up to 98% accuracy on unseen data. The resource matters as a first OSINT/CTI-oriented Arabic benchmark for threat triage.

Key points

• OBSIDIAN-AR has more than 15,000 manually annotated Arabic posts in five OSINT categories: Violence, Threat, Distress, Complaint and Neutral.
• The source pool exceeded 200,000 tweets from 2024–2025. It was collected via Twitter API v2, Apify and Hugging Face datasets, using keyword groups for security, complaints and help requests.
• The OBSIDIAN model is fine-tuned AraBERT. It reports up to 98% accuracy on unseen data and outperforms TF-IDF + Logistic Regression across all metrics.
• Threat is the easiest class, with the highest scores, which the authors attribute to explicit vocabulary. Distress is the weakest at F1 96.752%, and some Distress posts are confused with Violence or Complaint.
• A comparison table places OBSIDIAN-AR against earlier Arabic harm datasets, including L-HSAB, Let-Mi, ArCybC and the OSACT5 shared task data. It is the only security-focused one among them.

Interesting ideas

• I1. The label set follows what a reader must act on (threatening, reporting harm, asking for help, complaining) rather than how offensive the wording is, which yields categories that map onto concrete responses.
• I2. Ambiguous and multi-label items were filtered out to keep labels consistent, which the authors credit for balanced per-class performance; the same trick also makes the benchmark easier than real traffic.
• I3. A sparse lexical baseline (TF-IDF + Logistic Regression) was run next to the contextual model to show how much of the task requires more than word presence.
• I4. Per-class error analysis tied the confusions to semantic closeness: Distress overlaps with Violence and Complaint, while Threat is helped by explicit wording.
• I5. Collection was seeded by keyword groups per target category and then cleaned by manual annotation, which cheaply concentrates rare phenomena in a large raw pool.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): prag could add speech-act types such as threaten, plead-for-help and complain, derived from the frame and the speaker's relation to the harm rather than from cue words.
• I2 (learn): silver data could drop sentences where witness parsers disagree, but those dropped cases should be kept as a separate hard test set so scores are not inflated.
• I3 (learn): every frame or answer benchmark could include a bag-of-words baseline as a negative control, making the added value of the tree → frames route measurable.
• I4 (answer): confusion matrices over frame or answer labels would show where pragmatic categories blur, guiding which distinctions need world-state evidence such as who is harmed and who is speaking.
• I5 (world): keyword-seeded harvesting could gather texts rich in specific event types (complaints, requests, losses) to stress-test entity, state and event-order tracking.