📄 Arabic Topic Classification Corpus of the Nakba Short Stories

Summary

Arabic text classification lacks open, labeled corpora, and resources on Palestinian narratives are especially scarce. The authors build the Nakba Topic Classification Corpus (NTCC) from eight short stories taken from the Nakba Archive website. They convert the PDFs to a tabular format, remove diacritics and noise with PyArabic and regular expressions, normalize the text (extra whitespace, Tatweel), and split paragraphs into sentences with NLTK. Fragments with no useful content are then filtered out by hand. Each sentence receives one of five thematic labels.

About a third of the sentences were labeled manually, with high inter-annotator agreement. A keyword- and pattern-based rule system labeled the remainder. When the authors compared automatic annotators, this rule system was the most accurate. TF-IDF and AraBERT without fine-tuning scored far lower, which shows that generic models transfer poorly to this niche domain. The corpus serves Arabic NLP and also helps preserve Palestinian cultural memory.

Key points

• The source is eight Nakba short stories from the Nakba Archive (nakba-archive.org). Sentence splitting gave 605 sentences, and manual removal of meaningless fragments left 473.
• There are five categories: Historical Events and Politics, Emotions and Spirituality, Nature and Daily Life, Homesickness and War/Conflict, and Others.
• About one third of the data was hand-annotated, with an IAA of 87% that was resolved to 100% after discussion. A rule-based system labeled the rest.
• The tools used were PyArabic for diacritics, whitespace and Tatweel, regex for URLs, English text and special characters, and NLTK for sentence segmentation. Pandas/CSV stored text together with metadata.
• The rule-based classifier achieved the best accuracy. The TF-IDF and un-fine-tuned AraBERT baselines reached only 25% and 34%.

Interesting ideas

• I1. Hand-label a reliable seed portion with an agreement check, then extend it with transparent rules and judge those rules against the seed; this yields scalable annotation whose errors can be traced.
• I2. The automatic annotators were tested on a dataset small enough to label by hand, so that their trustworthiness could be measured before they were applied to millions of sentences.
• I3. Themes that naturally co-occur, such as emotion with spirituality and homesickness with conflict, were merged into paired labels to reduce annotator ambiguity, and the authors openly name multi-label annotation as the eventual fix.
• I4. An explicit "Other" class absorbs sentences that fit no theme, which avoids forcing a wrong label.
• I5. In a narrow domain, keyword plus contextual-pattern rules beat a generic pretrained model that was not adapted to it.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver data could be produced the same way, with a small adjudicated gold slice that scores each witness parser or rule set before it is trusted on the larger pool.
• I2 (learn): Before any new teacher tier or repair rule is run on bulk text, it could first be checked on a small fully hand-verified sample.
• I3 (frames): When frame or speech-act labels routinely overlap, a provisional merged label could be used and logged as a known simplification, with a later split into multiple labels.
• I4 (answer): Readers could be given a principled "none of these" outcome for questions or texts outside known categories, so they do not guess.
• I5 (world): Small, explainable thematic lexicons linked to the ontology could tag a story's topic or setting, giving a cheap and transparent baseline for domain texts.