📄 BERT-based Classical Arabic Poetry Authorship Attribution

Summary

Classical Arabic poetry has long had attribution problems. Much pre-Islamic verse was transmitted orally and later lost, and some poems composed after the rise of Islam were credited to earlier poets. Earlier computational work on Arabic poetry relied on classic classifiers such as SVM, RF and NB. This paper is among the first to use transformers on this problem, and it works with the whole Classical Arabic Poetry corpus.

The authors scraped the corpus from online sources. Because the traditional division into eras is unreliable, they hand-annotated each poet with a Date of Death (DoD). They also used Embedded Topic Modeling (ETM) to label every poem with its topic proportions. An ensemble of CAMeLBERT classifiers, each trained on a different data subset and combined by majority voting, was tested under different settings for the topic, how many poets were included and how many examples were available for training.

After tuning, F1 ranged from 0.97 to 1.0, and the ensemble far outperformed a single model. When applied to four disputed pre-Islamic poems, the model's confidence agreed with the doubts raised in historical and literary scholarship.

Key points

• The dataset covers the entire Classical Arabic Poetry corpus, with manually curated DoD metadata replacing traditional era sectioning.
• ETM worked best with 5 topics and 50 epochs, which gave the best trade-off between coherence and diversity. Example topics include Praise and Power.
• The CAMeLBERT ensemble with majority voting averaged F1 0.98, against 0.75 for a single model. Tuned settings reached F1 0.97–1.0.
• Topic-controlled (balanced) training gave slightly higher accuracy and removed low-accuracy outliers. Failure ratios were also computed per topic.
• Case studies:
  • A suspect Imru' al-Qays poem scored 0.71, compared with about 0.99 for his typical poems.
  • Two doubtful Al-Aʿshā poems scored 0.86 and 0.85.
  • A debated ʻUbayd Ibn Al-Abraṣ poem scored 0.99, supporting its attribution.

Interesting ideas

• I1. A low confidence score is read as evidence only relative to the class's usual confidence (0.71 against a normal 0.99), not against a global threshold.
• I2. Training several copies of one model on different data subsets and taking a majority vote gave much more stable results than a single model.
• I3. Balancing topics during training stops the classifier from learning theme instead of style, and per-topic failure ratios reveal where it breaks down.
• I4. Precise per-entity metadata (death dates) replaced coarse traditional period labels, which allowed finer modelling of change over time.
• I5. The authors admit the model cannot detect multi-author or fabricated texts, and they propose an explicit class for fake items.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Track each construction's or genre's typical parse confidence in gparse, and flag sentences that fall well below that baseline as candidates for silver-data rejection or repair rules.
• I2 (tree): Train several gparse or perceptron instances on different treebank slices and let them vote as extra witness parsers when building agreed silver trees.
• I3 (learn): Slice tagger and parser errors by domain or topic as a built-in control, so that apparent gains are not just topic memorisation.
• I4 (world): Anchor entities in the knowledge graph to exact Wikidata dates rather than era buckets, so event ordering and timeline questions rest on precise values.
• I5 (answer): Give readers an explicit "not supported / mixed source" outcome, so that when evidence is unclear they report it instead of forcing a best-match answer.