📄 Athership at AraGenEval Shared Task: Identifying Arabic Authorship with a Dual-Model Logit Fusion

Summary

This work addresses closed-set authorship attribution for Arabic: given one paragraph, the system must name its writer from 21 candidates. It was entered in Subtask 2 of AraGenEval 2025, a shared task held as part of ArabicNLP. The task is difficult in Arabic because of root-based morphology, diglossia and noisy spelling, such as hamza variants and optional diacritics. The organizers' data, cut from literary, philosophical and journalistic books, adds further problems: class sizes are imbalanced and paragraph lengths vary widely.

The team fine-tuned two Arabic encoders that were pretrained with different objectives:

• AraBERT, trained with MLM.
• AraELECTRA, trained with RTD.

Each model got a linear head on [CLS], was trained for 4 epochs, and saw inputs truncated to 512 tokens. Preprocessing was kept minimal so that stylistic cues would survive. The two models' raw logits are combined before a single softmax. This fused system beat both individual models on the dev set and placed third on the test leaderboard. Comparisons against a sliding-window model, a TF-IDF model and a contrastive model suggest that pretrained encoders and unbroken context matter for capturing style.

Key points

• Test result: third place in Subtask 2, with Macro-F1 0.85968 and accuracy 0.89516.
• Dev scores (accuracy / Macro-F1):
  • AraBERT: 0.90 / 0.84
  • AraELECTRA: 0.88 / 0.83
  • Logit ensemble: 0.92 / 0.86
• Other dev results:
  • TF-IDF over character and word n-grams, fed to a 2-layer FCN: about 0.75 / 0.70.
  • Contrastive learning on Qarib embeddings with k-NN: 0.84 / 0.79.
• Sliding-window variant: aubmindlab/bert-large-AraBERTv02 with 512-token windows, stride 128, balanced class weights, label smoothing 0.1 and chunk voting at inference. It reached 0.90 / 0.85, which is better than either single model but below the ensemble. Separately, naively splitting long texts and copying the document label to every chunk lowered accuracy.
• Fusion details are inconsistent in the paper. The fusion uses equal, untuned weights. The formula writes plain summation with 11-dimensional logits, while the prose calls it averaging over 21 authors. Learned weights and more ensemble members are listed as future work.

Interesting ideas

• I1. Fusing raw pre-softmax scores from models trained with different objectives keeps each model's decision margins, which is richer than voting on final labels.
• I2. Copying a document-level label onto every fragment of a split text produced noisy training examples, because short pieces often lack the evidence the label depends on.
• I3. Light preprocessing was chosen on purpose, because aggressive normalization can erase exactly the surface signal (spelling variants, punctuation) the task relies on.
• I4. A shallow TF-IDF n-gram model served as a floor that showed how much the contextual encoders really add.
• I5. Reporting Macro-F1 next to accuracy exposed how well the system handles minority classes in an imbalanced set.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): sum or interpolate arc and label scores from the perceptron parser and gparse before decoding, instead of reconciling finished trees.
• I2 (learn): when silver sentences or passages are segmented, re-check each fragment's label with the witness parsers rather than inheriting it blindly.
• I3 (tokens): keep the raw surface form next to the normalized form and lemma, so later stages can still use orthographic cues when needed.
• I4 (learn): use a bag-of-words classifier as a standing negative control in evaluations, so gains from the explainable pipeline are measured against what shallow lexical overlap already achieves.
• I5 (answer): score test sets per question type with macro averaging, so rare kinds of questions are not hidden behind frequent easy ones.