📄 Baflah-lamri at NAKBA-NLP 2026: Manual Ground Truth Enrichment

Summary

The paper describes how a two-person team from CRSTDLA (Algeria) produced reference transcriptions for the NAKBA NLP 2026 shared task on Arabic manuscript understanding, contributing specifically to its Transcription Track, designated Subtask 1. The source material is the Omar Al-Saleh memoirs (1951–1965), handwritten in Mashriqi Naskh. The team transcribed its full batch of 500 cropped line images, drawn from 368 pages. Two native-speaker annotators worked independently and settled disagreements by consensus. They followed diplomatic transcription: the text is copied as written, with no spelling or grammar fixes. Unclear lines were checked against the full-page scan. The resulting Ground Truth has 6,719 words and 37,646 characters, and about nine in ten lines were read completely. Beyond the numbers, the paper sorts the difficulties it met into three groups: hurried handwriting, physical traces of the pen, and errors from automatic line cropping. From these it draws suggestions for Arabic HTR, including context-aware language models, segmentation built for Arabic script, and lexicons that tolerate spelling variants. The data is to be released under CC-BY-4.0.

Key points

• Batch: 500 lines from 368 unique pages of the Omar Al-Saleh memoir collection. The full collection has 16 documents, about 6,395 pages and roughly 1.6M words.
• Readability:
  • 451 lines (90.2%) are fully readable.
  • 49 lines (9.8%) contain unreadable words, marked with dots.
  • Elsewhere the paper says about 20 lines (~4%), an internal inconsistency.
• Line profile:
  • On average 75.3 ± 16.7 characters and 13.4 ± 3.2 words per line.
  • 113 lines (22.6%) contain numbers.
  • 5 lines (1.0%) contain Latin characters.
• Orthographic rules:
  • Hamza forms, Alif Maqsura vs. Yaa, and visible diacritics are all copied exactly as drawn.
  • Ta' Marbuta vs. Ha' is decided by context only when the dots are missing.
  • Eastern and Western digits are kept as written.
  • Foreign names keep the author's Arabic-letter spelling.
• Handling rules:
  • Crossed-out words are dropped and only the final wording is kept.
  • Pre-printed notebook text is ignored.
  • Letters cut off at line edges were traced to the automatic cropping step, not to the manuscript itself.

Interesting ideas

• I1. Each defect is traced to its origin: either the document itself or the processing pipeline (cropping), confirmed by checking the full page. This stops tooling errors from being treated as properties of the data.
• I2. Diplomatic transcription keeps inconsistent spellings exactly as found instead of normalizing them, so the variation stays available for later study.
• I3. Unreadable words become a dot placeholder whose length roughly matches the missing span, instead of a guess. The gap is marked and localized but never invented.
• I4. Ambiguous strike-throughs are reduced to the author's final text rather than encoded with elaborate tags, on the grounds that complex markup can confuse model training.
• I5. The authors propose lexicons that accept several valid spellings of one word, such as transliterated foreign names, without flagging them as errors.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): When a parse goes wrong, first check whether sentence splitting or tokenization broke the input before blaming the parser or the treebank, and keep these two error classes separate in evaluation.
• I2 (tokens): Keep the original surface form next to the normalized token and lemma, so normalization is a separate, explainable layer rather than a silent rewrite.
• I3 (world): Unresolved entities or states could be stored as explicit "unknown" slots with a scope, so the answer stage can say "not stated" instead of hallucinating a filler.
• I4 (learn): Silver training data should keep annotation conventions minimal and consistent, and ambiguous corrections should be resolved before data reaches the trainer.
• I5 (tokens): The lemmatizer and lexicon could map known spelling variants (British/American spellings, transliterated names) to one entry and record the variant, so they are not treated as unknown words.