📄 Tracing Traditions: Automatic Extraction of Isnads from Classical Arabic Texts

Summary

The paper addresses automatically finding isnads, the chains of transmitters that classical Arabic authors attach to reports as a kind of citation. Isnads are hard to spot because the texts are long, mostly unpunctuated, and the chains blend into the surrounding prose. The authors annotated a training set drawn from the OpenITI corpus. They built it iteratively: a CRF pre-tagged new texts, and experts corrected the output. Two approaches were compared. The first labels every token of a whole document in a single pass. The second, a "retrieve-and-tag" pipeline, first retrieves short chunks likely to contain an isnad start and then runs a sequence tagger only on those chunks. Whole-document neural tagging was limited by memory and by weak modeling of very long inputs. The retrieval-plus-LSTM pipeline beat both a naïve baseline and a token-feature CRF at locating isnad start points, though its absolute scores remain modest. The work is a first step toward mapping, at scale, the social network of scholars who shaped the Arabic written tradition.

Key points

• Data source is the OpenITI corpus: 4,285 classical Arabic texts, about 1.5 billion words, with the longest single text over 112 million words. Existing punctuation was stripped and orthography was normalized.

• The labeled set covers 50 texts and 163 tagged regions. It contains 907,110 tokens with 3,071 isnads, which average 31.9 tokens each. Sources include hadith, history, tafsir, geography and literature.

• Five expert annotators labeled the same text (2,000 lines) for an inter-annotator agreement study, reaching Krippendorff's α = 0.84. All annotators agreed on span starts 70% of the time but on span ends only 53% of the time. Large outliers came mostly from merging adjacent isnads, and about 3% of isnads start right after another one ends.

• Results for isnad start detection (P/R/F1, micro-averaged over ten document-level splits):

  Model	Precision	Recall	F1
  Naïve baseline	.188	.378	.219
  CRF (Tokens)	.475	.422	.432
  Retrieval-LSTM	.571	.442	.476

  Retrieval uses 75-token chunks weighted by TFIDF.

• Planned future work includes a span model for isnad endpoints, transmissive-term gazetteers and NER features, and linking names into a transmitter network with help from biographical dictionaries.

Interesting ideas

• I1. The annotation loop has a model pre-tag text, and experts correct it; the experts then pick the next texts to annotate specifically where the model fails, such as student lists in biographies that the model mistook for chains. This produces targeted negative examples rather than random extra data.
• I2. When a target span is rare in a huge document, the work can be split into two stages: a cheap first pass narrows the text to candidate windows, and a precise tagger then runs only inside them. This avoids memory blow-ups and the dilution that comes from tagging everything.
• I3. Agreement was measured separately for span starts and span ends, which showed that end boundaries and adjacent-span splits are inherently fuzzy even for experts. This calibrates how strictly a model should be scored on each boundary.
• I4. An isnad is essentially a nested chain of reports, where one person relayed what another said. The authors chose to keep this surrounding structure rather than just tagging names, because later network inference depends on the links between people.
• I5. Train/test splits were made at the document level, so chunks of one tagged section never appear on both sides. This prevents leakage when the data is sliced into small windows.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver-data rounds could choose the next batch by clustering witness-parser disagreements and model errors, adding deliberate near-miss negatives instead of sampling sentences uniformly.
• I2 (answer): QA over long tales could first pick candidate passages using structural cues already in the world model (entities mentioned, event order), then run full frame-level reasoning only there, with the selection step logged so it stays explainable.
• I3 (learn): Evaluation of span-like outputs (argument spans, quoted-speech extents) could report left and right boundary accuracy separately and use human agreement as a ceiling, so boundary noise isn't misread as model failure.
• I4 (frames): Reported-speech handling could model attribution chains explicitly (A said that B said that C did X), with each layer a speech-act frame, so the world stage knows who vouches for which claim.
• I5 (learn): Any chunked or sentence-level training data drawn from the same story or treebank document should be split by source document so evaluation scores aren't inflated.