📄 Arabic Mention Detection: Toward Better Unit of Analysis

Summary

Arabic words often fuse conjunctions, articles, stems and pronoun suffixes, so the choice of token unit shapes how well a tagger generalises. The authors ask which unit best suits Arabic Mention Detection: finding named, nominal and pronominal references and labelling them with entity types. They segment the Arabic ACE 2007 data in four ways: punctuation splitting only, Arabic TreeBank (ATB) style, full morphological splitting, and characters. A MaxEnt sequence tagger is then trained on each version with feature sets that grow step by step. With only lexical, stem or syntactic cues, morph-level units win by roughly 4 F points over ATB tokens, because they shrink sparsity and the number of unseen tokens. Once outputs of taggers trained on morph-segmented data are added as features, ATB tokens perform best, since they give wider context while the extra features cover the sparsity. Characters perform very poorly. Combining both schemes through stacked predictions adds small further gains. The study shows that the best unit of analysis depends on which other resources are available.

Key points

• Data: Arabic ACE 2007 training data, split 85/15. Training uses 323 documents (~80k words, 17,634 mentions) and testing uses 56 documents (~18k words, 3,566 mentions), with 7 mention types (facility, GPE, location, organization, person, vehicle, weapon).
• The ATB and morphological segmenters are WFST-based and reach 99.4 and 98.1 F on ATB data. The tagger is MaxEnt with lexical, stem, syntactic (POS and chunk) and external-classifier features.
• With lexical features only, F was 74.1 on Morph_s, 70.1 on ATB_s, 66.4 on Word_s and 22.3 on Char_s. With the full feature set (Sem_f), F was 79.0 on ATB_s, 78.3 on Morph_s and 77.1 on Word_s.
• The test set had 1,518 OOV tokens under morph segmentation and 2,464 under ATB. ATB models handled long mentions better, and widening the Morph_s context window up to ±5 brought no gain.
• Stacking used predictions from the other scheme, produced by 15-way round-robin. It gave ATB_s plus M2A_f 79.1 with Sem_f and Morph_s plus A2M_f 75.2 with Stem_f. Significance was tested with stratified bootstrap (p < 0.02).

Interesting ideas

• I1. Predictions from fine-grained units can be projected onto coarser tokens by giving each token the label of its stem. This lets one tagger's output feed a model built on a different tokenisation.
• I2. Train a coarse-unit model on the predictions of a fine-unit model, and vice versa. Generate those predictions by round-robin so that no item is labelled by a model that saw it in training.
• I3. The authors tested a rival explanation directly. If morph models only lacked context, a wider window should have closed the gap with ATB, and it did not.
• I4. Feature sets were added in cumulative tiers, which showed that the best unit flips depending on what other resources exist. One fixed choice of granularity is therefore not universally right.
• I5. Auxiliary taggers were deliberately not retrained per segmentation, so their features would have comparable quality in every condition.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Signals from sub-word analysis, such as stem or suffix-level tags, could be carried up to full English tokens as features when the tokenizer's split differs from the analyser's.
• I2 (learn): When one witness parser's output becomes a feature for another, jackknifed predictions would keep silver-data training free of leakage.
• I3 (learn): Evaluations could routinely include such control runs that rule out the cheap explanation before a design choice is credited.
• I4 (tokens): Comparing surface-form against lemma-based features across resource levels could show where lemmatisation actually pays off in the tagger.
• I5 (learn): Keeping helper models frozen across ablations would make differences between parser configurations attributable to the configuration rather than to helper quality.