📄 muNERa at WojoodNER 2024: Multi-tasking NER Approach

Summary

The paper tackles fine-grained Arabic named entity recognition in the WojoodNER 2024 shared task, covering both the flat and nested subtasks. The Wojood Fine corpus labels entities on three levels: parent tags, sub-tags and sub-sub-tags. Spans can also sit inside other spans, so one token may carry many labels at once.

Instead of the usual encoder-only token tagging, the authors treat NER as sequence-to-sequence generation with the TANL framework on top of AraT5 v2-base. The model rewrites the input sentence with bracketed entity spans, and each span carries a natural-language type name. Sub-tags are encoded as extra spans that cover exactly the same text as their parent. TANL's decoder then cleans the output and aligns it back to the source tokens.

Training one model on all tag levels beat two pipelines that added a fastText classifier for the finer levels. muNERa placed first in nested NER (F-1 91.07) and second in flat NER (F-1 90.26). This shows that generative structure output can handle deep, overlapping label schemes.

Key points

• Wojood Fine has about 550K tokens of MSA and dialectal Arabic, 47.6K fine-grained nested annotations, 31 sub-types following the ACE guidelines, and 53 tags across the train and dev splits.
• On validation, TANLs (all levels) scored the best F-1: 91.16 nested and 90.55 flat.
• TANLp (parent tags only) had high precision but recall below 60%. Adding fastText for the lower levels raised nested F-1 to 90.37 but reached only 86.16 on flat.
• Data problems include:
  • some tokens carrying up to five parent tags and eight sub-tags;
  • labels that appear in dev but never in train (for example "ENT");
  • very rare labels such as "UNIT" (6 examples);
  • near-duplicate labels that differ only by letter case ("Path" vs "PATH").
• The authors list three limitations: separate models for each subtask, no explicit modelling of the tag hierarchy, and English (not Arabic) label words.

Interesting ideas

• I1. Tag codes like GPE or SPO are swapped for readable phrases ("geographical entity", "sports organizations"), so the pretrained model can draw on what it already knows about those words.
• I2. A span with several labels on different levels is written as the same span nested inside itself once per label, which turns a multi-level taxonomy into ordinary nesting.
• I3. Generated output is repaired by dropping malformed brackets and then aligning output tokens to the input with Needleman-Wunsch, so small misspellings by the generator still land on the right source tokens.
• I4. One joint model covering all levels beat a coarse model followed by a second-stage classifier, and the precision/recall breakdown showed that the extra stage was over-predicting types.
• I5. The authors audit the label inventory before training: labels missing from train, labels with only one example, and labels that differ only in case.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): When prompting LLM teacher tiers for silver role or entity labels, give VerbNet roles and ontology classes as plain descriptive phrases rather than bare codes, which should make the teacher's answers more consistent.
• I2 (world): Entities that hold both a coarse and a fine type (organization → municipality) could be serialized this way when the world graph is linearized for checking or training.
• I3 (learn): LLM teacher outputs could be aligned this way to Mova's own tokenization before they enter silver data, instead of being discarded whenever the text does not match exactly.
• I4 (frames): Before splitting frame detection and fine role assignment into a pipeline, compare it against a joint model and report precision and recall separately to see which stage is introducing errors.
• I5 (tree): Run the same audit on dependency relation and feature inventories when converting treebanks between UD dialects, so that unseen or near-duplicate labels are caught early.