📄 QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning

Summary

The paper describes QU-NLP's entry to the QIAS 2026 shared task. The task asks a model to solve Arabic Islamic inheritance (ilm al-mawarith) cases and return a full JSON reasoning trace, not just pick a multiple-choice answer. A solution has to name the eligible heirs, decide which heirs are blocked, assign fixed shares, detect whether an ʿawl or radd adjustment applies, and give the final distribution. The author fine-tunes Qwen3-4B with QLoRA in two stages. The first stage is causal language modelling on Islamweb fatwa texts, to absorb legal vocabulary. The second stage is supervised training on the 12,000 MAWARITH training cases, with JSON-only output. Light rule-based post-processing follows. On the 500-case test set, the multi-component MIR-E metric reaches 90.0%. This essentially ties Gemini-2.5-flash (90.1%) and far exceeds much larger open-weight baselines. A detailed error analysis finds that most remaining mistakes come from generating the final arithmetic or choosing between notations, not from misunderstanding the law. This matters because it suggests a small, cheaply adapted model can handle rule-heavy legal reasoning. It also points to fixes, such as constrained decoding, that need no retraining.

Key points

• Training setup:
  • Model and adapters: Qwen3-4B, 4-bit NF4 QLoRA, rank 128, α = 256, adapters on all projection layers.
  • Stage 1: 3,166 fatwa records, lr 2e-4, 2 epochs.
  • Stage 2: 12,000 cases, lr 3e-5, 6 epochs, output keys heirs / blocked / shares / awl_or_radd / post_tasil.
• MAWARITH data: 12,500 cases covering 36 heir categories. 92.3% are simple, 4.9% ʿawl and 2.8% radd.
• MIR-E weighting: heirs, shares and final allocation each count 0.30, and adjustment counts 0.10. The adjustment score is credited only when heirs and shares are both fully correct.
• Main scores:
  • Component scores: S_h 97.1%, S_s 94.3%, S_a 84.6%.
  • Best open baseline: Qwen3-32B at 43.7%.
  • Fully correct cases: only 65.5%. The largest drop in the pipeline (−13.9pp) occurs at the final-allocation step.
• Error profile:
  • Calculation errors: 29 cases (−1.7pp).
  • Residue label: the model uses it in only 24.7% of the 417 cases that require it. It usually writes an explicit fraction instead, and 83.1% of those fractions are numerically correct (cost −0.85pp).
  • Post-processing adds only +0.2pp.

Interesting ideas

• I1. Conservative repair: The post-processor replaces only labels it cannot recognise and never overwrites a valid label using sums computed from possibly wrong fractions, since doing so risks corrupting an output that was already correct.
• I2. Cumulative stage success: The authors count the share of cases that are perfect at every stage up to step k, which shows exactly where errors first enter a sequential pipeline.
• I3. Separating notation from computation: For each mismatch, they check whether the model's substituted value equals what a symbolic calculator would derive, which distinguishes representational failures from computational ones.
• I4. Difficulty versus rarity: ʿawl cases score lower than the rarer radd cases, so the authors attribute the gap to multi-step arithmetic rather than data scarcity. Slicing results by both frequency and operational complexity avoids blaming the wrong cause.
• I5. Domain-text adaptation first: Before teaching the structured output format, the model is first adapted on raw domain text so it learns the vocabulary separately from the format.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Self-induced repair rules could be restricted to firing on malformed or unknown outputs, never on well-formed ones whose evidence is itself unverified.
• I2 (answer): Reporting the fraction of test items correct through tree, then frames, then world, then answer would reveal which stage loses the most items end to end.
• I3 (world): When a derived quantity or holding disagrees with gold, comparing it against a recomputed symbolic value would tell whether the reasoning or only the surface form went wrong.
• I4 (answer): Maths word-problem evaluation could be split by the number and type of operations, not only by problem frequency, before deciding to add more training data.
• I5 (tree): gparse could be briefly pre-adapted on unlabeled in-domain text, such as fables or school texts, before treebank fine-tuning, to help with domain vocabulary.