📄 Answer Extraction for Why Arabic Questions Answering Systems: EWAQ

Summary

The paper presents EWAQ, a question answering system for Arabic that handles only why questions. Most earlier Arabic QA work skipped this type, and where it was handled (e.g., QArabPro) it performed worst. EWAQ takes passages returned by web search engines (Google, Yahoo, Ask) and does two things. First, it re-orders those passages. Second, it chooses the single sentence most likely to give the reason asked for.

Both steps use one score, a directional cosine-style textual entailment measure taken from Tatar et al. and adapted to Arabic in the author's earlier summarization work. Word overlap between question and passage counts shared roots and Arabic WordNet relations, not only identical surface forms. Preprocessing consists of stop-word removal and stemming.

The test set has 250 manually answered why-questions. On it, EWAQ reached 68.53% accuracy, ahead of the raw search-engine rankings. The gain is modest, but it suggests that lightweight entailment scoring can help answer extraction for causal questions in a language with few resources.

Key points

• Preprocessing:
  • Abu-Elkhair's stop-word list.
  • The hybrid Arabic stemmer of Dilekh & Behloul.
  • Arabic WordNet (AWN), used to expand every question and passage word with its related senses.
• Overlap: the common-word count c includes words that share a root or are linked by an AWN relation. Three similarities are computed from c, question length n and passage length m:
  • cosT = √(c/m)
  • cosH = √(c/n)
  • cosH∪T = √(4c²/((n+c)(m+c)))
• Acceptance conditions: an entailment candidate is accepted only if all of the following hold, and its final score is then cosH∪T.
  • The three scores are ordered cosH ≥ cosH∪T ≥ cosT.
  • The gaps between them stay within the thresholds τ1 = 0.095 and τ2 = 0.2.
  • The largest of the three scores is at least τ3 = 0.5.
• Pipeline: the system takes the top seven passages per engine and re-ranks them by entailment. It keeps the top five, splits them into sentences, and returns the sentence with the highest entailment score as the answer.
• Evaluation:
  • The test set has 250 questions, 50 from each of five domains: computer, religion, science, politics and history.
  • Reference answers were prepared by thirty people.
  • Accuracy was 68.53% for EWAQ, 66.19% for Google, 63.27% for Ask and 61.48% for Yahoo.
  • The paper's wording is inconsistent on whether accuracy means the answer at rank 1 or within the top 3.

Interesting ideas

• I1. Normalizing the same overlap count three ways (by hypothesis length, by text length, and jointly) yields an asymmetric signal that can tell which side covers which, unlike a single symmetric similarity.
• I2. Scores are only trusted after passing explicit gating conditions (expected ordering, bounded gaps, a minimum peak value), so every rejection has a nameable reason.
• I3. A match is counted when words share a root or a lexical-database relation, not just when strings are identical. This makes overlap robust to morphology and paraphrase.
• I4. A coarse-to-fine design first re-ranks whole passages, then searches for the best sentence only inside the top few. This keeps the expensive fine-grained step small.
• I5. The added component is evaluated against the very source it re-ranks (the raw search-engine order), which isolates the component's own contribution.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (answer): the same asymmetry could be computed over matched frame roles instead of words, flagging when a story sentence fully covers a question's frame rather than merely sharing some of it.
• I2 (answer): candidate answers could be accepted only after passing named checks, and the failing check would be reported as the explanation when no answer is given.
• I3 (world): when linking question entities to text entities, WordNet relations could license a match, and the relation type used would be recorded in the explanation trail.
• I4 (answer): QA over long tales could first select the relevant episode via event order in the world model, then run detailed frame matching only within it.
• I5 (learn): every repair rule or re-ranker could be scored against a "do nothing" baseline on identical inputs, serving as a built-in control for its real effect.