📄 Semantic Query Expansion for Arabic Information Retrieval

Summary

The paper addresses a weakness of plain keyword search for Arabic. It misses documents that express the query's meaning with different words, inflected forms, or more specific concepts, and ambiguous words make this worse. The authors build a query expansion layer on top of LUCENE (.Net). It draws on three resources:

• Arabic Wikipedia: redirect titles, the bolded gloss in the first paragraph, and subcategories.
• Al Raed: a monolingual dictionary of modern Arabic.
• Google_WordNet: a bilingual synonym dictionary the authors built by machine-translating WordNet entries and merging overlapping senses.

Morphological variants are added through a Light-10 stem dictionary. This dictionary only links words that co-occur in the same document.

They compare two strategies. One merges all expansions into a single query. The other expands each term separately, runs several queries, and merges the result lists with a frequency-based weighting.

On a small test set built from the book "Zad Al Ma'ad", both strategies beat the unexpanded baseline on F-score and NDCG. The per-term multi-query variant was usually best. This suggests that encyclopedic resources plus careful noise control can make semantic search practical for a lower-resourced language.

Key points

• From Arabic Wikipedia the authors extracted 397,552 semantic sets covering 690,236 collocations. The Al Raed dictionary contributes 204,303 modern expressions.
• Google_WordNet merges two translated sense groups when their English synonym lists share at least two words. The Arabic translations of the two groups are then pooled as well.
• Preprocessing uses:
  • diacritic removal,
  • a 75-word stop list,
  • three letter-normalization rules,
  • the Light-10 stemmer, applied only to words that appear together in some document rather than stemming the whole index.
• Each query receives at most 50 added terms in total. Rarer terms get a larger share of that budget, set by a relevancy factor computed from corpus frequency.
• The test set has 25 queries (chapter titles) and 2,730 documents (sections). Results for the three runs were:

Interesting ideas

• I1. Two machine-translated sense groups are treated as one sense only if they share at least two synonyms, which is a cheap, explainable rule for collapsing redundant senses.
• I2. Words with the same stem are linked only if some document contains both, which blocks false merges of homographs that differ in meaning.
• I3. Wikipedia redirect titles and the bolded alternative names in an article's opening paragraph give ready-made sets of names for one entity.
• I4. Expanding one term at a time keeps each query's noise tied to a single source, so its effect on results can be judged separately.
• I5. The expansion budget is spread in inverse proportion to term frequency, so rare terms get more help and common ones stay nearly untouched.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (world): Apply the same overlap test when aligning WordNet synsets with Wikidata or FrameNet entries, so that near-duplicate concepts merge only with visible evidence.
• I2 (tokens): When the lemmatizer proposes a shared lemma for unusual forms, require co-occurrence evidence in a corpus before accepting the merge, and log that evidence as the explanation.
• I3 (frames): Feed these alias sets into coreference so that mentions like a country's formal name and its common name resolve to the same entity.
• I4 (learn): Evaluate repair rules or knowledge additions in isolation, one change per run, so a regression can be attributed to exactly one cause.
• I5 (answer): When a reader searches a fable or textbook for supporting facts, spend more of the ontology lookup effort on rare content words than on frequent ones.