📄 Buckwalter-based Lookup Tool as Language Resource for Arabic Language Learners

Summary

Arabic is hard for learners to look up in a dictionary. Its words carry rich inflection for tense-aspect, person, number and gender, and they can have clitics attached, such as possessives, object pronouns and conjunctions. Written text also usually leaves out short-vowel diacritics. Before a reader can consult a dictionary, they must split a token into its parts and recover the lemma and root. The authors built the Buckwalter-based Lookup Tool (BBLT), a web interface on top of Buckwalter's morphological analyzer (BMA). Users paste or type one or more Arabic strings. BMA lists every possible analysis: vocalized forms, part of speech for each segment, lemma ids and English glosses. BBLT then reorganizes this output into a readable view. The tool began as an internal aid for Arabic-English MT developers, who used it to inspect words their engines left untranslated. The authors found that the most useful view groups analyses by distinct lemma and shows English glosses. This helps English speakers who cannot use the Hans Wehr dictionary without already knowing a word's base form.

Key points

• BMA (Buckwalter 2005) outputs XML that lists all candidate internal structures of a string: diacritized vocalizations, POS per segment, lemma ids and English glosses.
• BBLT was first built for MT development and evaluation (Voss et al. 2006), alongside tests of synonym lookup in Arabic WordNet (Elkateb et al. 2006).
• The display can be configured:
  • raw BMA XML, or glosses sorted by lemma;
  • glosses with or without clitics and person inflections;
  • diacritized forms in transliteration or in Arabic script;
  • a full table for glossing a whole sentence.
• The Hans Wehr dictionary has about 13,000 entries, but searching it requires knowing the underlying form.
• The implementation is a Java webapp on Tomcat. One servlet runs BMA as a socket-server process. A second servlet handles input from a .jsp page, filters the raw output according to the user's options and forwards the result to a display .jsp.

Interesting ideas

• I1. Show an exhaustive, ambiguous analyzer output collapsed by distinct lemma rather than as a flat list of parses, so a human can quickly see the real lexical choices.
• I2. Offer layered views of the same analysis, from raw machine output to a filtered, human-friendly view with toggles for clitics and inflections, so developers and end users share one tool.
• I3. Build a lookup tool aimed specifically at the words a pipeline failed to handle, here MT's untranslated strings, as a fast diagnostic for coverage gaps.
• I4. Keep the heavy analyzer running as a persistent socket-server process behind a thin request handler, separating analysis from presentation.
• I5. Use whole-sentence interlinear glossing tables, one column per token with segments and glosses, as a reading aid.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The morphology/lemma debug view in crate en could group candidate analyses by lemma with glosses, making ambiguity explainable at a glance.
• I2 (answer): Explanations of answers could offer a terse view and a full-trace view of the same underlying tokens-to-tree-to-frames evidence.
• I3 (learn): Unknown or low-confidence tokens from silver-data runs could be routed to a lookup/inspection view to drive repair rules and lexicon fixes.
• I4 (tree): gparse or the perceptron parser could run as a persistent service, with light front-ends formatting trees for different consumers.
• I5 (back): Generation and translation through the meta-world could emit interlinear token-to-gloss tables as a transparent check of what each source word contributed.