📄 OCR and Automated Translation for the Navigation of non-English Handsets: A Feasibility Study with Arabic

Summary

Forensic examiners often need to reach contacts, messages or calendars on a phone whose interface is set to a language they cannot read. This feasibility study asks whether off-the-shelf OCR plus automatic translation can guide such an examiner through Arabic handset menus. The authors photographed the monochrome screen of an old Nokia 2300 in both Arabic and English modes. They manually cropped, rescaled and binarised the images, then ran two unnamed commercial OCR engines at different levels of automation. Raw Arabic recognition was weak, with at most about a third of lines correct. Accuracy rose sharply once each OCR line was matched by edit distance against a dictionary of menu phrases and translated by bilingual lookup. Whole-line, untokenised matching worked best, especially when a human picked the right candidate from the top five. A demonstrator application built on this workflow reached similar accuracy with real handset menu phrases. The result suggests that domain-restricted, phrase-level correction can rescue noisy OCR well enough for practical navigation by non-linguists.

Key points

• Test corpus: 259 photographed displays of a Nokia 2300 (96 x 65 pixel backlit LCD), 118 in Arabic and 141 in English. The ground truth covers 407 Arabic and 474 English lines in four font sizes.
• With the best settings, COTS 1 scored about 75% character, 44% word and 34% line accuracy on Arabic. English scored about 92%, 82% and 78%. Fully automatic zoning and settings degraded results drastically.
• Correction used Levenshtein distance against a simulated menu glossary corpus of 1,665 unique 1–4-word n-grams, followed by bilingual dictionary translation.
• Untokenised line matching beat word-level matching for both engines. Picking from the top-5 candidates raised COTS 1 from 85.9% to 91.1% and COTS 2 from 80.3% to 86.5%. Tokenised COTS 2 stayed near 40%.
• After interactive correction, 81.3% of lines were right for both engines, 9.8% only for COTS 1 and 5.2% only for COTS 2. With 815 phrases taken from four handset manuals, the demonstrator reached 89.7% line accuracy.

Interesting ideas

• I1. Correcting a whole noisy line against a closed inventory of known phrases, instead of word by word, absorbs segmentation errors and beats token-level spell checking.
• I2. Measuring accuracy both for the top candidate and for "correct answer anywhere in the top five" separates ranking failures from coverage failures.
• I3. The two OCR engines fail on partly different lines, so their union covers more than either one alone. This motivates combining multiple engines.
• I4. The study replicates its simulated-corpus results on real-domain phrases to confirm that a synthetic dictionary was not inflating accuracy.
• I5. Human verification against the source is treated as unavoidable, so interactive selection costs little extra time. The authors also note that verification gets harder when the displayed font differs from the source.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): When the tokenizer meets garbled or run-together input, it could fall back to fuzzy matching against a lexicon of multiword expressions before splitting into tokens.
• I2 (learn): Evaluation reports for parser or frame candidates could give top-1 and top-k scores, showing whether errors come from bad scoring or from a missing candidate.
• I3 (learn): Measuring the pairwise overlap of correct outputs among witness parsers would show how much a silver-data agreement scheme can really gain and where all witnesses fail together.
• I4 (answer): Results on synthetic test items, such as generated word problems, should be rechecked on a small naturally sourced set to catch artifacts of the generator.
• I5 (back): Explanations shown to users could render the matched source span in its original form next to the generated text, which makes checking easy.