📄 Web-Based Corpus Compilation of the Emirati Arabic Dialect

Summary

Emirati Arabic has few written resources. Most Arabic writing online is MSA, dialect use is informal, and spelling varies widely. This makes it hard to gather raw text for fine-tuning Arabic LLMs on the dialect. El-Ghawi documents the search-and-collection stage of an ongoing Emirati corpus project. Cleaning and annotation are explicitly left out of scope. The workflow combines:

• advanced search-engine queries,
• inclusion and exclusion wordlists that separate Emirati from neighbouring Gulf dialects,
• batch querying with BootCat,
• YouTube caption search through Filmot,
• a mix of hand-gathering and scraping.

The paper sorts the sources found into types: articles, blogs, documents, forums, promotional text, social media and subtitles. It reports 18 sources ready for extraction, mostly general-topic material. It also lists practical failures, such as broken Arabic PDF parsing, noisy forum HTML and poor automatic captions for Emirati. The work is preliminary and mostly qualitative. Its value is a reusable recipe for locating web text in a low-resource dialect and a clear warning about where such pipelines break.

Key points

• Search settings. Google gave the most help, through filters for region, filetype (DOC/PDF) and the .ae domain, plus site targeting (blogspot.com, forum /vb/ lightweight pages) and the "Verbatim" mode. The minus operator removed results in Kuwaiti, Bahraini or Jordanian dialects.

• Wordlists. These were built two ways: from a reference book contrasting Gulf dialects, and from AntConc term-frequency and N-Gram queries over a seed corpus of Emirati books. Both were reviewed by hand and then fed to BootCat.

• Yield. 18 sources were found. Table 2 gives pre-processing word counts for 8 extracted ones:

  • Book1: 373,400
  • Forum1: 272,559
  • ForumStory1: 140,000
  • Articles2: 13,000
  • Subtitles1 and Subtitles2: 7,000 each
  • Subtitles3: 3,436
  • Articles1: 2,500

  These add up to about 819k words (our own sum).

• Extraction problems.

  • Arabic PDF text was scrambled, and OCR and file-conversion support for Arabic was weak.
  • Forum posts carried hand-written HTML, which made scraping noisy.
  • ASR captions made spelling errors on Emirati more than on Saudi or Egyptian Arabic.
  • Social media scraping was not attempted.

• Valuable sources. Heritage-institution documents in traditional Emirati were among the most valuable finds. Formal, non-user-generated dialect text remained scarce.

Interesting ideas

• I1. Use a list of near-neighbour variety words as a negative filter, so that closely related but unwanted varieties are excluded rather than only searching for target words.
• I2. Bootstrap a search lexicon from a small trusted seed corpus with frequency and n-gram counts, curate it by hand, then reuse it as queries to find more text.
• I3. Searching video subtitles by phrase is a fast way to reach spoken, conversational usage that rarely shows up in edited prose.
• I4. Exploit structural URL and site cues, such as lightweight forum paths or a specific blog host, to land on pages that are cleaner and more homogeneous before any parsing.
• I5. Tag every source with both genre and source type, and keep a per-source count table, so that skew toward one kind of text is visible.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): When gathering silver or evaluation text, add negative-control vocabularies (e.g., British or Scots spellings, legal register) to keep the target English variety clean and to measure leakage.
• I2 (tokens): Mine frequent n-grams from a seed of fables and school texts to find words and multiword expressions missing from the en lexicon and morphology, then hunt for more examples of each.
• I3 (frames): Subtitle dialogue could supply varied speech acts, imperatives and indirect requests for testing prag's speech-act and implicature handling on natural conversation.
• I4 (learn): When harvesting raw English text for witness parsers, pick sources by markup simplicity to cut cleanup noise that would otherwise corrupt silver trees.
• I5 (answer): Label QA and test sets by genre and type (tale, word problem, textbook) and report scores per label, so that a gain in one genre is not mistaken for general improvement.