📄 DiVE-Arabic: Gulf Arabic Dialogue in a Virtual Environment

Summary

The authors argue that most instruction-giving dialogue corpora, such as TRAINS and Map Task, record talk in fairly static settings and say little about the physical actions surrounding speech. Arabic resources share this gap. CallHome lacks both situational context and prosody, while the IVAr map-task material has prosody but no action record. To address this, the authors built DiVE-Arabic, a corpus of spoken Gulf Arabic dialogue recorded during a navigation-and-retrieval task in a 3D game world. It deliberately copies the design of the English SCARE corpus so the two can later be merged into a parallel situated English–Arabic resource. Recording used only portable equipment, so fieldwork in the UAE and Oman could approximate lab conditions. The corpus links three kinds of record:

• per-speaker audio,
• screen video,
• automatically logged movements of the instruction follower.

All layers are aligned to the signal in an NXT-style stand-off XML database, with prosodic annotation in a Gulf Arabic ToBI scheme. The authors present it as the first Arabic corpus pairing intonation with recorded actions, enabling cross-modal and cross-linguistic dialogue studies.

Key points

• DiVE-Arabic contains about 4 hrs 40 mins of speech from 26 tertiary-educated participants (17 female, 9 male, average age about 25). Participants were recorded in same-gender pairs in Al Ain (UAE) and Buraimi (Oman), as part of the IVAr project. SCARE, by comparison, has 30 participants (19 male, 11 female, average age 30) and 3 hrs 41 mins.

• The task reuses SCARE's QuakeII worlds, running on Jake2, a Java port of QuakeII. These worlds have two levels of 7–9 rooms, with buttons that open cabinets. The instruction giver (IG) holds the map and the task list, while the instruction follower (IF) moves through the world.

• Recording hardware was portable:

  • a Marantz PMD661 recorder,
  • two Shure SM10 head-mounted microphones,
  • a Windows 7 laptop.

  Action data comes straight from game logs (dm2 files), processed by an adapted SCARE parser.

• The annotation pipeline runs in four steps:

  1. A Praat script segments turns by detecting low-intensity stretches, and the boundaries are then corrected by hand.
  2. Turns receive an ASCII broad transcription and are force-aligned word by word with the ProsodyLab Aligner.
  3. Prosody is labelled in Gulf Arabic ToBI.
  4. ELAN is used for video-linked action labels.

  Everything is merged into a Nite NXT stand-off database, with release planned for 2015.

• Earlier work on SCARE (Gargett 2012) found that, after a verbal silence, English IGs tend to say more when no action is going on than when the follower is acting. DiVE is meant to test whether Arabic speakers show the same pattern.

Interesting ideas

• I1. Instead of translating texts, the authors obtain a parallel corpus by repeating the same task, world and demographic profile in a second language, so both languages describe identical situations.
• I2. Each annotation layer is stored separately and either anchored to the timeline or symbolically pointing at another layer, for example dialogue acts pointing to utterances that are aligned to audio.
• I3. Language is paired with an automatic log of what actually happened in the environment, giving a ground-truth record of actions alongside the words.
• I4. Feedback tokens like "ok", "mhm" and "um" are analysed according to whether a pause in speech coincides with a pause in action, which treats a physical action as a possible conversational turn.
• I5. Turns are first segmented by a cheap automatic heuristic (low signal intensity) and then corrected by hand, so human effort goes only to fixing errors.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (back): Fixing a meta-world scenario and gathering its realizations across languages would provide a natural test of whether translation through the meta-world keeps the same entities and events.
• I2 (world): Keeping tokens, UD tree, frames and world entities as distinct stand-off layers joined by IDs would let any answer be traced back through every step without merging layers.
• I3 (world): Simulated or logged event sequences paired with instruction texts could act as gold data for checking event order and state changes in Mova's text world.
• I4 (frames): Speech-act and implicature modules could allow an acknowledgement or a reported action to count as a reply, rather than requiring verbal responses only.
• I5 (learn): The same propose-then-correct scheme fits silver-data pipelines, where witness parsers draft annotations and the LLM teacher or a human reviews only disputed spots.