📄 ELAN: a Professional Framework for Multimodality Research

Summary

Studies of gesture, sign language and multimodal interaction often rely on video recorded at 50 frames per second. At that rate a single-frame error of 20 ms can change which phoneme or articulator movement an annotation refers to. The paper reports on ELAN 2.4.2, an annotation tool from the MPI for Psycholinguistics, developed with the researchers who use it. The main technical change is on Windows: the earlier Java Media Framework playback path was replaced by a thin bridge to native DirectX libraries, because Java has no high-performance media solution of its own. SPEX, an outside organization, tested the new version against the old JMF version and MediaTagger, using Praat as the timing reference. The DirectX version replayed segments most faithfully. The paper also describes other additions: more media formats, up to four synchronized videos, saved structured queries, search across many files, import and export converters, interlinear printing, and features that speed up annotation. Together these make ELAN a dependable desktop tool for annotation work where timing is critical, while the web tool ANNEX gives browser access to archives.

Key points

• SPEX timing test:
  • 100 speech segments, cut in Praat, were each replayed 25 times in every tool.
  • Human perception was part of judging the deviations.
  • The DirectX-based ELAN had very few glitches.
  • With an MPG main file, segments shorter than 40 ms overshot by 10 ms in about 20% of cases. With a WAV main file, short segments played correctly.
• Precision and design:
  • Annotations can be placed with 1 ms precision.
  • Annotations sit on "tiers" that can be independent, aligned or embedded.
  • ELAN is written in Java and runs on Windows, Mac OS and Linux.
• Media:
  • Before version 2.4, only MPEG-1 and WAV were supported. From 2.4 on, the underlying framework (DirectX, QuickTime or JMF) decides which formats work, which opens the door to higher-resolution MPEG-2.
  • The number of linked videos grew from 1 to 2 and then to 4. Each video can have its own stored time offset.
• Interoperability:
  • Shoebox/Toolbox and CHAT files can be both imported and exported.
  • Signstream and Transcriber files can be imported or converted.
  • The native archival format is EAF.
• Planned work: structured search across multiple files, links to the LEXUS lexicon and ISO DCR, and support for time-series data from eye trackers and cyber gloves.

Interesting ideas

• I1. Independent timing benchmark. Bring in a neutral party and use a trusted reference tool (here Praat) to measure playback accuracy over repeated trials. This gives defensible numbers for any annotation or labeling tool where alignment matters, for example speech corpora or video datasets.
• I2. Thin native bridge. When a portable language lacks a high-quality library, connect it to the platform's native framework through a small layer instead of building one yourself. The application stays portable while the critical component runs at native quality.
• I3. Per-stream offsets with bulk realignment. Store a time offset for each media file and allow shifting all existing annotations in one step after the offset changes. This prevents costly re-annotation whenever multi-camera or multi-sensor recordings turn out to be out of sync.
• I4. Reusable queries with plain output. Let users save structured queries that combine temporal and structural constraints, then export results as tab-delimited text. Corpus searches become repeatable and feed directly into external statistics tools.
• I5. One-time index for multi-file search. Build an index once and rebuild it only when the files change. Search across arbitrarily large user-defined collections stays fast, which suits any growing annotated archive.