📄 ALEP-Based Distributed Grammar Engineering

Summary

The paper describes how to organize the joint development of a large grammar when several linguists, sometimes at different institutions, work on it. The authors call this approach distributed grammar engineering (DGE). They distinguish two senses of the term. The first is the distribution of work among people. The second is the distribution of levels of linguistic description across processing stages. Both require a modular grammar whose modules can be reliably assembled into a coherent resource. The authors present the tools for this using the ALEP platform and its use in the LS-GRAM project. The first tool concerns data storage. The grammar code is split into files by phenomenon type, and sets of files are combined into objects in the AUL language within the AOR repository. This makes it possible to assemble specialized subgrammars and exchange them between teams. The second tool is specifiers, i.e., rule-class labels that internally divide the grammar into parts. Because the Refine operation can be run many times in succession, it supports the stepwise combination of modules. The authors conclude that modularity facilitates testing, parallel work, maintenance, and the adaptation of the grammar to different applications. In their view, the main shortcoming is the lack of sophisticated version control.

Key points

• The LS-GRAM project (LRE 61,029) developed broad-coverage grammars for nine EU languages. Several grammars were written jointly by two institutions: Dutch by SST (Utrecht) and KUL (Leuven), German by IAI (Saarbrücken) and IMS (Stuttgart), Italian by DIMA (Turin) and ILC (Pisa), and Spanish by FBG and UPF (Barcelona). A joint multilingual DGE was never implemented in the project.
• ALEP objects cover constituent rules (lg_ps_rules), type and feature declarations, the lexicon, and two-level morphology. A complete grammar is combined by an lg_lingware_group object. The nowv feature assigns each object a unique combination of name, owner, workspace, and version.
• The Export/Import functions pack selected objects into a tar archive and unpack it. On import, the objects are registered in the AOR, and all required directories and files are recreated.
• Specifiers divide the grammar along two dimensions. The "vertical" one is by operation: word segmentation, parsing, or Refine only. The "horizontal" one is by unit level: from morphemes to words, from words to sentences, from sentences to paragraphs. This division improves performance, and an underspecified specifier makes it possible to combine modules.
• The authors formulate requirements for future tools:
  • automatic logs of changes and authorship;
  • comparison and merging of parallel module versions;
  • checking how information is passed between modules;
  • definition of repository objects directly via specifiers.