📄 Lexical Markup Framework (LMF)

Summary

Lexical resources for NLP are expensive to build, maintain and connect to applications, and they rarely share a common format. This paper reports on ongoing work in the ISO-TC37/SC4/WG4 group to standardize lexicon structure as LMF (ISO 24613). The effort builds on earlier projects such as GENELEX, EAGLES, MULTEXT, PAROLE, SIMPLE and ISLE, and on the terminology standard TMF. LMF is defined as a UML model with two layers. A small core package (Database, Lexicon, Lexical Entry, Form, Sense, Representation Frame) is combined with optional extension packages for morphology, inflectional paradigms, MWE patterns, syntax, semantics, multilingual notations and MRDs. Linguistic values such as gender or transitivity are not built into the model. Instead they come from the ISO Data Category Registry. The authors illustrate the design with English inflection, an Italian verb frame from Parole/CLIPS and a French predicate from Mel'čuk's explanatory-combinatorial dictionary. There is no quantitative evaluation. The contribution is a shared, language-independent schema that lets monolingual and multilingual lexicons of any size be exchanged and merged. Publication as an ISO standard was targeted for 2007.

Key points

• LMF is ISO 24613 and belongs to a family of ISO-TC37 NLP standards:
  • word segmentation (ISO 24614);
  • annotation (ISO 24611/24612/24615);
  • feature structures (ISO 24610).
• These standards rely on low-level "constant" standards: data categories (ISO 12620 revision), ISO 639, ISO 15924, ISO 3166 and Unicode (ISO 10646).
• The Data Category Registry (DCR) follows ISO/IEC 11179-3:2003 and supplies all attribute names and values (e.g., /feminine/, /transitive/). LMF itself only defines structure.
• In the core model, each lexicon covers one language, and every single word, multi-word expression or affix is exactly one Lexical Entry. A Sense belongs to one entry and is never shared between entries.
• Examples:
  • Morphology: "clergyman" → "clergymen" is derived by removing 2 letters and adding "en".
  • Syntax: the Parole/CLIPS frame for "amare" has NP subject and object and the auxiliary "avere".
  • Semantics: the DEC predicate "aider1" has arguments X, Y, Z and W and eight syntactic constructions.
• In 2005 the group decided to add an XML serialization as an informative annex. A companion LIRICS report maps OLIF-2, Parole/Clips, LC-Star, WordNet, FrameNet and BDéf onto LMF.

Interesting ideas

• I1.

  Structure separate from vocabulary. The schema defines only containers and relations, while all linguistic labels point to an external registry. A Ukrainian lexicon could adopt LMF and reuse shared category definitions without changing the model.

• I2.

  Small mandatory core with opt-in extensions. Resource builders choose only the packages they need, but every extension must hang off the core. This keeps small and large resources compatible and is a good pattern for any shared data schema.

• I3.

  Three levels of morphological commitment. A lexicon can:

  • list all inflected forms explicitly;
  • only name a paradigm class such as "asMan" (underspecified);
  • fully define the paradigm as stem operations like remove/add letters.

  The paper notes that full paradigms pay off for rich-morphology languages such as German or Hungarian, which suggests the same holds for Ukrainian.

• I4.

  Linking syntax to semantics through arguments. Constructions are not tied directly to a predicate. Instead, each syntactic slot maps to a semantic argument. One predicate can then cover many alternative frames of a verb, or even senses with different parts of speech, such as a verb and its action noun.

• I5.

  Uneven depth is allowed. One lexicon can describe technical terms in rich detail while leaving general words shallow. It can also mix synsets and predicates across languages. This makes it possible to grow a resource gradually instead of requiring full coverage from the start.