📄 Reusable Tagset Conversion Using Tagset Drivers

Summary

Tagsets in corpora and treebanks are inconsistent across languages and projects. Moving tags from one scheme to another usually requires a dedicated converter for each pair of schemes, and that effort grows quickly as more tagsets appear. Zeman addresses this with a shared intermediate representation, Interset: a universal inventory of morphosyntactic features and values. Each tagset gets one "driver" with two operations. The decoder maps a tag to an Interset feature structure, and the encoder maps a feature structure back to a tag of that tagset. Any source tagset can then be converted to any target by decoding and re-encoding, so adding a new tagset requires only one new driver rather than converters to every existing scheme. The encoder must always produce a tag that is valid in the target tagset, even when the input feature combination never occurs there. The paper describes drivers for tagsets used in multilingual resources, including CoNLL shared-task treebanks, and tests the approach in experiments. The work matters because it makes tagset harmonization modular and reusable for cross-lingual NLP.

Key points

• Interset is a universal set of features (part of speech, gender, number, case, tense, etc.). It serves as an interlingua between tagsets.
• Each driver provides decode (tag → features) and encode (features → tag). With N tagsets you need about N drivers instead of roughly N² pairwise converters.
• The encoder chooses the closest permitted tag when a feature combination is not legal in the target tagset.
• Drivers cover tagsets from treebanks used in the CoNLL 2006/2007 shared tasks. Related standardization efforts such as EAGLES and MULTEXT-East are discussed.
• Caveat for the library: the supplied PDF extraction was almost entirely garbled (broken font encoding). Table numbers, the exact list of drivers and the evaluation results could not be verified, so check them against the original paper before citing.

Interesting ideas

• I1. Hub-and-spoke conversion: Routing every conversion through one neutral representation turns a quadratic integration problem into a linear one. The same idea suits mapping Ukrainian corpora tags (e.g., to UD features), product codes, or geographic coding schemes.
• I2. Encoder must be total: Requiring the encoder to output a valid target label for any input, by falling back to the nearest allowed combination, makes conversions robust. This is useful whenever source data are richer or messier than the target schema.
• I3. Preserving untranslatable detail: Information the universal schema cannot represent can be stored in a side slot so that a decode–encode round trip stays lossless. This helps keep annotation history when normalizing data.
• I4. Round-trip testing: Decoding and re-encoding the same tagset and checking whether the original tags come back is a cheap sanity check for any schema-mapping tool.