📄 Multilingual Gradient Word-Order Typology from Universal Dependencies

Summary

The typological databases WALS and Grambank describe languages through discrete categories. As a result, variation is lost, and contradictions arise between sources. For example, a language in which the "noun–adjective" order predominates but is not absolute may be classified as "mixed" by one database but not by another. The authors propose using gradient, i.e. continuous, values for word-order features instead of categories. For each treebank from Universal Dependencies (UD), they use the POS annotations to count how many times one constituent precedes another, and take this proportion as the feature value. In this way, they created a small initial (seed) dataset and described a procedure that can easily be extended to other features and languages. In addition, they proposed a new task: predicting typological features via regression rather than classification. The models were trained on language vector representations. On average, linear regression makes smaller errors than logistic regression on binarized data. The authors conclude that categorical databases are too coarse, which may partly explain why WALS features are of little help to NLP models.

Key points

• The dataset was compiled from 132 UD treebanks covering 91 languages. It contains five word-order features: adjective–noun (87A in WALS, GB025 in Grambank), numeral–noun, subject–verb, object–verb, and object–subject.
• To add a new feature, it is enough to change the set of UD tags in the code that are used to search for word pairs. The tags used are listed in the appendix.
• The experiment used language vectors from Östling & Tiedemann (2017) and Malaviya et al. (2017). For each feature, there were only about 40 languages for training and about 10 for testing. If a language had several treebanks, one was chosen at random. The baseline logistic model worked with proportions rounded to 0 or 1 using a threshold of 0.5.
• The mean squared error (MSE) of linear regression is lower in most cases. For subject–verb order on the Östling vectors, it is 0.078 versus 0.306 for the logistic model; for adjective–noun on the Malaviya vectors, it is 0.141 versus 0.378. However, r² is mostly negative even for the linear models, so these models are only baselines and very limited.
• The authors caution that the proportions depend on the source and domain of the treebank. Therefore, parallel corpora are preferable for a sound comparison of languages.