Elamur Research⌕
☰
home / papers / 2021 / Leveraging English Word Embeddings for Semi-Automatic Semantic Classification in Nêhiyawêwin (Plains Cree)

Leveraging English Word Embeddings for Semi-Automatic Semantic Classification in Nêhiyawêwin (Plains Cree)

Paper · 2021 · Atticus Harrigan and Antti Arppe

Summary

The authors wanted semantic subclasses of Nêhiyawêwin (Plains Cree) nouns and verbs to use as predictors when modelling a verbal alternation. No suitable ontology existed, and the native corpus of about 140,000 tokens was too small for useful vectors. Their workaround uses the English side of the Wolvengrey (2001) bilingual dictionary. They first cleaned the English glosses: stop-word removal, synonym substitution, spelling normalization, splitting of word senses and lemmatization. Each entry then received the average of the pre-trained Google News Word2vec vectors for its gloss words. These definition vectors were grouped with Ward hierarchical agglomerative clustering, run separately within each grammatical class. The resulting clusters were refined by hand. Nouns needed only light correction, while the large VAI and VTA verb classes needed substantial rework. The approach lets a lesser-resourced language define its own categories instead of borrowing a foreign ontology. It gives classification work a fast start, and it should transfer to any language with a dictionary glossed in a well-resourced language.

Key points

  • Source dictionary: Wolvengrey (2001), with 21,717 lemma entries. Vectors: Google News Word2vec, 300 dimensions, about 3 million words in the vocabulary.
  • Earlier attempts failed and were discarded: PCA on co-occurrence matrices from the 140,000-token corpus, and tf-idf or Word2vec trained on the dictionary alone.
  • Clustering setup:
    • Euclidean distance matrix, hclust in R with the Ward method.
    • The number of clusters was chosen impressionistically.
    • Verb clusters: VII 10, VAI 25, VTI 15, VTA 20.
    • Nouns were pre-split into NA, NDA, NI and NDI.
  • Largest manually adjusted classes: AI-state (2083), AI-action (1982), NI-nominal (1783), TI-action (1409), TA-action (1013), NA-persons (720).
  • Emotion verbs were grouped with thought verbs into a single cognition class. This follows the morpheme {-êyi-} ('thought') and the way the dictionary glosses these verbs. The clusters are freely available online.

Interesting ideas

  • I1. Split items by their grammatical class (transitivity, animacy, alienability) before clustering; without this split, clustering mostly rediscovers grammar instead of meaning.
  • I2. Represent a word without usable context by its dictionary gloss, embedded through a better-resourced vocabulary, and use the result only to propose candidate groupings.
  • I3. A hierarchical cluster tree can be cut at any depth after a single run, so the granularity of a classification can be revisited cheaply.
  • I4. When correcting clusters, assign each item by the core purpose of the action, not by shared surface words; smoking meat as cooking is a different act from smoking a cigarette.
  • I5. Check vocabulary coverage before embedding, for example by normalizing American and Canadian spellings, because unmatched items silently drop out of the analysis.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): when inducing verb classes or ontology groups, first partition by valency or subcategorization frame so that the semantic grouping is not swamped by syntax.
  • I2 (world): rare WordNet/Wikidata entries could get gloss-based proposals for ontology placement, which then pass explicit review rather than becoming opaque features.
  • I3 (world): store compiled verb or noun classes as a dendrogram so that coarse classes and fine classes come from one explainable structure.
  • I4 (frames): use purpose or goal as a tie-breaker when two VerbNet-style frames match a verb's lemma equally well.
  • I5 (tokens): normalize lemmas for spelling variants before lexicon lookup, and log every miss so that silent losses become visible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AlbertaEdmonton · Canada23 works · 2,655 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Atticus Harrigan and Antti Arppe. 2021. Leveraging English Word Embeddings for Semi-Automatic Semantic Classification in Nêhiyawêwin (Plains Cree). Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.americasnlp-1.12

@inproceedings{harrigan2021leveraging,
  title = {{Leveraging English Word Embeddings for Semi-Automatic Semantic Classification in Nêhiyawêwin (Plains Cree)}},
  author = {Atticus Harrigan and Antti Arppe},
  booktitle = {Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas},
  year = {2021},
  doi = {10.18653/v1/2021.americasnlp-1.12},
  url = {https://doi.org/10.18653/v1/2021.americasnlp-1.12}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antti Arppe, Katherine Schmirler, Atticus G Harrigan, and Arok Wolvengrey. 2020. A morphosyntactically tagged corpus for plains cree. In Papers of the Forty-Ninth Algonquian Conference. Michigan State University Press. find
  2. Antti Arppe et al. 2008. Univariate, bivariate, and multivariate methods in corpus-based lexicography: A study of synonymy. find
  3. Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1):1– 48. ↗
  4. Joan Bresnan, Anna Cueni, Tatiana Nikitina, and R Harald Baayen. 2007. Predicting the dative alternation. In Cognitive foundations of interpretation, pages 69–94. KNAW. find
  5. Clare Cook. 2014. The clause-typing system of Plains Cree: Indexicality, anaphoricity, and contrast, volume 2. OUP Oxford. find
  6. Daniel Dacanay, Antti Arppe, and Atticus Harrigan. 2021. Computational Analysis versus Human Intuition: A Critical Comparison of Vector Semantics with Manual Semantic Classification in the Context of Plains Cree. In Proceedings of the 4th Workshop on the Use of Computational Methods in the Study of Endangered Languages, volume 1, pages 33–43. find
  7. Dagmar Divjak and Stefan Th Gries. 2006. Ways of trying in russian: Clustering behavioral profiles. Corpus linguistics and linguistic theory, 2(1):23–60. ↗
  8. Birgit Hamp and Helmut Feldweg. 1997. Germanet-a lexical-semantic net for german. In Automatic information extraction and building of lexical semantic resources for NLP applications.
  9. Patrick Hanks. 1996. Contextual dependency and lexical sets. International journal of corpus linguistics, 1(1):75–98. ↗
  10. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. ↗
  11. George A Miller. 1998. WordNet: An electronic lexical database. MIT press. find
  12. Hien Pham and Harald Baayen. 2015. Vietnamese compounds show an anti-frequency effect in visual lexical decision. Language, Cognition and Neuroscience, 30(9):1077–1095. ↗
  13. R Core Team. 2017. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. find
  14. Linda Visitor, Marie-Odile Junker, and Mimie Nea-cappo. 2013. Eastern james bay cree thematic dictionary (southern dialect). Chisasibi: Cree School Board. find
  15. Shan Wang and Francis Bond. 2013. Building the chinese open wordnet (cow): Starting from core synsets. In Proceedings of the 11th Workshop on Asian Language Resources, pages 10–18. find
  16. Arok Wolvengrey. 2001. Nēhiyawēwin: itwēwina = Cree: words. University of Regina press. find
  17. Arok Wolvengrey. 2005. Inversion and the absence of grammatical relations in plains cree. Morphosyntactic expression in functional grammar, 27. find
  18. Liang-Chih Yu, Jin Wang, K Robert Lai, and Xuejie Zhang. 2017. Refining word embeddings for sentiment analysis. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 534–539. find
  19. Beth Levin. 1993. English verb classes and alternations: A preliminary investigation. University of Chicago press. find
  20. Olga Majewska, Ivan Vulić, Diana McCarthy, and Anna Korhonen. 2020. Manual clustering and spatial arrangement of verbs for multilingual evaluation and typology analysis. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4810–4824. find
  21. Daniel McFadden. 1977. Quantitative Methods for Analyzing Travel Behaviour of Individuals: Some Recent Developments. Technical report. find
  22. Daniel McFadden et al. 1973. Conditional logit analysis of qualitative choice behavior. find
  23. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/harrigan-2021-leveraging-english-word-embeddings-semi/