Elamur Research⌕
☰
home / papers / 2024 / Automated Cognate Detection as a Supervised Link Prediction Task with Cognate Transformer

Automated Cognate Detection as a Supervised Link Prediction Task with Cognate Transformer

Summary

The paper addresses automatic cognate detection, that is, identifying words in related languages that descend from a common ancestor. Cognates are needed to establish sound correspondences, reconstruct proto-languages, and build phylogenetic trees. The strongest previous approaches, notably LexStat with Infomap clustering, compute pairwise word similarity from phoneme distributions. They make almost no use of existing cognate-set annotation. The authors frame the task as supervised link prediction. Words for the same concept are aligned together into a multiple sequence alignment (MSA) with the SCA method from LingPy and recoded into ASJP. For every pair of languages, the model directly outputs the probability that their words are cognate. The core is Cognate Transformer, derived from the protein model MSA Transformer. It is extended with triangle multiplication and triangle attention modules from AlphaFold2, which account for the transitivity of links. When there is enough annotation, the method outperforms existing approaches and keeps improving as the annotation grows. It is also much faster than pairwise schemes. For ancient and low-resource languages, where there is too little text for language models, it is a practical tool.

Key points

  • The architecture is end-to-end: Cognate Transformer with row and column attention over the MSA, followed by an outer product mean, a pairwise module with triangle operations, and binary link classification (0/1).
  • On some datasets the model surpasses previous methods with as few as 20 concepts per family. It can also transfer to new datasets with a small amount of additional annotation.
  • On Sino-Tibetan the model improved the previous best result by about 5%. On all other datasets the B-Cubed F-score exceeds 80%. On Romance and Austronesian the model falls slightly behind SVM and LexStat-Infomap, and on Pama-Nyungan behind LexStat-Infomap.
  • Ablations show that MSA input instead of pairwise alignments yields higher quality and sharply reduces computation time. The triangle modules noticeably improve the result.
  • Error analysis showed that the model learns regular sound changes, such as *k > h under Grimm's law, from as few as two examples. At the same time, it does not recognize partial cognacy and does not distinguish cognates from loanwords. For a family of 136 languages, 10 GB of GPU memory fits only a batch size of 4.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.eacl-long.58 · arXiv 2402.02926

@inproceedings{akavarapu2024automated,
  title = {{Automated Cognate Detection as a Supervised Link Prediction Task with Cognate Transformer}},
  author = {Akavarapu, V.S.D.S.Mahesh and Bhattacharya, Arnab},
  booktitle = {EACL 2024 (St. Julian's)},
  year = {2024},
  doi = {10.18653/v1/2024.eacl-long.58},
  eprint = {2402.02926},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2024.eacl-long.58}
}

References

30 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. V.S.D.S.Mahesh Akavarapu and Arnab Bhattacharya. 2023. Cognate transformer for automated phonological reconstruction and cognate reflex prediction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6852–6862, Singapore. Association for Computational Lin… find
  2. Enrique Amigó, Julio Gonzalo, Javier Artiles, and Fe-lisa Verdejo. 2009. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information retrieval, 12:461–486. find
  3. Cecil H Brown, Eric W Holman, Søren Wichmann, and Viveka Velupillai. 2008. Automated classification of the world’s languages: a description of the method and preliminary results. Language Typology and Universals, 61(4):285–308. find
  4. Young Min Kim, Kalvin Chang, Chenxuan Cui, and David R. Mortensen. 2023. Transformed protoform reconstruction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 24–38, Toronto, Canada. Association for Computational Linguistics. find
  5. Johann-Mattis List. 2010. Sca: Phonetic alignment based on sound classes. In European Summer School in Logic, Language and Information, pages 32–51. Springer. find
  6. Johann-Mattis List. 2012. LexStat: Automatic detection of cognates in multilingual wordlists. In Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UN-CLH, pages 117–125, Avignon, France. Association for Computational Linguistics. find
  7. Johann-Mattis List and Robert Forkel. 2021. Lingpy. a python library for historical linguistics. version 2.6.9. find
  8. Lyle Campbell. 2013. Historical linguistics. Edinburgh University Press. find
  9. Johannes Dellert. 2018. Combining information-weighted sequence alignment and sound correspondence models for improved cognate detection. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3123–3133, Santa Fe, New Mexico, USA. Association for Computational Lingui… find
  10. Gerhard Jäger, Johann-Mattis List, and Pavel Sofroniev. 2017. Using support vector machines and state-of-the-art algorithms for phonetic alignment to identify cognates in multi-lingual wordlists. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Lingu… find
  11. John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. 2021. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589. find our card
  12. Diptesh Kanojia, Raj Dabre, Shubham Dewangan, Pushpak Bhattacharyya, Gholamreza Haffari, and Malhar Kulkarni. 2020. Harnessing cross-lingual features to improve cognate detection for low-resource languages. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1384–… find
  13. Johann-Mattis List, Simon J Greenhill, and Russell D Gray. 2017. The potential of automatic word comparison for historical linguistics. PloS one, 12(1):e0170046. find
  14. Johann-Mattis List, Philippe Lopez, and Eric Bapteste. 2016. Using sequence similarity networks to identify partial cognates in multilingual wordlists. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 599–605, Berlin, Germany.… find
  15. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  16. Roddy MacSween and Andrew Caines. 2020. An expectation maximisation algorithm for automated cognate detection. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 476–485, Online. Association for Computational Linguistics. find
  17. James P Mallory and Douglas Q Adams. 2006. The Oxford introduction to proto-Indo-European and the proto-Indo-European world. Oxford University Press, USA. find
  18. Abhijnan Nath, Rahul Ghosh, and Nikhil Krishnaswamy. 2022. Phonetic, semantic, and articulatory features in Assamese-Bengali cognate detection. In Proceedings of the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 41–53, Gyeongju, Republic of Korea. Association for Computa… find
  19. Saul B Needleman and Christian D Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology, 48(3):443–453. find
  20. Taraka Rama. 2016. Siamese convolutional networks for cognate identification. In Proceedings of COL-ING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1018– 1027, Osaka, Japan. find
  21. The COLING 2016 Organizing Committee. find
  22. Taraka Rama. 2018. Similarity dependent Chinese restaurant process for cognate identification in multilingual wordlists. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 271–281, Brussels, Belgium. Association for Computational Linguistics. matching consonant c… find
  23. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  24. Taraka Rama and Johann-Mattis List. 2019. An automated framework for fast cognate detection and Bayesian phylogenetic inference in computational historical linguistics. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6225–6235, Florence, Italy. Assoc… find
  25. Taraka Rama, Johann-Mattis List, Johannes Wahle, and Gerhard Jäger. 2018. Are automatic methods for cognate detection good enough for phylogenetic reconstruction in historical linguistics? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Lingui… find
  26. Priya Rani, Koustava Goswami, Adrian Doyle, Theodorus Fransen, Bernardo Stearns, and John P. McCrae. 2023. Findings of the SIGTYP 2023 shared task on cognate and derivative detection for low-resourced languages. In Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and… find
  27. Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. 2021. Msa transformer. In International Conference on Machine Learning, pages 8844–8856. PMLR. find
  28. Robert R. Sokal and Charles Duncan Michener. 1958. A statistical method for evaluating systematic relationships. University of Kansas science bulletin, 38:1409–1438. find
  29. Julie D Thompson, Toby J Gibson, and Des G Higgins. 2003. Multiple sequence alignment using clustalw and clustalx. Current protocols in bioinformatics, (1):2–3. find
  30. Peter Turchin, Ilia Peiros, and Murray Gell-Mann. 2010. Analyzing genetic connections between languages by find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/akavarapu-2024-cognate-link-prediction/