Deciphering Undersegmented Ancient Scripts Using Phonetic Prior

What it is about

The paper asks how to match an unsegmented or undersegmented script against the vocabulary of a known language when the language family of the lost script is not known. The model jointly segments the text and aligns each matched span to a known stem with a monotonic, edit-distance-style alignment. Characters are embedded from their IPA features, so that similar sounds get similar vectors, and a penalty keeps the lost script from losing sounds (each lost symbol should map to one known sound). It also proposes a closeness measure: coverage of the lost text against a known language's vocabulary, plotted against confidence. On deciphered test cases the full model scores A@10 of 0.661 averaged over settings for Gothic (0.419 with no phonological knowledge), and it finds Proto-Germanic as the closest known language, with 75 percent A@10 even with no prior knowledge. On Iberian personal names it reaches 75 percent top-10 with full knowledge; the closeness measure does not single out Basque.

Ideas

• I1. Closeness by coverage: for a text in one Aramaic variety, match its words against each other variety's stem list and plot coverage against match confidence. This gives an empirical test of which variety (languages/arc-x-targum, languages/tmr, languages/jpa) is nearest, and it can be checked against the known history.
• I2. Monotonic alignment with insertion penalty: align consonant skeletons of cognate forms with substitution, deletion and insertion costs, where insertion is penalised by a parameter alpha. Use it to match a word in one variety with its counterpart in another, and as a distance for tools/ocrfix repairs.
• I3. Sound-inventory check: require each consonant of a variety to correspond to a consistent consonant of the known language, and flag the cases where one letter maps to several sounds or where a sound disappears. Write the resulting table into languages/arc/script-phonology.md.

For Mova

I1 belongs in languages/arc/family.md and in the closeness tests for the Aramaic varieties. I2 is an alignment routine for tools/ocrfix and for cognate lookup across languages/arc. I3 is a check for languages/arc/script-phonology.md. The decipherment goal itself is outside Mova; the parts that transfer are the alignment, the closeness measure and the inventory check.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00354

@article{luo2021deciphering,
  title = {{Deciphering Undersegmented Ancient Scripts Using Phonetic Prior}},
  author = {Luo, Jiaming and Hartmann, Frederik and Santus, Enrico and Barzilay, Regina and Cao, Yuan},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00354},
  url = {https://doi.org/10.1162/tacl_a_00354}
}

References

16 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eduardo Orduña Aznar. 2005. Sobre algunos posibles numerales en textos ibéricos. Palaeohispanica, 5:491–506. find
2. Taylor Berg-Kirkpatrick and Dan Klein. 2013. Decipherment with a million random restarts. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 874–878. Association for Computational Linguistics. find
3. Kevin Knight and Kenji Yamada. 1999. A computational approach to deciphering unknown scripts. Unsupervised Learning in Natural Language Processing. find
4. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018a. Unsupervised machine translation using monolingual corpora only. find our card
5. Donald A. Ringe. 2017. From Proto-Indo-European to Proto-Germanic. Oxford. find
6. Alejandro Garcia Sinner and Javier Velaza. 2019. Palaeohispanic Languages and Epigraphies. Oxford University Press. find
7. Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018b. Word translation without parallel data. In International Conference on Learning Representations. find
8. Benjamin Snyder, Regina Barzilay, and Kevin Knight. 2010. A statistical model for lost language decipherment. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1048–1057, Uppsala, Sweden. Association for Computational Linguistics. DOI: https://doi.org/10.1093/oso/9780198792581.001.0001 ↗
9. Jiaming Luo, Yuan Cao, and Regina Barzilay. 2019. Neural decipherment via minimum-cost flow: From ugaritic to linear b. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3146–3155, Florence, Italy. Association for Computational Linguistics. find our card
10. Kenneth N. Stevens. 2000. Acoustic phonetics, volume 30. MIT Press. find
11. Noemı́ Moncunill Martı́ and others. 2017. Indigenous naming practices in the western Mediterranean: The case of Iberian. Studia Antiqua et Archaeologica, 23(1):7–20. find
12. Larry Trask. 2008. Etymological Dictionary of Basque. http://www.bulgari-istoria -2010.com/Rechnici/baski-rechnik .pdf. ↗
13. David R. Mortensen, Siddharth Dalmia, and Patrick Littell. 2018. Epitran: Precision G2P for many languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA). find
14. Norbert Wagner. 2006. Zu got. hv, q und ai, au. Historische Sprachforschung/Historical Linguistics, pages 286–291. find
15. Julius Zacher. 1855. Das Gothische Alphabet Vulfilas und das Runen Alphabet: eine Sprachwissenschaftliche Untersuchung, FA Brockhaus. find
16. Malte Nuhn, Julian Schamper, and Hermann Ney. 2013. Beam search for solving substitution ciphers. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1568–1576, Sofia, Bulgaria. Association for Computational Linguistics. A Appendices A.1 Derivations for Dynamic Programming find