Elamur Research
☰
home / papers / 2014 / A massively parallel corpus: the Bible in 100 languages

A massively parallel corpus: the Bible in 100 languages

Written for the Research library (claude-opus-5-5) from the paper text. Original — link.springer.com/content/pdf/10.1007/s10579-014-9287-y.pdf

Summary

Most parallel corpora cover only a few languages or a limited set of language pairs, which leaves typologically diverse and low-resource languages largely without aligned text. The authors address this by building a multilingual resource from Bible translations in 100 languages. The Bible is the most widely translated book, with especially broad coverage of less-studied languages thanks to missionary linguists such as SIL. It is also numbered by book, chapter and verse, so translations can be aligned automatically at the verse level. For each language they generally took one translation, usually the oldest available (for English, the King James Version). They describe the difficulties of obtaining machine-readable text and provide a statistical profile of the collection, including a comparison of the English text with other English corpora. The corpus is too small for full SMT systems. However, it gives far more language pairs than earlier collections. This makes it useful for annotation projection, dictionary extraction, typological comparison and Digital Humanities work. Ukrainian is among the included languages.

Key points

  • The corpus contains 100 languages, giving 4,950 unique language pairs. For comparison, DGT-Acquis has 253 pairs, InterCorp 351 and OPUS about 3,800.

  • Coverage varies: some languages have the full Bible, while others have only the New Testament or selected parts.

  • The English text runs to about 800k words. Alignment is by verse (roughly 30,000 verses), not by sentence.

  • Verse and sentence boundaries often differ. Counting only "." and "?", about 4,000 verses hold several sentences. Treating ";" as sentence-final raises this to about 10,000, and adding ":" makes it more than half of all verses.

  • The authors identify three main limitations:

    • archaic language;
    • meaning-oriented rather than literal translations;
    • no modern named entities.

    All texts are public domain or free for research use.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

127 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 119 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s10579-014-9287-y

@inproceedings{christodouloupoulos2014massively,
  title = {{A massively parallel corpus: the Bible in 100 languages}},
  author = {Christodouloupoulos, Christos and Steedman, Mark J.},
  booktitle = {Language Resources and Evaluation},
  year = {2014},
  doi = {10.1007/s10579-014-9287-y},
  url = {https://doi.org/10.1007/s10579-014-9287-y}
}

References

30 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abney, S., & Bird, S. (2010). The human language project: Building a universal corpus of the world’s languages. In Proceedings of the 48th annual meeting of the association for computational linguistics (pp. 88–97). Uppsala: Association for Computational Linguistics. find
  2. Abney, S., & Bird, S. (2011). Towards a data model for the Universal Corpus. In: Proceedings of the 4th workshop on building and using comparable corpora: Comparable corpora and the web. ACL, pp. 120–127. Allen, J. D., Anderson, D., Becker, J., Cook, R., Davis, M., Edberg, P., Everson, M., Freytag,… find
  3. Unicode Consortium, Mountain View, CA. Brend, R. M., & Pike, K. L. (1977). The summer Institute of linguistics: Its works and contributions. find
  4. The Hague: Mouton. Catholic Church. (2001). Liturgiam authenticam: Fifth instruction on vernacular translation of the Roman liturgy. United States Conference of Catholic Bishops, Washington, DC. Čermák, F., & Rosen, A. (2012). The case of InterCorp, a multilingual parallel corpus. International Jo… find
  5. Cohen, S. B., Das, D., & Smith, N. A. (2011). Unsupervised structure prediction with non-parallel multilingual guidance. In Proceedings of EMNLP, pp. 50–61. find
  6. Das, D., & Petrov, S. (2011). Unsupervised part-of-speech tagging with bilingual graph-based projections. In Proceedings of ACL-HLT, pp. 600–609. find
  7. Dipper, S., & Schultz-Balluff, S. (2013). The anselm corpus: Methods and perspectives of a parallel aligned corpus. In Proceedings of the workshop on computational historical linguistics at NODALIDA, pp. 27–42. Dryer, M. S., & Haspelmath, M. (Eds.). (2013). WALS online. Max Planck Institute for Evol… find
  8. Ehrmann, M., Turchi, M., & Steinberger, R. (2011). Building a multilingual named entity-annotated corpus using annotation projection. In: RANLP, pp 118–124. find
  9. Erjavec, T. (2004). MULTEXT-East version 3: Multilingual morphosyntactic specifications, lexicons and corpora. Proceedings of LREC (pp. 1535–1538). find
  10. Paris: France. Francis, W. N. (1964). A standard sample of present-day English for use with digital computers. Tech. rep., Dept. of Linguistics, Brown University, Providence, RI, USA, report to the US Office of Education on Co-operative Research Project no. E-007. find
  11. Germann, U. (2001). Aligned Hansards of the 36th parliament of Canada. Natural Language Group of the USC Information Sciences Institute. find
  12. Ide, N. (1998). Encoding linguistic corpora. In Proceedings of the sixth workshop on very Large Corpora, pp. 9–17. find
  13. Kanungo, T., Resnik, P., Mao, S., Kim, D., & Zheng, Q. (2005). The Bible and multilingual optical character recognition. Communications of the ACM, 48, 124–130. find
  14. Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. MT summit, 5, 79–86. find
  15. Landauer, T. K., & Littman, M. L. (1991). A statistical method for language-independent representation of the topical content of text segments. In: Proceedings of the Eleventh International Conference: Expert Systems and Their Applications, vol 8, p 85. find
  16. Lewis, W. (2010). Haitian creole: How to build and ship an MT engine from scratch in 4 days, 17 hours, & 30 minutes. In EAMT 2010: Proceedings of the 14th annual conference of the European association for machine translation (pp. 8–13). France: Saint-Raphaël. Lewis, M. P., Simons, G. F., & Fennig,… find
  17. MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk, Volume II: The database. find
  18. Lawrence Erlbaum. Marcus, M., Santorini, B., & Marcinkiewicz, M. A. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19, 331–330. find our card
  19. Neubig, G., Matsubayashi, Y., Hagiwara, M., & Murakami, K. (2011). Safety information mining: What can NLP do in a disaster. In: IJCNLP, pp. 965–973. find
  20. Nida, E. A., & Taber, C. R. (1969). The theory and practice of translation. Helps for Translators, Leiden: EJ Brill. find
  21. Och, F. J., & Ney, H. (2003). A systematic comparison of various statistical alignment models. Computational Linguistics, 29, 19–51. Potthast, M., Barrón-Cedeño, A., Stein, B., & Rosso, P. (2011). Cross-language plagiarism detection. Language Resources and Evaluation, 45(1), 45–62. Proctor, P. (Ed… find
  22. Harlow: Longman Group. Resnik, P., Olsen, M., & Diab, M. (1999). The Bible as a parallel corpus: Annotating the ‘‘Book of 2000 Tongues’’. Computers and the Humanities, 33, 129–153. find
  23. Steinberger, R., Pouliquen, B., Widiger, A., Ignat, C., Erjavec, T., Tufis, D., & Varga, D. (2006). The JRC-Acquis: A multilingual aligned parallel corpus with 20? languages. In Proceedings of the 5th international conference on language resources and evaluation (LREC’2006), Genoa, Italy. find
  24. Steinberger, J., Lenkova, P., Kabadjov, M. A., Steinberger, R., & Van der Goot, E. (2011). Multilingual entity-centered sentiment analysis evaluated by parallel corpora. In: RANLP, pp. 770–775. Steinberger, R., Eisele, A., Klocek, S., Pilos, S., & Schlüter, P. (2013). DGT-TM: A freely available tra… find
  25. Tiedemann, J. (2012). Parallel data, tools and interfaces in opus. In: Chair, N. C. C., Choukri, K., Declerck, T., Doğan, M. U., Maegaard, B., Mariani, J., Moreno, A., Odijk, J., & Piperidis, S. (Eds.), Proceedings of the eight international conference on language resources and evaluation (LREC’12)… find
  26. Turchi, M., Steinberger, J., Kabadjov, M., & Steinberger, R. (2010). Using parallel corpora for multilingual (multi-document) summarisation evaluation. In Multilingual and multimodal information access evaluation (pp. 52–63). find
  27. Berlin: Springer. United Bible Societies (2013) Bible translation. http://www.unitedbiblesocieties.org/sample-page/ bible-translation Vinokourov, A., Cristianini, N., & Shawe-taylor, J. S. (2002). Inferring a semantic representation of text via cross-language correlation analysis. In Advances in neu… ↗
  28. Wei, C. P., Yang, C. C., & Lin, C. M. (2008). A latent semantic indexing-based approach to multilingual document clustering. Decision Support Systems, 45(3), 606–620. find
  29. Wierzbicka, A. (2001). What did Jesus mean? Explaining the Sermon on the Mount and the parables in simple and universal human concepts. Oxford: Oxford University Press on Demand. find
  30. Wikipedia (2013) List of literary works by number of translations. http://en.wikipedia.org/ wiki/List\_of\_literary\_works\_by\_number\_of\_translations Yarowsky, D., & Ngai, G. (2001). Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora. In Proceedings of NAACL… ↗