Elamur Research⌕
☰
home / papers / 2020 / Adapting Language Specific Components of Cross-Media Analysis Frameworks to Less-Resourced Languages: the Case of Amharic

Adapting Language Specific Components of Cross-Media Analysis Frameworks to Less-Resourced Languages: the Case of Amharic

Paper · 2020 · Yonas Woldemariam and Adam Dahlgren

Summary

The paper looks at how to make an open-source cross-media analysis framework (CMAF), specifically the MICO platform, usable for Amharic. Amharic has more than 22 million native speakers and a fast-growing body of online audio and video, but that media is mostly unindexed and cannot be found by keyword search. Because MICO's language-dependent extractors existed mainly for English, the authors built an Amharic ASR-NER chain that follows MICO's own design principles and standards. The chain has three parts:

  • Kaldi-based acoustic models, for which several acoustic modeling approaches were compared
  • a transliterator that converts the fidel script output of ASR into a form the NER component can consume
  • an OpenNLP-based named entity extractor

Together these turn spoken Amharic content into entity metadata that can serve as an index. The authors present this as a first step toward fuller MICO support for Amharic. They also argue that the same recipe should carry over to related Semitic languages, such as Tigrinya (shared script) and Arabic (similar phonology).

Key points

  • The target platform is MICO (Media in Context), with EUMSSI discussed as a comparable framework. For NER, MICO provides OpenNLP models covering English, German, Italian and Spanish, and models for further languages can be added through its API.
  • MICO chose Kaldi for ASR after comparing open-source and proprietary engines on sample videos, judging mainly on accuracy and multilingual support.
  • MICO's speech-to-text chain runs in four stages: audio demultiplexing and downsampling, speaker diarization with gender classification, transcription, and then text analytics such as NER.
  • Amharic is written in fidel, a consonant-vowel syllabary shared with Tigrinya. Vowels are often left implicit in writing, which complicates processing.
  • The related resources cited include the ALFFA speech corpus, the ELRA-W0074 Amharic-English bilingual corpus, and HaBiT web text data. The authors note that most earlier Amharic tools were small proof-of-concept prototypes.

Interesting ideas

  • I1. The transliterator is a dedicated adapter between two components that expect different scripts, so neither component has to be retrained to match the other.
  • I2. MICO picked its core engine only after benchmarking candidate engines on samples of the actual target content, rather than relying on published scores.
  • I3. The paper stresses that downstream extractors like NER depend heavily on upstream ASR quality, so upstream errors flow into every later stage.
  • I4. In EUMSSI, an auxiliary component normalizes raw transcription before NER runs on it. This acknowledges that noisy upstream text needs cleanup before structured analysis.
  • I5. Entity mentions extracted from speech are fused with face detection, so that two independent evidence channels point to the same real-world person.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A separate, inspectable normalization step could handle orthographic variants (quotes, ligatures, British vs. American spelling) before the tagger, keeping the tagger simpler and its errors easier to trace.
  • I2 (learn): Candidate parsers or witness models could be compared on in-domain samples such as fables and school texts before being allowed to vote on silver data.
  • I3 (learn): Each stage could be evaluated both on gold input and on the actual output of the previous stage, to separate a stage's own errors from inherited ones.
  • I4 (tokens): A similar cleanup pass for messy input (OCR, chat text) could sit ahead of tokenization and log every change it makes, so the explanation chain stays complete.
  • I5 (world): Entity resolution could combine separate evidence sources, such as coreference chains and knowledge-graph matches, while recording which source supported each link.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Umeå UniversityUmeå · Sweden4 works · 43 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Semitic languages · 2,908 worksSpeech & spoken language · 480 worksLow-resource languages · 267 works

Cite

Cite as Yonas Woldemariam and Adam Dahlgren. 2020. Adapting Language Specific Components of Cross-Media Analysis Frameworks to Less-Resourced Languages: the Case of Amharic. Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{woldemariam2020adapting,
  title = {{Adapting Language Specific Components of Cross-Media Analysis Frameworks to Less-Resourced Languages: the Case of Amharic}},
  author = {Yonas Woldemariam and Adam Dahlgren},
  booktitle = {Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL)},
  year = {2020},
  url = {https://aclanthology.org/2020.sltu-1.42/}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abate, S., Tachbelie, M., and Menze, W. (2009). Amharic speech recognition: Past, present and future. In Proceedings of the 16th International Conference of Ethiopian Studies, pages 1391–1401. find
  2. Aichroth, P., Weigel, C., Kurz, T., Stadler, H., Drewes, F., Björklund, J., Schlegel, K., Berndl, E., Perez, A., Bowyer, A., and Volpini, A. (2015). Mico-media in context. In Proceedings of 2015 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pages 1–4. find
  3. Belay, M. (2014). Amharic Named Entity Recognition Using a Hybrid Approach. Master’s thesis, School of Information Informatics, Addis Ababa University. find
  4. Berger, A. L., Pietra, S. D., and Pietra, V. J. D. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22:39–71. ↗
  5. Besacier, L., Barnard, E., Karpov, A., and Schultz, T. (2014). Automatic speech recognition for under-resourced languages. Speech Communication, 56:85– 100. find
  6. Chang, S.-F., Manmatha, R., and Chua, T.-S. (2005). Combining text and audio-visible features in video indexing. In Acoustics, Speech, and Signal Processing, pages 1005–1008. find
  7. Demeke, Y. and Hailemariam, S. (2012). Duration modeling of phonemes for amharic text to speech system. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems, pages 1–7. find
  8. Feng, S. and Lee, T. (2018). Improving cross-lingual knowledge transferability using multilingual tdnn-blstm with language-dependent pre-final layer. In Proceedings of Interspeech, pages 2439–2443. find
  9. Finkel, J. R., Grenager, T., and Manning, C. (2005). Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 363–370.
  10. Gambäck, B. and Sikdar, U. K. (2017). Named entity recognition for amharic using deep learning. In 2017 IST-Africa Week Conference (IST-Africa), pages 1–8. find
  11. Ghahremani, P., Manohar, V., Hadian, H., Povey, D., and Khudanpur, S. (2017). Investigation of transfer learning for asr using lf-mmi trained neural networks. In Proceedings of 2017 IEEE Automatic Speech Recognition and Understanding Workshop, pages 279–286. find
  12. Ghoshal, A., Swietojanski, P., and Renals, S. (2013). Multilingual training of deep neural networks. In Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7319–7323. find
  13. Hori, T. and Nakamura, A. (2006). An extremely large vocabulary approach to named entity extraction from speech. In 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, pages 973–976. find
  14. Huang, J.-T., Li, J., Yu, D., Deng, L., and Gong, Y. (2013). Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. In Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7304–7308. find
  15. Ivanov, A. V., Ramanarayanan, V., Suendermann-Oeft, D., Lopez, M., Evanini, K., and Tau, J. (2015). Automated speech recognition technology for dialogue interaction with non-native interlocutors. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 134–138. find
  16. Kazama, J. and Torisawa, K. (2007). A new perceptron algorithm for sequence labeling with non-local features. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 315–324. find
  17. Kurz, T., Schlegel, K., and Kosch, H. (2015). Enabling access to linked media with sparql-mm. In Proceedings of the 24th International Conference on World Wide Web. find
  18. Le, N., Bredin, H., Sargent, G., India, M., Lopez-Otero, P., Barras, C., Guinaudeau, C., Gravier, G., da Fonseca, G. B., Freire, I. L., do Patrocínio, Z. K. G., Guimarães, S. J. F., Martí, G., Morros, J. R., Hernando, J., Fernández, L. D., García-Mateo, C., Meignier, S., and Odobez, J.-M. (2017). Towards large scale multimedia indexing: A case study on person discovery in broadcast news. In Proceedings of International Workshop on Content-Based Multimedia Retrieval, pages 1–6. find
  19. Magnini, B., Cutugno, F., Falcone, M., and Pianta, E. (2013). Evaluation of natural language and speech tools for italian. In Lecture Notes in Computer Science, pages 98–106. find
  20. Marneffe, M.-C., MacCartney, B., and Manning, C. (2006). Generating typed dependency parses from phrase structure parses. In InProc. 5th Interna-tional Conference on Language Resources and Evaluation (LREC 2006), pages 449–454.
  21. Mehamed, M. (2010). Amharic Named Entity Recognition. Master’s thesis, College of Natural Sciences, Addis Ababa University. find
  22. Meignier, S. and Merlin, T. (2010). Lium spkdiarization: An open source toolkit for diarization. In CMU SPUD Workshop. find
  23. Povey, D., Burget, L., Agarwal, M., Akyazi, P., Feng, K., Ghoshal, A., Glembek, O., Goel, N. K., Karafiát, M., Rastrow, A., Rose, R. C., Schwarz, P., and Thomas, S. (2011a). The subspace gaussian mixture model - a structured model for speech recognition. Computer Speech and Language, 25(2):404–439. ↗
  24. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., and Vesely, K. (2011b). The Kaldi speech recognition toolkit. In Proceedings of IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. find
  25. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286. ↗
  26. Sebsibe, H., Prahallad, K., Alan, B., Rohit, K., and Rajeev, S. (2004). Unit selection voice for amharic using festvox. In Fifth ISCA Workshop on Speech Synthesis, pages 103–107. find
  27. Sobhana, N., Mitra, P., and Ghosh, S. (2010). Conditional random field based named entity recognition in geological text. International Journal of Computer Applications, 1(3):143–147. ↗
  28. Stüker, S., Kilgour, K., and Niehues, J. (2011). Quaero speech-to-text and text translation evaluation systems. In High Performance Computing in Science and Engineering’10, pages 529–542. Springer. ↗
  29. Tachbelie, M., Abate, S., and Besacier, L. (2014). Using different acoustic, lexical and language modeling units for asr of an under-resourced language - amharic. Speech Communication, 56:181–194. ↗
  30. Tranter, S. and Reynolds, D. A. (2006). An overview of automatic speaker diarization systems. IEEE Transactions on Audio, Speech, and Language Processing, 14:1557– 1565. ↗
  31. Wang, D. and Zheng, T. F. (2015). Transfer learning for speech and language processing. In Proceedings of 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pages 1225–1237. find
  32. Xiong, W., Wu, L., Alleva, F., Droppo, J., Huang, X., and Stolcke, A. (2018). The microsoft 2017 conversational speech recognition system. In Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5934–5938. find
  33. Yifiru, M. (2003). Automatic Amharic Speech Recognition System to Command and Control Computers. Master’s thesis, School of Information Studies for Africa, Addis Ababa University. 9. Language Resource References ELRA-W0074. (2014). Amharic-English bilingual corpus, distributed via ELRA,1.0. distributed via ELRA,1.0, 1.0, ISLRN 590-255-335-719-0. find
  34. Elodie Gauthier and Laurent Besacier and Sylvie Voisin and Michael Melese and Uriel Pascal Elingui. (2016). ALFFA (African Languages in the Field: speech Fundamentals and Automation). European Language Resources Association (ELRA), Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), ISLRN SLR25. find
  35. HaBiT. (2016). Harvesting big text data for under-resourced languages. distributed via Natural Language Processing Centre, Faculty of Informatics, Masaryk University. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/woldemariam-2020-adapting-language-specific-components-cross/