Elamur Research⌕
☰
home / papers / 2012 / Analyse des performances de modèles de langage sub-lexicale pour des langues peu-dotées à morphologie riche (Performance analysis of sub-word language modeling for under-resourced languages with rich morphology: case study on Swahili and Amharic) [in French]

Analyse des performances de modèles de langage sub-lexicale pour des langues peu-dotées à morphologie riche (Performance analysis of sub-word language modeling for under-resourced languages with rich morphology: case study on Swahili and Amharic) [in French]

Paper · 2012 · Hadrien Gelas, Solomon Teferra Abate, Laurent Besacier et al.

Summary

The paper looks at speech recognition (ASR) for two African languages that have few resources and complex morphology: Amharic, which is inflectional with root-pattern morphology, and Swahili, which is agglutinative. Because words in these languages take so many forms, word-based n-gram language models leave many test words out of vocabulary (OOV), and this raises word error rate. The authors replace whole words with smaller units in the n-gram LM and compare four kinds: rule-based syllables, supervised morphemes (an FSM segmenter for Amharic, TreeTagger for Swahili), and unsupervised morphs from Morfessor. They also compare several ways of marking unit boundaries so that words can be rebuilt from the decoder's output. Morfessor units gave the best results for both languages. The gain was large for Amharic, whose LM text is tiny, and smaller for Swahili, which has much more text. With a suitable reconstruction scheme, up to 75% of Amharic OOV words were recovered. The study shows that unsupervised segmentation that needs no language knowledge can work well for under-resourced languages.

Key points

  • Amharic setup: 20 h of read speech from 100 speakers (10,850 sentences) and only 2.3M words of LM text. The test set is 1.5 h (758 sentences).
  • Swahili setup: about 12 h of audio (2.5 h read by 5 speakers plus roughly 10 h of web broadcast news of mixed quality) and 28M words of text from 12 news sites. The test set is 2 h of broadcast news (1,997 sentences).
  • All LMs were built with SRILM using a 65k lexicon. For Swahili, the 65k-word baseline covered only 13.95% of the full text vocabulary. Syllables covered the full vocabulary but produced very short units.
  • Three boundary-tagging schemes were tested, each with its own rule for rebuilding words:
    • UNIT_AFX: a marker on affixes only.
    • UNIT_ALL: markers on both sides, so one root can have up to four variants.
    • UNIT_POS: syllables tagged with their position in the word.
  • Morfessor, used with default settings, beat all supervised options. The authors attribute this to its balance between n-gram span and lexical coverage. Up to 75% of Amharic OOV words were recovered.

Interesting ideas

  • I1. The way segment boundaries are encoded in the text (affix-only markers, markers on both sides, or positional indices) changes vocabulary size, coverage and accuracy, so it should be treated as a design choice that gets tested.
  • I2. An unsupervised MDL segmenter can beat linguistically built analysers because it balances how long units are against how many words they cover.
  • I3. The authors evaluate OOV recovery as its own metric, separate from overall error, and hand-check rebuilt outputs for non-words such as stray affixes or illegal joins.
  • I4. Testing the same method on two opposite resource profiles (strong audio with little text, and the reverse) shows which bottleneck the method actually fixes.
  • I5. Turning sub-units back into words with simple, deterministic joining rules keeps the pipeline transparent.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): When splitting clitics or rare compounds in the en tokenizer, compare boundary-marking conventions and require that each convention rebuilds the original surface form without loss.
  • I2 (tokens): A Morfessor-style fallback could split unknown English words (rare derivations, neologisms) into parts for lemma and POS guessing, which is better than giving up on an opaque OOV.
  • I3 (learn): Add a dedicated OOV slice and a check for malformed reconstructions to evaluation, so gains on rare words are measured explicitly and do not get lost in averages.
  • I4 (learn): Compare silver-data or repair-rule techniques across domains with deliberately different amounts of training data, to learn where each technique really helps.
  • I5 (back): Generation from the meaning graph can emit morpheme-level pieces with explicit join markers and assemble surface words with traceable rules, so every inflected form can be explained.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 5Research subjects and languages the work is about.Language models & pretraining · 3,894 worksSemitic languages · 3,066 worksMorphology · 420 worksLow-resource languages · 287 worksFrench · 83 works

Cite

Cite as Hadrien Gelas, Solomon Teferra Abate, Laurent Besacier and François Pellegrino. 2012. Analyse des performances de modèles de langage sub-lexicale pour des langues peu-dotées à morphologie riche (Performance analysis of sub-word language modeling for under-resourced languages with rich morphology: case study on Swahili and Amharic) [in French]. JEP-TALN-RECITAL 2012, Workshop TALAf 2012: Traitement Automatique des Langues Africaines (TALAf 2012: African Language Processing).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gelas2012analyse,
  title = {{Analyse des performances de modèles de langage sub-lexicale pour des langues peu-dotées à morphologie riche (Performance analysis of sub-word language modeling for under-resourced languages with rich morphology: case study on Swahili and Amharic) [in French]}},
  author = {Hadrien Gelas, Solomon Teferra Abate, Laurent Besacier and François Pellegrino},
  booktitle = {JEP-TALN-RECITAL 2012, Workshop TALAf 2012: Traitement Automatique des Langues Africaines (TALAf 2012: African Language Processing)},
  year = {2012},
  url = {https://aclanthology.org/W12-1305/}
}

References

25 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A BATE , S., M ENZEL , W. et T AFILA , B. (2005). An Amharic speech corpus for large vocabulary continuous speech recognition. In Interspeech, pages 67–76. find
  2. A RISOY , E., C AN , D., P ARLAK , S., S AK , H. et S ARAÇLAR , M. (2009). Turkish broadcast news transcription and retrieval. Audio, Speech, and Language Processing, IEEE Transactions on, 17(5):874–883. find
  3. A RISOY , E. et S ARAÇLAR , M. (2009). Lattice extension and vocabulary adaptation for Turkish LVCSR. Audio, Speech, and Language Processing, IEEE Transactions on, 17(1):163–173. find
  4. B ARNARD , E., D AVEL , M. et van H UYSSTEEN , G. (2010a). Speech technology for information access : a South African case study. In AAAI Symposium on Artificial Intelligence, pages 22–24. find
  5. B ARNARD , E., S CHALKWYK , J., van H EERDEN , C. et M ORENO , P. (2010b). Voice search for development. In Interspeech. find
  6. C REUTZ , M. et L AGUS , K. (2005). Unsupervised morpheme segmentation and morphology induction from text corpora using morfessor 1.0. Rapport technique, Computer and Information Science, Report A81, Helsinki University of Technology. find
  7. D IEHL , F., G ALES , M., T OMALIN , M. et W OODLAND , P. (2011). Morphological decomposition in Arabic ASR systems. Computer Speech & Language. find
  8. G ELAS , H., B ESACIER , L. et P ELLEGRINO , F. (2012). Developments of swahili resources for an automatic speech recognition system. In SLTU. find
  9. G UIJARRUBIA , V., T ORRES , M. et J USTO , R. (2009). Morpheme-based automatic speech recognition of basque. Pattern Recognition and Image Analysis, pages 386–393. find
  10. H IRSIMAKI , T., C REUTZ , M., S IIVOLA , V., K URIMO , M., V IRPIOJA , S. et P YLKKONEN , J. (2006). Unlimited vocabulary speech recognition with morph language models applied to Finnish. Computer Speech & Language. find
  11. H IRSIMAKI , T., P YLKKONEN , J. et K URIMO , M. (2009). Importance of high-order n-gram models in morph-based speech recognition. Audio, Speech, and Language Processing, IEEE Transactions on, 17(4):724–732. find
  12. K UMAR , A., T EWARI , A., H ORRIGAN , S., K AM , M., M ETZE , F. et C ANNY , J. (2011). Rethinking speech recognition on mobile devices. In IUI4DR. ACM. find
  13. K URIMO , M., C REUTZ , M., V ARJOKALLIO , M., A RISOY , E. et S ARACLAR , M. (2006). Unsupervised segmentation of words into morphemes–morpho challenge 2005, application to automatic speech recognition. In Interspeech. find
  14. L EI , X., S IU , M., H WANG , M., O STENDORF , M. et L EE , T. (2006). Improved tone modeling for Mandarin broadcast news speech recognition. In Interspeech. find
  15. M ARTEN , L. (2006). Swahili. In B ROWN , K., éditeur : The Encyclopedia of Languages and Linguistics, 2nd ed., volume 12, pages 304–308. Oxford : Elsevier. find
  16. M OHRI , M., P EREIRA , F. et R ILEY , M. (1998). A rational design for a weighted finite-state transducer library. In Lecture Notes in Computer Science, pages 144–158. Springer. find
  17. P ATEL , N., C HITTAMURU , D., J AIN , A., D AVE , P. et P ARIKH , T. (2010). Avaaj otalo : a field study of an interactive voice forum for small farmers in rural India. In CHI, pages 733–742. ACM. find
  18. P ELLEGRINI , T. et L AMEL , L. (2009). Automatic word decompounding for ASR in a morphologically rich language : Application to Amharic. Audio, Speech, and Language Processing, IEEE Transactions on, 17(5):863–873. find
  19. S ARIKAYA , R., K IRCHHOFF , K., S CHULTZ , T. et H AKKANI -T UR , D. (2009). Introduction to the special issue on processing morphologically rich languages. Audio, Speech, and Language Processing, IEEE Transactions on, 17(5). find
  20. S ENG , S., S AM , S., B ESACIER , L., B IGI , B. et C ASTELLI , E. (2008). First broadcast news transcription system for Khmer language. In LREC. find
  21. S HAIK , M., M OUSA , A., S CHLUTER , R. et N EY , H. (2011a). Hybrid language models using mixed types of sub-lexical units for open vocabulary German LVCSR. In Interspeech. find
  22. S HAIK , M., M OUSA , A., S CHLUTER , R. et N EY , H. (2011b). Using morpheme and syllable based sub-words for Polish LVCSR. In ICASSP. find
  23. T ACHBELIE , M., A BATE , S. et B ESACIER , L. (2011a). Part-of-speech tagging for under-resourced and morphologically rich languages - the case of Amharic. In HLTD. find
  24. T ACHBELIE , M., A BATE , S. et M ENZEL , W. (2010). Morpheme-based automatic speech recognition for a morphologically rich language - amharic. In SLTU. find
  25. T ACHBELIE , M., A BATE , S. et M ENZEL , W. (2011b). Morpheme-based and factored language modeling for Amharic speech recognition. In V ETULANI , Z., éditeur : Human Language Technology. Challenges for Computer Science and Linguistics, volume 6562 de Lecture Notes in Computer Science, pages 82–93. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/gelas-2012-analyse-des-performances-de-modeles/