Elamur Research⌕
☰
home / papers / 2024 / One-Vs-Rest Neural Network English Grapheme Segmentation: A Linguistic Perspective

One-Vs-Rest Neural Network English Grapheme Segmentation: A Linguistic Perspective

Paper · 2024 · Samuel Rose, Nina Dethlefs and C. Kambhampati

Summary

Most grapheme-to-phoneme (G2P) systems treat each letter as a grapheme. Every letter is force-aligned either to a phoneme or to an empty placeholder. The authors argue this conflicts with linguistics, where a grapheme is the minimal written unit standing for a phoneme and can span up to four letters. They adopt Kohrt's Referential Conception and Brooks' (2019) inventory of British English graphemes. Using that inventory, they hand-annotated 9,641 common words, all also present in CMUDict, and release the result as the EngGraph corpus.

Their proof of concept has two stages. First, ten small binary networks (One-vs-Rest) estimate how many graphemes a word contains, using only word length, vowel count and consonant count. Second, a greedy longest-match procedure with backtracking splits the word into exactly that many valid graphemes, falling back to the next most confident count if no split works. The pipeline reaches a 31.86% WER against the CMUDict benchmark. That is behind, but in range of, LSTM and attention G2P systems. The main contribution is segmentations that are linguistically meaningful, for example trigraphs and quadgraphs like "dge" or "ough".

Key points

  • Brooks (2019) lists 284 British English graphemes (89 "main system", 195 "extended system") mapping to 43 phonemes, including units of three and four letters and split digraphs like "a.e".
  • EngGraph contains 9,641 words drawn from the 10,000 most Google-indexed British English words. Each word has its grapheme transcription, grapheme count, length, and vowel/consonant counts, and the corpus is released on GitHub.
  • The model is ten identical feedforward nets, one per grapheme count from 1 to 10. Each takes 3 inputs and has two dense layers of 128 units with 30% dropout. Training used Adam (lr 0.001), batch size 8, up to 150 epochs, and early stopping with patience 20.
  • Per-class accuracy ranges from about 0.78 for counts 5–6 to 0.95 for count 1. A single multi-class network failed, and an earlier Fuzzy Inference System scored only 50.18% exactly but 95.51% within ±1.
  • The end-to-end segmentation plus transcription pipeline scores 31.86% WER. For comparison, cited LSTM and attention baselines fall roughly between 20% and 26%.

Interesting ideas

  • I1. Predict a structural quantity first (how many units), then search only among segmentations with that many pieces, retrying with the next-ranked count on failure; this turns an open search into a constrained one.
  • I2. Segment by trying the longest inventory unit first, then recursively backtrack when the remainder cannot be covered, so every output is built from a closed, linguistically justified list.
  • I3. Replace one hard multi-class problem with per-class binary models, each trained on a balanced set of positives versus an equal random sample of all other classes, which isolates what distinguishes each class.
  • I4. Report accuracy both exactly and within a ±1 tolerance; a large gap reveals that errors come from overlapping neighbouring classes rather than random failure.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Morphological splitting in crate en could first guess the number of morphemes, then enumerate only splits of that size, with the ranked fallback keeping the choice explainable.
  • I2 (tokens): The tokenizer could handle contractions, clitics and multiword expressions with a curated inventory plus backtracking, avoiding opaque learned subword pieces.
  • I3 (learn): Silver-data classifiers for skewed labels could be trained one-vs-rest on balanced slices, giving per-label diagnostics that show where witness parsers disagree.
  • I4 (learn): Ordinal outputs such as counts in maths word problems or event-order positions could be scored with tolerance bands to separate near-misses from real reasoning faults.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of HullHull · United Kingdom3 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Samuel Rose, Nina Dethlefs and C. Kambhampati. 2024. One-Vs-Rest Neural Network English Grapheme Segmentation: A Linguistic Perspective. Proceedings of the 28th Conference on Computational Natural Language Learning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.conll-1.36

@inproceedings{rose2024one,
  title = {{One-Vs-Rest Neural Network English Grapheme Segmentation: A Linguistic Perspective}},
  author = {Samuel Rose, Nina Dethlefs and C. Kambhampati},
  booktitle = {Proceedings of the 28th Conference on Computational Natural Language Learning},
  year = {2024},
  doi = {10.18653/v1/2024.conll-1.36},
  url = {https://doi.org/10.18653/v1/2024.conll-1.36}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammad Jahid Ibna Basher, Mohammad Raghib Noor, Sadia Afroze, Ikbal Ahmed, and Mohammed Moshiul Hoque. 2023. BnGraphemizer: A Grapheme-based Tokenizer for Bengali Handwritten Text Recognition. In 2023 IEEE 9th International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE), pages 183–188. ISSN: 2837-8245. find
  2. Greg Brooks. 2019. Dictionary of the British English Spelling System. Open Book Publishers. Accepted: 2021-02-11T11:23:40Z. find
  3. Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. Long Short-Term Memory-Networks for Machine Reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 551–561. Association for Computational Linguistics.
  4. Lu Dong, Zhi-Qiang Guo, Chao-Hong Tan, Ya-Jun Hu, Yuan Jiang, and Zhen-Hua Ling. 2022. Neural Grapheme-To-Phoneme Conversion with Pre-Trained Grapheme Models. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6202–6206. ISSN: 2379-190X. ↗
  5. Heting Gao, Mark Hasegawa-Johnson, and Chang D. Yoo. 2024. G2PU: Grapheme-To-Phoneme Transducer with Speech Units. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10061– 10065. ISSN: 2379-190X. find
  6. Manfred Kohrt. 1986. THE TERM ‘GRAPHEME’ IN THE HISTORY AND THEORY OF LINGUISTICS. In THE TERM ‘GRAPHEME’ IN THE HISTORY AND THEORY OF LINGUISTICS, pages 80–96. De Gruyter. ↗
  7. Duc Le, Thilo Koehler, Christian Fuegen, and Michael L. Seltzer. 2020. G2G: TTS-Driven Pronunciation Learning for Graphemic Hybrid ASR. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6869–6873. ISSN: 2379-190X. ↗
  8. David G. Lockwood. 2000. Phoneme and grapheme: how parallel can they be? LACUS Forum, 27:307– 317. Publisher: Linguistic Association of Canada and the United States. find
  9. Amr El-Desoky Mousa and Björn Schuller. 2016. Deep Bidirectional Long Short-Term Memory Recurrent Neural Networks for Grapheme-to-Phoneme Conversion Utilizing Complex Many-to-Many Alignments. In Interspeech 2016, pages 2836–2840. ISCA. find
  10. Oxford Dictionaries. 2023. Oxford English Corpus -Facts About the Language. find
  11. Kanishka Rao, Fuchun Peng, Haşim Sak, and Françoise Beaufays. 2015. Grapheme-to-phoneme conversion using Long Short-Term Memory recurrent neural networks. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4225–4229. ISSN: 2379-190X. find
  12. Gailius Raškinis, Gintarė Paškauskaitė, Aušra Saudargienė, Asta Kazlauskienė, and Airenas Vaičiūnas. 2019. Comparison of Phonemic and Graphemic Word to Sub-Word Unit Mappings for Lithuanian Phone-Level Speech Transcription. Informatica, 30(3):573–593. Publisher: Vilnius University Institute of Mathematics and Informatics. ↗
  13. Samuel Rose and Chandrasekhar Kambhampati. 2024. Classifying Graphemes in English Words Through the Application of a Fuzzy Inference System. ArXiv:2404.01953 [cs]. ↗
  14. Jason Taylor. 2022. Pronunciation modelling in end-to-end text-to-speech synthesis. Ph.D. thesis, University of Edinburgh. Accepted: 2022-06-13T13:54:29Z Publisher: The University of Edinburgh. ↗
  15. Shubham Toshniwal and Karen Livescu. 2016. Jointly learning to align and convert graphemes to phonemes with neural attention models. In 2016 IEEE Spoken Language Technology Workshop (SLT), pages 76–82. find
  16. Sonjia Waxmonsky and Sravana Reddy. 2012. G2P Conversion of Proper Names Using Word Origin Information. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 367–371, Montréal, Canada. Association for Computational Linguistics. find
  17. Samantha Williams, Paul Foulkes, and Vincent Hughes. 2024. Analysis of forced aligner performance on L2 English speech. Speech Communication, 158:103042. ↗
  18. WorldlyWisdom. 2021. Google 10000 English. find
  19. Kaisheng Yao and Geoffrey Zweig. 2015. Sequence-to-Sequence Neural Net Models for Grapheme-to-Phoneme Conversion. ArXiv:1506.00196 [cs]. ↗
  20. Zelin Ying, Chen Li, Yu Dong, Qiuqiang Kong, Qiao Tian, Yuanyuan Huo, and Yuxuan Wang. 2024. A Unified Front-End Framework for English Text-to-Speech Synthesis. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10181–10185. ISSN: 2379-190X.
  21. Markéta Řezáčková, Jan Švec, and Daniel Tihelka. 2021. T5G2P: Using Text-to-Text Transfer Transformer for Grapheme-to-Phoneme Conversion. International Speech Communication Association. Accepted: 2022-03-28T10:00:27Z ISSN: 2308-457X. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rose-2024-one-vs-rest-neural-network/