Elamur Research⌕
☰
home / papers / 2023 / Deep Active Learning for Morphophonological Processing

Deep Active Learning for Morphophonological Processing

Paper · 2023 · Seyed Morteza Mirbostani, Yasaman Boreshban, Salam Khalifa et al.

Summary

The paper tackles the cost of annotating data for morphophonological processing in low-resource settings, using Egyptian Arabic as a test case. The main task is generation: converting an underlying representation (UR) into a surface form (SF). The reverse task, analysis, is also examined. The authors wrap the character-level transformer of Wu et al. (2021) in a pool-based deep active learning (DAL) loop. In each cycle, the trained model scores every unlabeled UR by the highest entropy found at any single character of its predicted SF. The most uncertain candidates are shortlisted, embedded with a small character-level LSTM, reduced by PCA and clustered with k-means. The next batch is then drawn from the clusters in proportion to their size, so that uncertainty and variety are balanced. Run on the Khalifa et al. (2022) dataset, the method matches the full-data state of the art with roughly 30% of the training pool. An error analysis shows which mistakes active learning removes and which remain. The approach is language-independent and could guide smarter dataset building for under-resourced dialects.

Key points

  • Data: the Khalifa et al. (2022) Egyptian Arabic morphophonology set, with URs produced by CALIMA EGY for SFs taken from ECAL.
    • Split sizes: TRAIN 13,170; DEV 5,180 (DEV-OOV 2,189); EVAL 6,974 (EVAL-OOV 2,271).
    • The OOV subsets hold only words that never appear in TRAIN.
  • Loop setup:
    • It starts with 1,400 randomly chosen labeled items: 500 form a fixed tune set and 900 form the initial training set.
    • Each cycle adds δ = 250 new samples.
    • Candidates for clustering are the α = 1,000 highest-entropy words.
  • Uncertainty score: a word's entropy is the maximum of its per-character entropies. The authors report this beat the word-level entropy used by Muradoglu and Hulden (2022).
  • Diversity step:
    • A siamese pair of LSTMs is trained so that true (UR, SF) pairs count as related and maximally Levenshtein-distant pairs count as unrelated.
    • Its embeddings are standardized and reduced to 3 PCA components.
    • k-means is then run, with k chosen through SSE across cluster counts.
  • Results and errors: all experiments were averaged over 5 runs, and about 30% of the data was enough to reach full-data accuracy. Compared with random selection, out of 100 errors:
    • letter deletion fell from 25 to 11;
    • affix deletion fell from 2 to 0;
    • letter change fell from 17 to 11;
    • letter-addition errors persisted or grew, and gold-standard errors became a larger share.

Interesting ideas

  • I1. Scoring a whole output by its single least-confident character, rather than an average, catches items where one local decision is shaky even though the rest looks safe.
  • I2. Shortlisting the most uncertain items first and then clustering them, with sampling proportional to cluster size, keeps each annotation batch from piling onto one confusing pattern.
  • I3. Keeping a small tune set frozen across all cycles while the training set grows gives a stable yardstick for choosing the best model each round.
  • I4. The error analysis separates genuinely learnable mistakes from optional variation, lexically idiosyncratic foreign words and errors in the gold data, so remaining errors are not all blamed on the model.
  • I5. A cheap embedding is learned by training a siamese character encoder on input–output pairs, using maximally edit-distant pairs as negatives, which yields features for clustering word transformations without any extra labels.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when choosing sentences to send to the LLM teacher tiers, rank them by their single most uncertain dependency arc or label rather than by mean parse confidence.
  • I2 (learn): pick silver-data disagreements among witness parsers by uncertainty, then cluster by construction type so each teacher batch covers varied phenomena.
  • I3 (learn): hold a fixed checkpoint-selection set when iteratively retraining the perceptron or gparse parser on growing silver data, so cycles stay comparable.
  • I4 (tokens): tag lemmatizer and morphology failures with similar buckets (irregular or loan forms, true variation, gold mistakes) before writing repair rules.
  • I5 (tokens): cluster unknown word-form/lemma pairs this way to group similar inflection patterns and propose candidate lemma rules for review.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stony Brook UniversityStony Brook · United States43 works · 2,571 citations🎓University of GuilanRasht · Iran2 works · 14 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Seyed Morteza Mirbostani, Yasaman Boreshban, Salam Khalifa, SeyedAbolghasem Mirroshandel and Owen Rambow. 2023. Deep Active Learning for Morphophonological Processing. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-short.69

@inproceedings{mirbostani2023deep,
  title = {{Deep Active Learning for Morphophonological Processing}},
  author = {Seyed Morteza Mirbostani, Yasaman Boreshban, Salam Khalifa, SeyedAbolghasem Mirroshandel and Owen Rambow},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-short.69},
  url = {https://doi.org/10.18653/v1/2023.acl-short.69}
}

References

28 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bashar Alhafni, Nizar Habash, and Houda Bouamor. 2020. Gender-aware reinflection using linguistically enhanced neural models. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 139–150. find
  2. Khuyagbaatar Batsuren, Gábor Bella, Aryaman Arora, Viktor Martinovic, Kyle Gorman, Zdeněk Žabokrtský, Amarsanaa Ganbold, Šárka Dohnalová, Magda Ševčíková, Kateřina Pelegrinová, Fausto Giunchiglia, Ryan Cotterell, and Ekaterina Vylomova. 2022. find
  3. The SIGMORPHON 2022 shared task on morpheme segmentation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 103–116, Seattle, Washington. Association for Computational Linguistics.
  4. Caleb Belth, Sarah Payne, Deniz Beser, Jordan Kodner, and Charles Yang. 2021. The greedy and recursive Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian Arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9.
  5. Nizar Habash, Reham Marzouk, Christian Khairallah, and Salam Khalifa. 2022. Morphotactic modeling in an open-source multi-dialectal Arabic morphological analyzer and generator. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 92–102, Seattle, Washington. Association for Computational Linguistics. ↗
  6. Nizar Habash and Owen Rambow. 2006. MAGEAD: A morphological analyzer and generator for the Arabic dialects. In Proceedings of the International Conference on Computational Linguistics and the Conference of the Association for Computational Linguistics (COLING-ACL), pages 681–688, Sydney, Australia. ↗
  7. Nawar Halabi. 2016. Modern standard Arabic phonetics for speech synthesis. Ph.D. thesis, University of Southampton. find
  8. Bruce Hayes. 2008. Introductory phonology, volume 7. John Wiley & Sons. find
  9. Go Inoue, Salam Khalifa, and Nizar Habash. 2021. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. arXiv preprint arXiv:2110.06852. ↗
  10. Salam Khalifa, Jordan Kodner, and Owen Rambow. 2022. Towards learning Arabic morphophonology. In Proceedings of the seventh Arabic Natural Language Processing Workshop (WANLP) at EMNLP 2022, pages 295–301s.
  11. Salam Khalifa, Nasser Zalmout, and Nizar Habash. 2020. Morphological analysis and disambiguation for Gulf Arabic: The interplay between resources and methods. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 3895–3904.
  12. H Kilany, H Gadalla, H Arram, A Yacoub, A El-Habashi, and C McLemore. 2002. Egyptian colloquial Arabic lexicon. LDC catalog number LDC99L22. find
  13. Christo Kirov and Ryan Cotterell. 2018. Recurrent neural networks in linguistic theory: Revisiting pinker and prince (1988) and the past tense debate. Transactions of the Association for Computational Linguistics, 6:651–665. ↗
  14. Ming Liu, Wray Buntine, and Gholamreza Haffari. 2018. Learning to actively learn neural machine translation. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 334–344. find
  15. Mingyi Liu, Zhiying Tu, Zhongjie Wang, and Xiaofei Xu. 2020. Ltp: A new active learning strategy for bert-crf based named entity recognition. ArXiv, abs/2001.02524. ↗
  16. Silvan Wehrli, Simon Clematide, and Peter Makarov. 2022. Cluzh at sigmorphon 2022 shared tasks on morpheme segmentation and inflection generation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 212–219. find
  17. Shijie Wu, Ryan Cotterell, and Mans Hulden. 2021. Applying the transformer to character-level transduction. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1901–1907, Online. Association for Computational Linguistics. ↗
  18. Changbing Yang, Garrett Nicolai, Miikka Silfverberg, et al. 2022. Generalizing morphological inflection systems to unseen lemmas. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 226–235. find
  19. Ye Zhang, Matthew Lease, and Byron Wallace. 2017. Active discriminative text representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31. find
  20. Yuekai Zhao, Haoran Zhang, Shuchang Zhou, and Zhihua Zhang. 2020. Active learning approaches to enhancing neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1796–1806. ↗
  21. Saliha Muradoglu and Mans Hulden. 2022. Eeny, meeny, miny, moe. how to choose data for morphological inflection. arXiv preprint arXiv:2210.14465. ↗
  22. A Karthik Narasimhan, Regina Barzilay, and Tommi Jaakkola. 2015. An unsupervised method for uncovering morphological chains. Transactions of the Association for Computational Linguistics, 3:157–167. ↗
  23. Álvaro Peris and Francisco Casacuberta. 2018. Active learning for interactive neural machine translation of data streams. arXiv preprint arXiv:1807.11243. ↗
  24. Ameya Prabhu, Charles Dognin, and Maneesh Singh. 2019. Sampling bias in deep active classification: An empirical study. arXiv preprint arXiv:1909.09389. ↗
  25. Dongyu Ru, Jiangtao Feng, Lin Qiu, Hao Zhou, Mingxuan Wang, Weinan Zhang, Yong Yu, and Lei Li. 2020. Active sentence learning by adversarial uncertainty sampling in discrete space. arXiv preprint arXiv:2004.08046. ↗
  26. Rico Sennrich and Barry Haddow. 2016. Linguistic input features improve neural machine translation. arXiv preprint arXiv:1606.02892. ↗
  27. Yanyao Shen, Hyokun Yun, Zachary C Lipton, Yakov Kronrod, and Animashree Anandkumar. 2017. Deep active learning for named entity recognition. arXiv preprint arXiv:1707.05928. ↗
  28. Dima Taji, Jamila El Gizuli, and Nizar Habash. 2018. An Arabic dependency treebank in the travel domain. In Proceedings of the Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT), Miyazaki, Japan. Implementation Details ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mirbostani-2023-deep-active-learning-morphophonological-processing/