Elamur Research⌕
☰
home / papers / 2024 / Leveraging Syntactic Dependencies in Disambiguation: The Case of African American English

Leveraging Syntactic Dependencies in Disambiguation: The Case of African American English

Paper · 2024 · Wilermine Previlon, Alice Rozet, Jotsna Gowda et al.

Summary

African American English (AAE) has a "habitual be": an invariant, non-agreeing be that marks recurring situations, as in "I be in my office by 7:30." It is rare in the data and looks the same as ordinary be on the surface. WordNet has no sense for it, so it cannot be mapped to a lexical resource. The authors frame detection as a binary disambiguation task on transcribed natural speech rather than on social-media text.

First, they parsed 250 hand-labelled sentences from the Joel Buchanan Archive with spaCy and looked for dependency configurations that separate the two readings. They encoded these configurations as Boolean rule features. They added an extended version of an earlier POS-based filter and extra rules found through error analysis. An ensemble of classical machine-learning classifiers was trained on these features and evaluated on a manually annotated CORAAL sample.

The syntactic features raised habitual-class F1 by roughly 65 points over an n-gram baseline, and by up to 74 points. The broader point is that modelling a dialect's own grammar, rather than normalising it away, helps low-resource varieties.

Key points

  • Data split across corpora:
    • Pattern discovery used 250 "be" instances from the Joel Buchanan Archive (132 habitual, 118 non-habitual), parsed with spaCy 3.2.2.
    • Training and testing used a separate collection, CORAAL: 465 habitual vs 3,610 non-habitual for training, and 52 vs 402 for testing.
    • The splits were identical across all experiments.
  • Four dependency patterns:
    • SynPar1 (be tagged as auxiliary with an aux child; 36 cases) points to non-habitual be.
    • SynPar2 (be with an aux sibling; 22 cases) also points to non-habitual be. The auxiliary in both is often a modal.
    • SynPar3 (auxiliary be with a verbal head, e.g. "pronoun be V-ing" or before a passive) points to habitual be.
    • SynPar4 (be tagged as a main verb, typically before non-verbal predicates) also points to habitual be.
  • Extended POS filter: The POS1 filter from Santiago et al. (2022) was extended with transcription spellings: "wanna", "trynna", "gonna", "gotta". The full POS1–POS6 rules and the new ad-hoc patterns are listed in the appendix.
  • Post-hoc rules: Error analysis of a preliminary model produced five additional syntactic rules.
  • Label definition: Invariant be with future, equivalence or emphatic meaning is grouped with habitual be. These uses are structurally identical, so syntax cannot separate them; "non-habitual" covers only agreeing forms.

Interesting ideas

  • I1. Mining a small annotated sample for dependency configurations and turning them into Boolean features gives a classifier whose every decision traces back to a readable syntactic rule.
  • I2. Applying high-precision rules first to remove obvious negatives (e.g. be after a modal or "to") rebalances a very skewed rare-class problem before any learning happens.
  • I3. Rules written for standard spellings silently fail on speech transcripts, so the authors extended them with reduced forms like "gonna" and "wanna".
  • I4. Patterns were discovered on one corpus and evaluated on a different one, which guards against rules that only memorise a single collection.
  • I5. The label scheme merges meanings that syntax cannot distinguish, so the classifier is only asked what structure can actually answer.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Rules like SynPar3/SynPar4 over the UD tree could set a "habitual/recurring" aspect flag on event frames, explainably.
  • I2 (learn): Silver-data pipelines could pre-filter easy negatives with cheap rules, so witness parsers and LLM teachers spend effort only on genuinely ambiguous cases.
  • I3 (tokens): Normalising such contractions in the tokenizer/lemmatizer (gonna → going to) would let downstream tree and frame rules fire on informal text.
  • I4 (learn): Self-induced repair rules could be induced on one treebank and scored on another as a built-in generalisation check.
  • I5 (tree): When converting dialect data to UD, a non-agreeing be tagged VERB, or AUX over a verbal head, could serve as an explicit dialect cue rather than being treated as a tagging error.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of FloridaGainesville · United States33 works · 364 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Wilermine Previlon, Alice Rozet, Jotsna Gowda, Bill Dyer, Kevin Tang and Sarah Moeller. 2024. Leveraging Syntactic Dependencies in Disambiguation: The Case of African American English. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{previlon2024leveraging,
  title = {{Leveraging Syntactic Dependencies in Disambiguation: The Case of African American English}},
  author = {Wilermine Previlon, Alice Rozet, Jotsna Gowda, Bill Dyer, Kevin Tang and Sarah Moeller},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  doi = {10.63317/4w6faq8ec8tz},
  url = {https://doi.org/10.63317/4w6faq8ec8tz}
}

References

58 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Svenja Adolphs, Brian Brown, Ronald Carter, Paul Crawford, Opinder Sahota, Mariya Limerick, Alison Pilnick, Linda Gibson, and Stacy Johnson. 2004. Applying corpus linguistics in a health care context. Journal of Applied Linguistics, 1. ↗
  2. H. Samy Alim. 2004. You know my steez: An ethnographic and sociolinguistic study of styleshifting in a Black American Speech Community. Duke https://credit.niso.org/University Press for the American Dialect Society. ↗
  3. Soumya Barikeri, Anne Lauscher, Ivan Vulic, and Goran Glavas. 2021. Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 1941–1955. Association for Computational Linguistics. find
  4. Jonah Berger and Grant Packard. 2022. Using natural language processing to understand people and culture. American Psychologist, 77(4):525– 537. ↗
  5. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly, Beijing. find
  6. Su Lin Blodgett, Lisa Green, and Brendan T. O’Connor. 2016. Demographic dialectal variation in social media: A case study of african-american english. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1119– 1130. The Association for Computational Linguistics. at Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency parsing for African-American and mainstream American English. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics.
  7. Peter Broadwell, Nicole Davis, and Sunmoo Yoon. 2022. Using Artificial Intelligence to Develop a Lexicon-Based African American Tweet Detection Algorithm to Inform Culturally Sensitive Twitter-Based Social Support Interventions for African American Dementia Caregivers. In Studies in Health Technology and Informatics. IOS Press. ↗
  8. Lu Cheng, Nayoung Kim, and Huan Liu. 2022. Debiasing word embeddings with nonlinear geometry. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 1286–1298. International Committee on Computational Linguistics. ↗
  9. Jamell Dacon. 2022. Towards a deep multi-layered dialectal language analysis: A case study of African-American English. Proceedings of the Second Workshop on Bridging Human– Computer Interaction and Natural Language Processing.
  10. Jamell Dacon, Haochen Liu, and Jiliang Tang. 2022. Evaluating and Mitigating Inherent Linguistic Bias of African American English through Inference. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1442–1454, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  11. Alexis Davis, Joshua L. Martin, Eric Cooks, Melissa J. Vilaro, Danyell Wilson-Howard, Kevin Tang, and Janice Krieger. 2024. From English to “Englishes”: A process perspective on enhancing the linguistic responsiveness of culturally tailored cancer prevention interventions. JMIR Preprints. ↗
  12. Nicholas Deas, Jessica Grieser, Shana Kleiner, Desmond Patton, Elsbeth Turcan, and Kathleen McKeown. 2023. Evaluation of African American language bias in natural language generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6805–6824, Singapore. Association for Computational Linguistics. find
  13. Ralph W. Fasold. 1969. Tense and the Form Be in Black English. Language, 45(4):763–776. Publisher: Linguistic Society of America. ↗
  14. Ralph W. Fasold. 1972. Tense marking in Black English: A linguistic and social analysis. Center for Applied Linguistics. find
  15. Aparna Garimella, Rada Mihalcea, and Akhash Amarnath. 2022. Demographic-aware language model fine-tuning as a bias mitigation technique. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2022 - Volume 2: Short Papers, Online only, November 20-23, 2022, pages 311–319. Association for Computational Linguistics. find
  16. Lisa J. Green. 2002. African American English: A Linguistic Introduction. Cambridge University Press. ↗
  17. Alysia Nicole Harris. 2019. The Non-Aspectual Meaning of African American English ‘Aspectual’ Markers. Ph.D. thesis, Yale University. find
  18. Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, and Diyi Yang. 2022. Exploring the role of grammar and word choice in bias toward African American English (AAE) in hate speech classification. In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 789–798, New York, NY, USA. Association for Computing Machinery. find
  19. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  20. Anne H Charity Hudley, Christine Mallinson, and Mary Bucholtz. 2022. Talking college: Making space for Black language practices in Higher Education. Teachers College Press. find
  21. Alyssa Hwang, William R. Frey, and Kathleen McKeown. 2020. Towards augmenting lexical resources for slang and African American English. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 160–172, Barcelona, Spain (Online). International Committee on Computational Linguistics (ICCL).
  22. Zhiying Jiang, Matthew Yang, Mikhail Tsirlin, Raphael Tang, Yiqin Dai, and Jimmy Lin. 2023. “low-resource” text classification: A parameter-free classification method with compressors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6810–6828, Toronto, Canada. Association for Computational Linguistics. find
  23. Taylor Jones. 2015. Toward a Description of African American Vernacular English Dialect Regions Using “Black Twitter”’. American Speech, 90(4):403–440. find
  24. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. Learning a POS tagger for aave-like language. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 1115–1120. The Association for Computational Linguistics.
  25. Tyler Kendall and Charlie Farrington. 2021. The Corpus of Regional African American Language. Publisher: Online Resources for African American Language Project. ↗
  26. Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, pages 43–53. Association for Computational Linguistics. ↗
  27. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684– 7689. ↗
  28. Bernd Kortmann. 2020. Syntactic Variation in English, chapter 16. John Wiley Sons, Ltd. find
  29. Bernd Kortmann, Kerstin Lunkenheimer, and Katharina Ehret, editors. 2020. eWAVE. find
  30. Anna Koufakou, Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2020. HurtBERT: Incorporating lexical features with BERT for the detection of abusive language. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 34–43, Online. Association for Computational Linguistics. find
  31. Savannah Larimore, Ian Kennedy, Breon Haskett, and Alina Arseniev-Koehler. 2021. Reconsidering annotator disagreement about racist language: Noise or signal? In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, SocialNLP@NAACL 2021, Online, June 10, 2021, pages 81–90. Association for Computational Linguistics. ↗
  32. Donghee N. Lee, Myiah J. Hutchens, Thomas J. George, Danyell Wilson-Howard, Eric J. Cooks, and Janice L. Krieger. 2022. Do they speak like me? exploring how perceptions of linguistic difference may influence patient perceptions of healthcare providers. Medical Education Online, 27(1):2107470. PMID: 35912473. ↗
  33. Ernesto William De Luca and Oliver Streiter. 2003. Example-based NLP for Minority Languages: Tasks, Resources and Tools. find
  34. Gary Marcus. 2018. Deep learning: A critical appraisal. CoRR, abs/1801.00631. ↗
  35. Antonis Maronikolakis, Philip Baader, and Hinrich Schütze. 2022. Analyzing hate speech data along racial, gender and intersectional axes. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 1–7, Seattle, Washington. Association for Computational Linguistics. find
  36. Joshua L. Martin. 2022. Automatic Speech Recognition Systems, Spoken Corpora, and African American Language: An Examination of Linguistic Bias and Morphosyntactic Features. Ph.D. thesis, University of Florida. find
  37. Joshua L. Martin and Kevin Tang. 2020. Understanding Racial Disparities in Automatic Speech Recognition: The Case of Habitual “be”. In Proc. Interspeech 2020, pages 626–630. find
  38. Tessa Masis, Chloe Eggleston, Lisa J Green, Taylor Jones, Meghan Armstrong, and Brendan O’Connor. 2023. Investigating morphosyntactic variation in African American English on Twitter. Proceedings of the Society for Computation in Linguistics, 6(1):392–393. ↗
  39. Tessa Masis, Anissa Neal, Lisa Green, and Brendan O’Connor. 2022. Corpus-guided contrast sets for morphosyntactic feature detection in low-resource English varieties. In Proceedings of the first workshop on NLP applications to field linguistics, pages 11–25, Gyeongju, Republic of Korea. International Conference on Computational Linguistics. Sarah Moeller, Alexis Davis, Wilermine Previlon, Michael Bottini, and Kevin Tang. to appear. Compiling spoken and transcribed corpus of African American language from oral histories. ↗
  40. Marcyliena Morgan. 1994. Theories and politics in African American English. Annual Review of Anthropology, 23:325–345. ↗
  41. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Demonstrations, pages 48–53. Association for Computational Linguistics. ↗
  42. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  43. Princeton University. 2010. “About WordNet.”, WordNet. https://wordnet.princeton.edu/. Accessed: 2024-03-25. ↗
  44. Katja Roller. 2015. Towards the ‘oral’ in oral history: using historical narratives in linguistics. Oral History, 43(1):73–84. find
  45. Samuel Proctor Oral History Program. Curriculum. Accessed: 2023-10-20. AALGS Harrison Santiago, Joshua Martin, Sarah Moeller, Disambiguation of and Kevin Tang. 2022. morpho-syntactic features of African American English – the case of habitual be. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 70–75, Dublin, Ireland. Association for Computational Linguistics. find
  46. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1668–1678. Association for Computational Linguistics.
  47. Andrew Silva, Pradyumna Tambwekar, and Matthew Gombolay. 2021. Towards a comprehensive understanding and accurate evaluation of societal biases in pre-trained transformers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2383–2389, Online. Association for Computational Linguistics. find
  48. Ian Stewart. 2014. Now we stronger than ever: African-American English syntax in Twitter. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, April 26-30, 2014, Gothenburg, Sweden, pages 31–37. The Association for Computer Linguistics.
  49. M. Stone. 1974. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society: Series B (Methodological), 36(2):111–133. ↗
  50. Anna F. Syrquin. 2006. Registers in the academic writing of African American college students. Written Communication, 23(1):63–90. ↗
  51. Samson Tan, Shafiq Joty, Lav Varshney, and Min-Yen Kan. 2020. Mind your inflections! Improving NLP for non-standard Englishes with Base-Inflection Encoding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5647– 5663, Online. Association for Computational Linguistics. find
  52. Textual Optics Lab and History of Black Writing. History of Black Writing Novel Corpus. Accessed: 2023-10-20. Joel Buchanan University of Florida. 2023. archive. find
  53. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, pages 6000–6010, Long Beach, California, USA. Curran Associates Inc.
  54. Tracey L. Weldon. 2021. Middle-Class African American English. Cambridge University Press. ↗
  55. Erin Wolfe. 2019. Natural language processing in the humanities: A case study in automated metadata enhancement. Code4Lib Journal, (46). find
  56. Shijie Wu, Ryan Cotterell, and Mans Hulden. 2021. Applying the transformer to character-level transduction. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 - 23, 2021, pages 1901– 1907. Association for Computational Linguistics. ↗
  57. Sunmoo Yoon, Peter Broadwell, Frederick F. Sun, Maria De Planell-Saguer, and Nicole Davis. 2023. Application of Topic Modeling on Artificial Intelligence Studies as a Foundation to Develop Ethical Guidelines in African American Dementia Caregiving. In Studies in Health Technology and Informatics. IOS Press. ↗
  58. Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. Multi-VALUE: A framework for cross-dialectal English NLP. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Toronto, Canada. Association for Computational Linguistics. 15. Appendices 15.1. Rules to filter non-habitual “be” • POS1: If the word immediately preceding “be” is a modal, adjective, or “to”.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/previlon-2024-leveraging-syntactic-dependencies-disambiguation-case/