Elamur Research⌕
☰
home / papers / 2020 / Towards Augmenting Lexical Resources for Slang and African American English

Towards Augmenting Lexical Resources for Slang and African American English

Paper · 2020 · Alyssa Hwang, William R. Frey and Kathleen McKeown

Summary

Mainstream lexical resources such as WordNet and ConceptNet describe formal Standard American English well. They offer little for slang or African American English (AAE), which change quickly on social media. The authors try to fill this gap by grouping words with similar meanings, so that an unfamiliar term can be understood through the known words around it. They first compare clustering methods on the fiction portion of the Brown Corpus: Brown clustering on bigrams, and agglomerative and k-means clustering over GloVe and Word2Vec vectors. The chosen setup, which the paper mostly reports as GloVe with k-means, is then applied to TwitterAAE and the Gang Violence tweets.

Clusters are scored automatically against WordNet synsets and ConceptNet "relatedto" links, and also by humans. Mechanical Turk workers and a social-work expert often judge low-precision clusters to be coherent. This shows that existing resources are poor gold standards for nonstandard English. The authors also introduce a new human-based metric, the Cluster Split Score.

Key points

  • The motivating gap is large: SlangNet has about 3,000 words against more than 150,000 in WordNet, and no AAE lexical resource existed when the study was done.
  • The data comes from three corpora:
    • the Brown Corpus fiction subsection, used as an interpretable pilot;
    • TwitterAAE from the UMass SLANG Lab, with 830,000 tweets (about 500,000 words);
    • the Gang Violence dataset from Columbia's SAFE Lab, with 5,000 labeled tweets centred on Gakirah Barnes and her contacts.
  • The Cluster Split Score is computed as follows:
    • annotators count how many splits a cluster needs to become coherent;
    • the score is the logistic function of words divided by splits, which keeps it between 0 and 1.
  • In the AMT evaluation, 300 Brown clusters with at least 3 words each were split into three batches. Each batch was judged by three Turkers. Agreement was very low (Fleiss' kappa 0.046, 0.066 and 0.084), yet mean split counts were small.
  • An expert reviewed 14 Gang Violence clusters and gave a mean CSS of 0.96. He reviewed 12 TwitterAAE clusters, splitting every one in two, for a mean CSS of 0.83. ConceptNet precision and CSS did not track each other: many low-precision clusters scored high with humans.

Interesting ideas

  • I1. An automatic score against a reference lexicon can mislead when the lexicon simply lacks the words, so "missing from the resource" should not be counted as "wrong".
  • I2. A new or rare word can get a provisional meaning from the known words that land in its embedding cluster, and the neighbour list itself serves as the explanation.
  • I3. Asking judges only how many pieces a group must be broken into is a cheap, size-aware quality check, and the logistic squashing makes clusters of different sizes comparable.
  • I4. Tune and compare methods first on a clean, standard corpus where outputs are easy to read, then move the winning configuration to the hard dialect data.
  • I5. Crowd judges unfamiliar with a dialect give near-random agreement, so dialect evaluation needs domain experts and a precise definition of relatedness.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): before scoring silver data against WordNet or VerbNet, measure how much of it the resource covers, and report uncovered items as a separate "unknown" bucket rather than as errors.
  • I2 (tokens): an out-of-vocabulary slang form could receive a tentative POS, lemma or WordNet sense from its nearest known neighbours, marked as inferred and showing the neighbours that justify it.
  • I3 (world): this could be used to audit automatically induced verb classes or ontology groupings, where a reviewer just counts the splits needed instead of writing full corrections.
  • I4 (tree): validate a UD dialect conversion or parser change on standard treebank text before applying it to slang-heavy social-media text, so that errors from the method and errors from the dialect stay separable.
  • I5 (learn): when an LLM teacher tier or human review judges dialectal input, route it to a specialised judge and track agreement, treating low-kappa batches as unreliable signals.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States149 works · 48,852 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Alyssa Hwang, William R. Frey and Kathleen McKeown. 2020. Towards Augmenting Lexical Resources for Slang and African American English. Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hwang2020augmenting,
  title = {{Towards Augmenting Lexical Resources for Slang and African American English}},
  author = {Alyssa Hwang, William R. Frey and Kathleen McKeown},
  booktitle = {Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2020},
  url = {https://aclanthology.org/2020.vardial-1.15/}
}

References

25 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Bird, E. Loper, and E. Klein. Natural Language Processing with Python. O’Reilly Media Inc, 2009. find
  2. T. Blevins, R. Kwiatkowski, J. MacBeth, K. McKeown, D. Patton, and O. Rambow. Automatically Processing Tweets from Gang-Involved Youth: Towards Detecting Loss and Aggression. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2196–2206, Osaka, Japan, Dec. 2016. find
  3. The COLING 2016 Organizing Committee. URL https://www.aclweb.org/anthology/C16-1207. ↗
  4. S. L. Blodgett, L. Green, and B. O’Connor. Demographic Dialectal Variation in Social Media: A Case Study of African-American English. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas, Nov. 2016. Association for Computational Linguistics. doi: 10.18653/v1/D16-1120. URL https://www.aclweb.org/ anthology/D16-1120. ↗
  5. P. F. Brown, V. J. Della Pietra, P. V. deSouza, J. C. Lai, and R. L. Mercer. Class-Based n-gram Models of Natural Language. Computational Linguistics, 18(4):467–480, 1992. URL https://www.aclweb.org/anthology/J92-4003. ↗
  6. S. Chang, R. Zhong, E. Adams, F.-T. Lee, S. Varia, D. Patton, W. Frey, C. Kedzie, and K. McKeown. Detecting Gang-Involved Escalation on Social Media Using Context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 46–56, Brussels, Belgium, Oct. 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1005. URL https://www.aclweb.org/anthology/D18-1005. ↗
  7. T. F. Costa Bertaglia and M. d. G. Volpe Nunes. Exploring Word Embeddings for Unsupervised Textual User-Generated Content Normalization. In Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT), pages 112–120, Osaka, Japan, Dec. 2016. ↗
  8. The COLING 2016 Organizing Committee. URL https://www.aclweb.org/anthology/W16-3916. ↗
  9. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. URL https://www.aclweb.org/ anthology/N19-1423. ↗
  10. S. Dhuliawala, D. Kanojia, and P. Bhattacharyya. Slangnet: A wordnet like resource for english slang. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), page 4329–4332. European Language Resources Association (ELRA), May 2016. URL https://www.aclweb.org/anthology/L16-1686. ↗
  11. J. Eisenstein, B. O’Connor, N. A. Smith, and E. P. Xing. Diffusion of Lexical Change in Social Media. PLoS ONE, 9(11):e113114, Nov. 2014. ISSN 1932-6203. doi: 10.1371/journal.pone.0113114. URL http://arxiv.org/abs/1210.5268. arXiv: 1210.5268. ↗
  12. W. N. Francis and H. Kucera. Brown Corpus Manual, 1961. URL http://icame.uib.no/brown/ bcm.html. ↗
  13. R. Goel, S. Soni, N. Goyal, J. Paparrizos, H. Wallach, F. Diaz, and J. Eisenstein. The Social Dynamics of Language Change in Online Networks. arXiv:1609.02075 [physics], Sept. 2016. URL http://arxiv.org/abs/1609.02075. arXiv: 1609.02075. ↗
  14. W. L. Hamilton, K. Clark, J. Leskovec, and D. Jurafsky. Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 595–605, Austin, Texas, 2016. Association for Computational Linguistics. doi: 10.18653/v1/D16-1057. URL http://aclweb.org/anthology/D16-1057. ↗
  15. X. Han and J. Eisenstein. Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4238–4248, Hong Kong, China, Nov. 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1433. URL https://www.aclweb.org/anthology/D19-1433. ↗
  16. M. Meyerhoff. Methods, innovations and extensions: Reflections on half a century of methodology in social dialectology. Journal of Sociolinguistics, 20(4):431–452, Sep 2016. ISSN 1360-6441, 1467-9841. doi: 10.1111/josl.12195. ↗
  17. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781 [cs], Sept. 2013. URL http://arxiv.org/abs/1301.3781. arXiv: 1301.3781. ↗
  18. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12: 2825–2830, 2011. ↗
  19. J. Pennington, R. Socher, and C. Manning. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, 2014. Association for Computational Linguistics. doi: 10.3115/v1/ D14-1162. URL http://aclweb.org/anthology/D14-1162. ↗
  20. W. Roberts and M. Egg. A large automatically-acquired all-words list of multiword expressions scored for compositionality. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018. European Language Resources Association (ELRA). URL https://www.aclweb.org/anthology/L18-1046. ↗
  21. R. Sinha and R. Mihalcea. Unsupervised Graph-based Word Sense Disambiguation Using Measures of Word Semantic Similarity. page 7, 2007. find
  22. R. Speer, J. Chin, and C. Havasi. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. arXiv:1612.03975 [cs], Dec. 2018. URL http://arxiv.org/abs/1612.03975. arXiv: 1612.03975. ↗
  23. I. Stewart and J. Eisenstein. Making ”fetch” happen: The influence of social and linguistic context on nonstandard word growth and decline. arXiv:1709.00345 [physics], Aug. 2018. URL http://arxiv.org/abs/1709.00345. arXiv: 1709.00345. ↗
  24. P. University. About WordNet, 2010. find
  25. C. Whissell. The dictionary of affect in language. 1989. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hwang-2020-augmenting-lexical-resources-slang-african/