Elamur Research⌕
☰
home / papers / 2017 / Learning to Explain Non-Standard English Words and Phrases

Learning to Explain Non-Standard English Words and Phrases

Paper · 2017 · Ke Ni and William Yang Wang

Summary

The paper addresses how to automatically explain slang and other non-standard English expressions that a fixed slang dictionary has not yet recorded. Earlier systems looked words up in small lists of about 5K terms, so coverage was poor and the lists went stale quickly. The authors instead treat explanation as generation. They built a corpus from Urban Dictionary (UD; here this means the slang website, not Universal Dependencies): 421K crowdsourced entries from 1999–2014, each with definitions and usage examples. On this corpus they trained an attentive LSTM sequence-to-sequence model that writes a definition for a marked expression in a sentence. Its key design is a dual encoder:

  • a word-level LSTM reads the surrounding sentence;
  • a character-level LSTM reads only the target expression;
  • the two hidden states are combined linearly before decoding.

Test expressions never appear in training. On this split, the large dual encoder reached the best BLEU, ahead of a single word-level encoder and a fully character-level model. This shows a learned, open-vocabulary slang explainer is feasible, and the dataset was released.

Key points

  • Corpus: 421K Urban Dictionary entries (1999–2014), released publicly. Training uses 371,028 entries (907,624 example–definition pairs). Testing uses 50,000 entries (61,330 sentences). Training and test share no target expressions.

  • Architecture:

    • word-level context encoder plus character-level target encoder, merged as h₁W₁ + h₂W₂ + B;
    • LSTM decoder with attention following Vinyals et al. (2015);
    • implemented in Tensorflow, trained with SGD on cross-entropy.

  • BLEU-1 / BLEU-2 results:

    ModelHidden unitsBLEU-1BLEU-2
    single encoder102421.062.1
    small dual encoder51221.842.2
    large dual encoder102424.582.37
    full char-level model25621.131.8

    The char-level model was capped at 256 units by GPU memory.

  • Other findings: LSTM cells beat GRUs, and attention helped.

  • Qualitative examples: blends such as "loltastic" and "adoucheous" got more sensible glosses from the dual model. The authors attribute this partly to the model knowing which span it must explain.

Interesting ideas

  • I1. Encoding the span to be explained separately (characters) from its sentence context (words) tells the model which of several odd tokens is the target, and the character view handles blends and misspellings.
  • I2. A crowdsourced dictionary whose entries carry example sentences doubles as a parallel corpus of (usage in context → definition) pairs, with vote-based moderation as a light quality filter.
  • I3. Splitting train/test by dictionary entry rather than by sentence guarantees the system is judged on genuinely unseen expressions, not memorised ones.
  • I4. The very low BLEU-2 (about 2) next to BLEU-1 (about 21–25) suggests n-gram overlap is a weak judge of free-text definitions. The authors therefore backed their numbers with inspected examples.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): an unknown-token module could pair a character-level view of the OOV form with its sentence context to propose a lemma, POS and standard-English counterpart, rather than guessing from context alone.
  • I2 (world): Urban Dictionary entries could feed a flagged slang lexicon in the compiled knowledge layer, kept as low-trust facts that the reader cites when it uses them.
  • I3 (learn): evaluations of lexical components should hold out whole lexemes, so that apparent gains on new words cannot come from leakage of seen forms.
  • I4 (answer): when Mova explains a word, grading should check meaning, for example by parsing the gloss and comparing its frames to the reference, rather than relying on surface overlap.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Ke Ni and William Yang Wang. 2017. Learning to Explain Non-Standard English Words and Phrases. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ni2017learning,
  title = {{Learning to Explain Non-Standard English Words and Phrases}},
  author = {Ke Ni and William Yang Wang},
  booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2017},
  url = {https://aclanthology.org/I17-2070/}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, pages 311–318.
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 . ↗
  3. Clint Burfoot and Timothy Baldwin. 2009. Automatic satire detection: Are you having a laugh? In Proceedings of the ACL-IJCNLP 2009 conference short papers. Association for Computational Linguistics, pages 161–164. find
  4. Sara Rosenthal and Kathleen McKeown. 2011. Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, pages 763–772. ↗
  5. Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder–decoder approaches. Syntax, Semantics and Structure in Statistical Translation page 103.
  6. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. pages 3104–3112. ↗
  7. Johnny Davis. 2011. In praise of urban dictionaries. The Guardian. find
  8. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in Neural Information Processing Systems. pages 2773–2781. ↗
  9. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2010. A latent variable model for geographic lexical variation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 1277–1287.
  10. William Yang Wang and Kathleen McKeown. 2010. ”got you!”: Automatic vandalism detection in wikipedia with web-based shallow syntactic-semantic modeling. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010). ↗
  11. Stephan Gouws, Donald Metzler, Congxing Cai, and Eduard Hovy. 2011. Contextual bearing on linguistic variation in social media. In Proceedings of the Workshop on Languages in Social Media. Association for Computational Linguistics, pages 20–29. find
  12. Michele Zappavigna. 2012. Discourse of Twitter and social media: How we use language to create affiliation on the web. A&C Black. find
  13. Bo Han and Timothy Baldwin. 2011. Lexical normalisation of short text messages: Makn sens a# twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, pages 368–378.
  14. Leo Hickman. 2013. Why IBM’s Watson supercomputer can’t speak slang. The Guardian. find
  15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. ↗
  16. Joe Jackson. 2011. Feds Check Urban Dictionary to Crack Gun-Store Death Threat. TIME. find
  17. Leslie Kaufman. 2013. For the Word on the Street, Courts Call Up an Online Witness. The New York Times. find
  18. Leah Kolt and Matt Lazier. 2009. Alumni in the News: Summer & Fall 2009. Cal Poly Magazine. find
  19. JoAnn Lloyd. 2011. Alum Aaron Peckham’s Urban Dictionary Redefines Language. Cal Poly Magazine. find
  20. Thanapon Noraset, Liang Chen, Larry Birnbaum, and Doug Downey. 2016. Definition modeling: Learning to define word embeddings in natural language. In AAAI. pages 3259–3266. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ni-2017-learning-explain-non-standard-english/