Elamur Research⌕
☰
home / papers / 2018 / Hurtlex: A Multilingual Lexicon of Words to Hurt

Hurtlex: A Multilingual Lexicon of Words to Hurt

Summary

Lexicons for hate speech detection are often compiled ad hoc for a single task. They rarely rest on linguistic expertise and tend to cover only explicit slurs and swear words. This paper turns Tullio De Mauro's Italian list "Le parole per ferire" into a machine-readable, multilingual resource called HurtLex. The original list has over a thousand words in 17 categories, including stereotype-laden and context-dependent terms. The authors deduplicated the entries, added part-of-speech tags through MultiWordNet (filling gaps by hand), and attached BabelNet definitions. They then used BabelNet senses to obtain translations. Before translating, experts graded each lemma–sense pair by how offensive it is, so that only hurtful meanings were carried into other languages. The result is a strict and a broader lexicon, each covering 53 languages. The evaluation had two parts. First, the authors analysed word and category frequencies in an Italian Twitter corpus about immigrants. Second, HurtLex features fed a misogyny classifier that ranked first at AMI@IberEval 2018. The resource gives a reusable base for abusive-language detection, especially for languages with few such lexicons.

Key points

  • Building the Italian base took several steps:
    • De Mauro's list yielded 1,138 items, of which 1,082 were unique.
    • Removing 10 lemmas that only occur inside idioms left 1,072 lemmas.
    • Splitting lemmas by part of speech raised the count to 1,156 entries.
  • MultiWordNet supplied POS tags for 59.2% of lemmas, and BabelNet supplied definitions for 71.1%. Everything is distributed as a single XML file that also encodes the category hierarchy.
  • Sense annotation went through two rounds:
    • Pilot: two experts labeled 200 senses and agreed on 87.6% of them.
    • Refined scheme: after splitting the Neutral class, a second pair of annotators agreed on 61%. Most disagreements involved "not offensive vs. not literally pejorative" and "negative connotation vs. offensive".
  • In the immigration hate corpus of 6,008 tweets (Sanguinetti et al.), the crime-related category RE appeared in 45.10% of messages and the QAS category in 23.32%.
  • At AMI@IberEval 2018, the Unito system used HurtLex category features and won sub-task 1 in English and Spanish, plus sub-task 2 in Spanish. The most useful categories were prostitution, male and female genitalia, and physical and cognitive disability.

Interesting ideas

  • I1. Filter senses before translating. Annotating which senses of a word are offensive, and only then mapping through a multilingual sense network, keeps irrelevant meanings out of the translations. Homograph named entities are a typical example of what gets removed. The same trick could help when bootstrapping any domain lexicon, such as Ukrainian sentiment or toxicity lists, from another language.
  • I2. Release two versions with different coverage. Publishing a conservative and an inclusive version lets users trade precision for recall for their own task. This avoids forcing a single fixed threshold on what counts as offensive.
  • I3. Use graded offensiveness labels. The scheme separates crystallized insults, metaphorical shifts, and words that are merely negative. This makes ambiguity explicit and also reveals where annotators naturally disagree.
  • I4. Track word frequency across annotation levels. Plotting how often a word appears at each level of aggression, offensiveness or irony shows distinct behaviours. Some words fade as hostility rises, swear words grow, and some neutral-looking words follow a peak-shaped curve. This could serve as a cheap diagnostic of how a lexicon entry actually behaves in a corpus.
  • I5. Match on lemma and POS together. Matching lexicon entries as (lemma, POS) pairs against Universal Dependencies annotations reduces false hits from homographs. It transfers directly to any language that has a UD-parsed corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

87 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 84 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{bassignana2018hurtlex,
  title = {{Hurtlex: A Multilingual Lexicon of Words to Hurt}},
  author = {Bassignana, Elisa and Basile, Valerio and Patti, Viviana},
  booktitle = {Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018)},
  year = {2018},
  doi = {10.4000/books.aaccademia.3085},
  url = {https://doi.org/10.4000/books.aaccademia.3085}
}

References

13 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Claudia Bianchi. 2014. The speech acts account of derogatory epithets: some critical notes. In J. Du-tant, D. Fassio, and Meylan A., editors, Liber Amicorum Pascal Engel, University of Geneva, pages pp. 465–480. find
  2. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In International AAAI Conference on Web and Social Media. find our card
  3. Tullio De Mauro. 2016. Le parole per ferire. Internazionale. 27 settembre 2016. Compiled for the “Joe Cox” Committee on intolerance, xenophobia, racism and hate phenomena, of the Italian Chamber of Deputies, which issued a Final Report in 2017. find
  4. Francesca D’Errico, Marinella Paciello, and Matteo Amadei. 2018. Prosocial words in social media discussions on hosting immigrants. insights for psychological and computational field. In Symposium on Emotion Modelling and Detection in Social Media and Online Interaction, In conjunction with the 2018… find
  5. Christiane Fellbaum. 1998. WordNet: an electronic lexical database. MIT Press. find
  6. Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile, and Viviana Patti. 2018. 14-ExLab@UniTo for AMI at IberEval2018: Exploiting Lexical Knowledge for Detecting Misogyny in English and Spanish Tweets. In Proc. of 3rd Workshop on Evaluation of Human Language Technologies for Iberian… find
  7. Serena Pelosi, Alessandro Maisto, Pierluigi Vitale, and Simonetta Vietri. 2017. Mining offensive language on social media. In Proceedings of the Fourth Italian Conference on Computational Linguistics (CLiC-it 2017), Rome, Italy, December 11-13, 2017. find
  8. Emanuele Pianta, Luisa Bentivogli, and Christian Girardi. 2002. Multiwordnet: developing an aligned multilingual database. In Proceedings of the First International Conference on Global WordNet, January. find
  9. Manuela Sanguinetti, Cristina Bosco, Alberto Lavelli, Alessandro Mazzei, Oronzo Antonelli, and Fabio Tamburini. 2018a. PoSTWITA-UD: an Italian Twitter Treebank in Universal Dependencies. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazak… find
  10. Manuela Sanguinetti, Fabio Poletto, Cristina Bosco, Viviana Patti, and Marco Stranisci. 2018b. An italian Twitter corpus of hate speech against immigrants. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France, may. European Language… find our card
  11. Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10. Association for Computational Linguistics. find our card
  12. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93. ACL. find our card
  13. Michael Wiegand, Josef Ruppenhofer, Anna Schmidt, and Clayton Greenberg. 2018. Inducing a lexicon of abusive words – a feature-based approach. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bassignana-2018-hurtlex-multilingual-lexicon-words-hurt/