Elamur Research⌕
☰
home / papers / 2020 / Offensive Language Identification in Greek

Offensive Language Identification in Greek

Summary

Most resources for detecting abusive online content exist for English, and Greek had no annotated dataset for this task. The authors built the Offensive Greek Tweet Dataset (OGTD), the first such resource for Greek. They gathered tweets from May to June 2019, a period that included Greek and European elections. Searches targeted trending TV and politics hashtags, common Greek expletives and the second-person phrase "είσαι" ("you are"). After cleaning and deduplication, they randomly sampled 5,000 tweets. Three volunteers labelled them in LightTag as offensive, not offensive or spam, and the final set has 4,779 labelled posts. The labelling scheme follows the top level of the OLID scheme, so OGTD is comparable with English and other OLID-style corpora. The authors normalized Greek accents and tested several models: classical classifiers with TF-IDF, POS and dependency features, deep networks using Greek word embeddings, and multilingual BERT. The best model, LSTMs and GRU with attention, reached an F1 of 0.89. The resource supports offensive language research for Greek and cross-lingual work.

Key points

  • Collection and cleaning: 49,154 tweets were collected, leaving 46,218 after duplicate removal. Of these, 5,000 were sampled for annotation, and OGTD v1.0 ended with 4,779 tweets.
  • Label distribution: 1,401 tweets are offensive (about 29%) and 3,378 are not offensive. The split is 3,345 tweets for training and 1,434 for testing.
  • Preprocessing: URLs, emojis and emoticons were removed, and user mentions were replaced with @USER, as in OLID. Repeated punctuation was normalized.
  • Annotation:
    • Labels with full agreement were accepted, and otherwise the majority label (above 66%) was used.
    • When all annotators disagreed, an author and two extra judges resolved the tweet.
    • Agreement was reported as pairwise Cohen's kappa.
  • Models and features:
    • Features included TF-IDF unigrams and bigrams, POS tags and dependency relation tags.
    • The deep models used 300-dimensional Greek word embeddings (Outsios et al., 2018).
    • Multilingual BERT cased was used because no Greek BERT existed at the time.
    • The best result was 0.89 F1, from LSTM+GRU with attention.

Interesting ideas

  • I1. Query by insult structure, not only swear words: The authors searched for a frequent pronoun-plus-verb phrase ("you are") because direct insults often have the form "be + noun/adjective." This recovered targeted insults that a list of expletives would miss. The same trick could seed offensive or toxic data collection in Ukrainian or other languages with similar constructions.
  • I2. Time collection to high-conflict events: Scraping during elections and around popular TV shows raised the share of complaints and profanity. This helps when the target class is rare in random samples.
  • I3. Add a "Spam" label during annotation: A third label let annotators flag junk instead of forcing it into one of two classes. This keeps the final binary labels cleaner and is a cheap addition to any crowd or volunteer labelling setup.
  • I4. Normalize diacritics before modelling: Users type Greek accents inconsistently. Lower-casing and stripping accents removed spurious variation between spellings of the same word. This fits any language where informal writing drops diacritics or mixes spelling conventions.
  • I5. Reuse an established annotation scheme: Following OLID's first-level definition makes the new corpus directly comparable with English, Danish and Turkish datasets. That enables cross-lingual and multilingual experiments without extra relabelling.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

92 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 89 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{pitenis2020offensive,
  title = {{Offensive Language Identification in Greek}},
  author = {Pitenis, Zesis and Zampieri, Marcos and Ranasinghe, Tharindu},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.629/}
}

References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aragón, M. E., Carmona, M. Á. Á., y Gómez, M. M., Escalante, H. J., Pineda, L. V., and Moctezuma, D. (2018). Overview of MEX-A3T at IberLEF 2019: Authorship and Aggressiveness Analysis in Mexican Spanish Tweets. In Proceedings of IberLEF. find
  2. Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F. M., Rosso, P., and Sanguinetti, M. (2019). SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of SemEval. find
  3. Bosco, C., Dell’Orletta, F., Poletto, F., Sanguinetti, M., and Tesconi, M. (2018). Overview of the EVALITA 2018 Hate Speech Detection Task. In Proceedings of EVALITA. find
  4. Burnap, P. and Williams, M. L. (2015). Cyber hate speech on twitter: An application of machine classification and statistical modeling for policy and decision making. Policy & Internet, 7(2):223–242. find
  5. Çöltekin, C. (2020). A Corpus of Turkish Offensive Language on Social Media. In Proceedings of LREC. find our card
  6. Chen, Y., Zhou, Y., Zhu, S., and Xu, H. (2012). Detecting offensive language in social media to protect adolescent online safety. In Proceedings of SocialCom. find
  7. Davidson, T., Warmsley, D., Macy, M. W., and Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Proceedings of ICWSM. find our card
  8. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL. find
  9. Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., and Bhamidipati, N. (2015). Hate speech detection with comment embeddings. In Proceedings of WWW. find
  10. Hettiarachchi, H. and Ranasinghe, T. (2019). Emoji Powered Capsule Network to Detect Type and Target of Offensive Posts in Social Media. In Proceedings of RANLP. find
  11. Kumar, R., Ojha, A. K., Malmasi, S., and Zampieri, M. (2018). Benchmarking Aggression Identification in Social Media. In Proceedings of TRAC. find
  12. Malmasi, S. and Zampieri, M. (2017). Detecting Hate Speech in Social Media. In Proceedings of RANLP. find
  13. Malmasi, S. and Zampieri, M. (2018). Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30(2):187–202. find
  14. Mehdad, Y. and Tetreault, J. (2016). Do Characters Abuse More Than Words? In Proceedings of SigDial. find
  15. Mubarak, H., Darwish, K., and Magdy, W. (2017). Abusive Language Detection on Arabic Social Media. In Proceedings of ALW. find
  16. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016). Abusive Language Detection in Online User Content. In Proceedings of WWW. find
  17. Outsios, S., Skianis, K., Meladianos, P., Xypolopoulos, C., and Vazirgiannis, M. (2018). Word Embeddings from Large-Scale Greek Web Content. ArXiv, abs/1810.06694. — [arXiv:1810.06694](https://arxiv.org/abs/1810.06694) find
  18. Pelosi, S., Maisto, A., Vitale, P., and Vietri, S. (2017). Mining Offensive Language on Social Media. In Proceedings of CLiC-it. find
  19. Plum, A., Ranashinghe, T., Orasan, C., and Mitkov, R. (2019). RGCL at GermEval 2019: Offensive Language Detection with Deep Learning. In Proceedings of KON-VENS. find
  20. Ranasinghe, T., Zampieri, M., and Hettiarachchi, H. (2019). BRUMS at HASOC 2019: Deep Learning Models for Multilingual Hate Speech and Offensive Language Identification. In Proceedings of HASOC. find
  21. Razavi, A. H., Inkpen, D., Uritsky, S., and Matwin, S. (2010). Offensive language detection using multi-level classification. In Atefeh Farzindar et al., editors, Advances in Artificial Intelligence, pages 16–27. Springer Berlin Heidelberg. find
  22. Ross, B., Rist, M., Carbonell, G., Cabrera, B., Kurowsky, N., and Wojatzki, M. (2016). Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis. In Proceedings of NLP4CMC. find
  23. Sigurbergsson, G. I. and Derczynski, L. (2020). Offensive Language and Hate Speech Detection for Danish. In Proceedings of LREC. find our card
  24. Tulkens, S., Hilte, L., Lodewyckx, E., Verhoeven, B., and Daelemans, W. (2016). A Dictionary-based Approach to Racism Detection in Dutch Social Media. In Proceedings of TA-COS. find
  25. Van Hee, C., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., De Pauw, G., Daelemans, W., and Hoste, V. (2015). Automatic detection and prevention of cyberbullying. In Proceedings of HUSO. find
  26. Waseem, Z., Davidson, T., Warmsley, D., and Weber, I. (2017). Understanding abuse: A typology of abusive language detection subtasks. In Proceedings of ICWSM. find our card
  27. Wiegand, M., Siegel, M., and Ruppenhofer, J. (2018). Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language. In Proceedings of GermEval. find
  28. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019a). Predicting the Type and Target of Offensive Posts in Social Media. In Proceedings of NAACL. find our card
  29. Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., and Kumar, R. (2019b). SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of SemEval. find our card
  30. Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., and Çöltekin, C. (2020). SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/pitenis-2020-offensive-language-identification-greek/