Elamur Research⌕
☰
home / papers / 2018 / Aggression-annotated Corpus of Hindi-English Code-mixed Data

Aggression-annotated Corpus of Hindi-English Code-mixed Data

Summary

Hostile online behaviour such as trolling, cyberbullying, flaming and hate speech has grown too fast for manual moderation. Existing research usually treats each phenomenon separately and offers little theoretical grounding, and almost none of it addresses Indian social media. The authors argue that these behaviours overlap heavily and can be handled within a single framework of verbal aggression. They propose a hierarchical typology. At the top level, aggression is split by how it is expressed: overt, covert, or none. Finer categories describe the target, such as physical threat, sexual aggression, identity-based aggression (gendered, geographical, political, casteist, communal, racial) and non-threatening aggression. Using this scheme, they annotated Hindi-English code-mixed posts from Twitter and public Facebook pages, producing roughly 18k tweets and 21k Facebook comments for public release. Early classification experiments on the three top-level classes reach an F1 of only about 0.70, which shows how hard even coarse aggression detection is. The resource matters because it is presented as the first dataset labelled for both level and kind of aggression, and it targets code-mixed text.

Key points

  • The tagset is hierarchical, with 3 top-level tags (Overtly Aggressive, Covertly Aggressive, Non-aggressive) and 10 second-level tags.
  • The corpus contains about 18k tweets and about 21k Facebook comments in Hindi, English and Hindi-English code-mixed language.
  • Facebook data was crawled from more than 40 public pages, including news outlets (NDTV, ABP News, Zee News), web portals, political parties (INC, BJP), student groups (SFI, JNUSU, AISA) and campus-incident groups. Twitter data was gathered through hashtags on divisive topics such as the beef ban, India vs. Pakistan cricket, elections and films.
  • Corpus observations:
    • Facebook users tend to be more overtly aggressive, while Twitter users lean toward covert aggression.
    • Political aggression dominates on both platforms.
    • Code-mixed posts are mostly aggressive, Hindi posts are evenly split, and English posts are mostly non-aggressive.
  • First baseline results reach a top-level F1 of only about 0.70. The annotation was funded by Microsoft Research India.

Interesting ideas

  • I1. One umbrella category instead of many overlapping ones. The authors model trolling, bullying, flaming and hate speech as forms of a single aggression phenomenon rather than building a separate detector for each. This avoids label conflicts and could simplify toxicity work in other languages, including Ukrainian.
  • I2. Two independent axes: form and target. Separating how aggression is expressed (overt vs. covert) from whom or what it targets lets one corpus support both coarse moderation and fine-grained analysis. For example, it could track political versus ethnic hostility separately.
  • I3. Explicit covert class. Sarcasm, rhetorical questions and insincere politeness get their own label. This forces annotators and models to deal with indirect hostility that keyword lists miss, which is useful wherever irony is a common weapon.
  • I4. Topic-driven collection. Crawling divisive hashtags and partisan pages concentrates the rare aggressive cases efficiently. A similar strategy could cheaply bootstrap datasets for low-resource languages.
  • I5. Cross-tabulating labels with language mix and platform. Comparing aggression rates across code-mixed, monolingual and platform subsets reveals sociolinguistic patterns. This kind of cheap analysis is worth running on any multilingual social-media corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Treebanks

Cited by

68 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 67 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kumar2018aggression,
  title = {{Aggression-annotated Corpus of Hindi-English Code-mixed Data}},
  author = {Kumar, Ritesh and Reganti, Aishwarya N. and Bhatia, Akshit and Maheshwari, Tushar},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/2otsapvt2qu9},
  url = {https://doi.org/10.63317/2otsapvt2qu9}
}

References

17 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Cambria, Erik, Praphul Chandra, Avinash Sharma, and Amir Hussain. (2010). Do not feel the trolls. ISWC, Shanghai. find
  2. Chen, Ying, Yilu Zhou, Sencun Zhu, and Heng Xu. (2012). Detecting offensive language in social media to protect adolescent online safety. Privacy, Security, Risk and Trust (PASSAT), International Conference on Social Computing (SocialCom). pp. 71–80 find
  3. Cynthia Van Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart Desmet, Guy De Pauw, Walter Daelemans, and Véronique Hoste. (2015). Detection and fine-grained classification of cyberbullying events. In Proceedings of International Conference Recent Advances in Natural Language Processing (RANLP), pp… find
  4. Culpeper, Jonathan. (2011). Impoliteness Using Language to cause Offense. Cambridge: Cambridge University Press. find
  5. Dadvar, Maral, Dolf Trieschnigg, and Franciska de Jong. (2014). Experts and machines against bullies: a hybrid approach to detect cyberbullies. In Advances in Artificial Intelligence, pp. 275–281. find
  6. Berlin: Springer Hardaker, Claire. (2010). Trolling in Asynchronous Computer-Mediated Communication: From User Discussions to Academic Definitions. Journal of Politeness Research. Language, Behaviour, Culture, 6 (2): 215–242. find
  7. Hardaker, Claire. (2013). “Uh. . . . not to be nitpicky,,,,,but...the past tense of drag is dragged, not drug.” An overview of trolling strategies. Journal of Language Aggression and Conflict, 1(1): 58–86. find
  8. Karthik Dinakar, Birago Jones, Catherine Havasi, Henry Lieberman, and Rosalind Picard. (2012). Common sense reasoning for detection, prevention, and mitigation of cyberbullying. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(3):18. find
  9. Krol, E. (1992). The whole internet: User's guide & catalog. Sebastopol, CA: O'Reilly & Associates, Inc. find
  10. Kumar, Sudhakar, Francesca Spezzano, and VS Subrahmanian. (2014). Accurately detecting trolls in slashdot zoo via decluttering. In Proceedings of IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). pp. 188–195. find
  11. Mojica, Luis G. (2016). Modeling Trolling in Social Media Conversations. arXiv:1612.05310 [cs.CL]. https://arxiv.org/pdf/1612.05310.pdf [Retrieved on Feb 16, 2017] ↗
  12. Mihaylov, Todor, Georgi D Georgiev, AD Ontotext, and Preslav Nakov. (2015). Finding opinion manipulation trolls in news community forums. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, CoNLL, pp. 310–314. find
  13. Nitin, Ankush Bansal, Siddhartha Mahadev Sharma, Kapil Kumar, Anuj Aggarwal, Sheenu Goyal, Kanika Choudhary, Kunal Chawla, Kunal Jain, Manav Bhasinar. (2012). Classification of Flames in Computer Mediated Communications. arXiv:1202.0617 [cs.SI]. https://arxiv.org/pdf/1202.0617.pdf [Retrieved on Feb… — [arXiv:1202.0617](https://arxiv.org/abs/1202.0617) ↗
  14. Nitta, Taisei, Fumito Masui, Michal Ptaszynski, Yasutomo Kimura, Rafal Rzepka, and Kenji Araki. (2013). Detecting cyberbullying entries on informal school websites based on category relevance maximization. In IJCNLP, pp. 579–586. find
  15. Nobata, Chikashi, Joel Tetreault, Achint Thomas, Yashar Mehdad and Yi Chang. (2016). Abusive Language Detection in Online User Content. In Proceedings of WWW2016. pp. 145 – 153. find
  16. Sax, Sasha. (2016). Flame Wars: Automatic Insult Detection. Stanford Reports. http://cs224d.stanford.edu/reports/Sax.pdf [Retrieved on Feb 20, 2017] ↗
  17. Schultz, Jeff. (2016). How Much Data is Created on the Internet Each Day? https://www.gwava.com/blog/internet-data-created-daily . Retrieved on February 28, 2017. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/kumar-2018-aggression-annotated-corpus-hindi-english/