Elamur Research⌕
☰
home / papers / 2012 / Learning from Bullying Traces in Social Media

Learning from Bullying Traces in Social Media

Summary

The paper argues that social media can supply abundant data for studying bullying, a youth health problem that social scientists have mostly examined through small, slow school surveys. Earlier computational work covered only cyberbullying. The authors define "bullying traces": public posts in which participants talk about a bullying experience, whether it happened online or in the physical world. Most traces are reactions to an episode rather than the attack itself.

Using Twitter, they show that such traces are rare in relative terms but plentiful in absolute numbers. They then frame the research needs as standard NLP problems: text classification, role labeling, sentiment/teasing analysis, and topic modeling. For each, they report baselines built with off-the-shelf tools on a keyword-filtered tweet collection. A linear SVM using unigram and bigram features identifies bullying traces with about 81% accuracy, and LDA recovers topics that make sense for bullying research.

The work matters mainly as a field-opening paper. It supplies task definitions, released annotations and software, and baseline numbers for others to improve on.

Key points

  • Prevalence estimate. In a uniform sample of 990 tweets from the 2011 TREC Microblog corpus (16M tweets), annotators found 617 non-English tweets, 371 English non-traces and only 2 English bullying traces.
    • This gives a frequency of about 0.002, with an exact Binomial 95% CI of (0.0002, 0.0073).
    • At roughly 250M tweets per day, even the lower bound implies at least 50,000 English traces daily.
  • Enriched dataset. Tweets were collected from the streaming API if they contained "bully", "bullied" or "bullying", and retweets marked "RT" were removed.
    • Of 1762 annotated tweets, 684 (39%) were bullying traces.
  • Classification setup. The authors compared Naive Bayes, SVM(linear), SVM(RBF) and Logistic Regression on three feature sets: 1g, 1g2g and 1g2gPOS.
    • They used 262 held-out test tweets, training sizes from 100 to 1500, and 30 repetitions.
  • Classification results. SVM(linear) scored 79.7% with 1g and 81.3% with 1g2g, a significant gain (p = 4×10⁻⁶).
    • Adding POS coloring (81.6%) gave no significant further improvement.
  • Roles and topics. The classic bullying roles (bully, victim, bystander, defender, assistant, reinforcer) were extended with two social-media roles, reporter and accuser.
    • LDA was run on 188K classified traces with 50 topics. Interpretable topics included feelings, suicide, family, school, verbal bullying and physical bullying.

Interesting ideas

  • I1. Estimate rarity before building a pipeline. Uniformly sampling and labeling a small set, then reporting an exact binomial confidence interval, quantifies how rare a phenomenon is. This tells you whether to invest in keyword-based enrichment, and the same check applies to any rare-event corpus project, such as hate speech or misinformation in a low-resource language.
  • I2. Accept selection bias deliberately for annotation efficiency. Keyword filtering produced a pool where about 39% of items were positive. A learned classifier then cleaned the noise. This two-stage design makes annotation affordable when positives are very scarce.
  • I3. Study the traces, not the event. Collecting posts about an event (reports, accusations, self-disclosures) widens coverage far beyond the directly observable phenomenon. The idea transfers to studying offline incidents, conflicts or harms through how people discuss them online.
  • I4. Use classifier output as a time series. Plotting daily counts of predicted traces exposed weekly cycles and spikes tied to news events. This is a cheap sanity check on a classifier and also a way to generate research findings in its own right.
  • I5. Use learning curves with repeated runs and significance tests. Varying training size, repeating 30 times and t-testing feature sets shows whether additions like POS tags are actually worth their cost. This is a solid evaluation habit for small annotated datasets.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

84 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 78 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{xu2012learning,
  title = {{Learning from Bullying Traces in Social Media}},
  author = {Xu, Jun-Ming and Jun, Kwang-Sung and Zhu, Xiaojin and Bellmore, Amy},
  booktitle = {Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2012},
  url = {https://aclanthology.org/N12-1084/}
}

References

51 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. American Psychological Association. 2004. APA resolution on bullying among children and youth. http://www.apa.org/about/governance/council/policy/bullying.pdf. ↗
  2. Andrzejewski, David, Xiaojin Zhu, Mark Craven, and Ben Recht. 2011. A framework for incorporating general domain knowledge into Latent Dirichlet Allocation using First-Order Logic. In the 22nd IJCAI, pages 1171–1177. find
  3. Archer, John and Sarah M. Coyne. 2005. An integrated review of indirect, relational, and social aggression. Personality and Social Psychology Review, 9:212–230. find
  4. Bellmore, Amy D., Melissa R. Witkow, Sandra Graham, and Jaana Juvonen. 2004. Beyond the individual: The impact of ethnic context and classroom behavioral norms on victims’ adjustment. Developmental Psychology, 40:1159–1172. find
  5. Biggs, Bridget K., Jennifer Mize Nelson, and Marilyn L. Sampilo. 2010. Peer relations in the anxiety-depression link: Test of a mediation model. Anxiety, Stress & Coping, 23(4):431–447. find
  6. Blei, David M., Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. JMLR, 3:993–1022. find our card
  7. Blitzer, John. 2008. Domain Adaptation of Natural Language Processing Systems. Ph.D. thesis, University of Pennsylvania. School Psychology Int’l, 30(4):383–402. find
  8. Chang, Chih-Chung and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. on Intelligent Systems and Technology, 2:27:1–27:27. find
  9. Cook, Clayton R., Kirk R. Williams, Nancy G. Guerra, Tia E. Kim, and Shelly Sadek. 2010. Predictors of bullying and victimization in childhood and adolescence: A meta-analytic investigation. School Psychology Quarterly, 25(2):65–83. find
  10. Dinakar, K., R. Reichart, and H. Lieberman. 2011. Modeling the detection of textual cyberbullying. In International Conference on Weblog and Social Media - Social Mobile Web Workshop, Barcelona, Spain. find
  11. Fekkes, Minne, Frans I.M. Pijpers, A. Miranda Fredriks, Ton Vogels, and S. Pauline Verloove-Vanhorick. 2006. Do bullied children get ill, or do ill children get bullied? a prospective cohort study on the relationship between bullying and health-related symptoms. Pediatrics, 117:1568–1574. find
  12. Finkel, Jenny R., Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by Gibbs sampling. In the 43rd ACL, pages 363–370. find our card
  13. Fredstrom, Bridget K., Ryan E. Adams, and Rich Gilman. 2011. Electronic and school-based victimization: Unique contexts for adjustment difficulties during adolescence. J. Youth and Adolescence, 40(4):405–415. find
  14. Gildea, Daniel and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Comput. Linguist., 28(3):245–288. find our card
  15. Gini, Gianluca and Tiziana Pozzoli. 2009. Association between bullying and psychosomatic problems: a meta-analysis. Pediatrics, 123(3):1059–1065. find
  16. Graham, Sandra, Amy Bellmore, and Jaana Juvonen. 2007. Peer victimization in middle school: When self- and peer views diverge. In Joseph E. Zins, Maurice J. Elias, and Charles A. Maher, editors, Bullying, victimization, and peer harassment: A handbook of prevention and intervention. Haworth Press, N… find
  17. Griffiths, Thomas L. and Mark Steyvers. 2004. Finding scientific topics. PNAS, 101(suppl. 1):5228–5235. find
  18. Hall, Mark, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The weka data mining software: an update. ACM SIGKDD Explorations Newsletter, 11:10–18. find
  19. Hawker, David S. J. and Michael J. Boulton. 2000. Twenty years’ research on peer victimization and psychosocial maladjustment: A meta-analytic review of cross-sectional studies. J. of Child Psychology And Psychiatry, 41(4):441–455. find
  20. Janosz, Michel, Isabelle Archambault, Linda S. Pagani, Sophie Pascal, Alexandre J.S. Morin, and François Bowen. 2008. Are there detrimental effects of witnessing school violence in early adolescence? J. of Adolescent Health, 43(6):600–608. find
  21. Jimerson, Shane R., Susan M. Swearer, and Dorothy L. Espelage. 2010. Handbook of Bullying in Schools: An international perspective. Routledge/Taylor & Francis Group, New York, NY. find
  22. Juvonen, Jaana and Sandra Graham. 2001. Peer harassment in school: The plight of the vulnerable and victimized. Guilford Press, New York, NY. find
  23. Juvonen, Jaana and Elisheva F. Gross. 2008. Extending the school grounds? – Bullying experiences in cyberspace. J. of School Health, 78:496–505. find
  24. Kontostathis, April, Lynne Edwards, and Amanda Leatherman. 2010. Text mining and cybercrime. In Michael W. Berry and Jacob Kogan, editors, Text Mining: Applications and Theory. John Wiley & Sons, Ltd, Chichester, UK. find
  25. Ladd, Gary W., Becky J. Kochenderfer, and Cynthia C. Coleman. 1997. Classroom peer acceptance, friendship, and victimization: Distinct relational systems that contribute uniquely to children’s school adjustment? Child Development, 68:1181–1197. find
  26. Latham, Annabel, Keeley Crockett, and Zuhair Bandar. 2010. A conversational expert system supporting bullying and harassment policies. In the 2nd ICAART, pages 163–168. find
  27. Lieberman, Henry, Karthik Dinakar, and Birago Jones. 2011. Let’s gang up on cyberbullying. Computer, 44:93–96. find
  28. Little, Todd D., Christopher C. Henrich, Stephanie M. Jones, and Patricia H. Hawley. 2003. Disentangling the “whys” from the “whats” of aggressive behavior. Int’l J. of Behavioral Development, 27:122–133. find
  29. Liu, Xiaohua, Kuan Li, Bo Han, Ming Zhou, Long Jiang, Zhongyang Xiong, and Changning Huang. 2010. Semantic role labeling for news tweets. In the 23rd COLING, pages 698–706. find
  30. Màrquez, Lluı́s, Pere Comas, Jesús Giménez, and Neus Català. 2005. Semantic role labeling as sequential tagging. In the 9th CoNLL, pages 193–196. find
  31. Mihalcea, Rada and Carlo Strapparava. 2005. Making computers laugh: Investigations in automatic humor recognition. In EMNLP 2005, pages 531–538. find
  32. Moore, Mark H., Carol V. Petrie, Anthony A. Braga, and Brenda L. McLaughlin. 2003. Deadly lessons: Understanding lethal school violence. The National Academies Press, Washington, DC. find
  33. Nansel, Tonja R., Mary Overpeck, Ramani S. Pilla, W. June Ruan, Bruce Simons-Morton, and Peter Scheidt. 2001. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. J. Amer. Medical Assoc., 285(16):2094–2100. find
  34. Nishina, Adrienne and Amy D. Bellmore. 2010. When might aggression, victimization, and conflict matter most?: Contextual considerations. J. of Early Adolescence, pages 5–26. find
  35. Nishina, Adrienne and Jaana Juvonen. 2005. Daily reports of witnessing and experiencing peer harassment in middle school. Child Development, 76:435–450. find
  36. Nylund, Karen, Amy Bellmore, Adrienne Nishina, and Sandra Graham. 2007. Subtypes, severity, and structural stability of peer victimization: What does latent class analysis say? Child Development, 78:1706–1722. find
  37. Olweus, Dan. 1993. Bullying at school: What we know and what we can do. Blackwell, Oxford, UK. find
  38. Pang, Bo and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In the 42nd ACL, pages 271–278. find our card
  39. Ptaszynski, Michal, Pawel Dybala, Tatsuaki Matsuba, Fumito Masui, Rafal Rzepka, and Kenji Araki. 2010. Machine learning and af-Feature-rich part-of-speech tagging with a cyclic defect analysis against cyber-bullying. In the 36th AISB, pendency network. In NAACL-HLT 2003, pages 173– pages 7–16. 180. find
  40. Punyakanok, Vasin, [Vaillancourt et al.2010] Vaillancourt, Tracy, Vi Trinh, Patricia McDougall, Eric Duku, Lesley Cunning-Dan Roth, and Wen-tau Yih. 2008. The importance ham, Charles Cunningham, Shelley Hymel, and Kathy of syntactic parsing and inference in semantic role Short. 2010. Optimizing popu… find
  41. Ratinov, Lev and Dan Roth. 9:233–250. 2009. Design challenges and misconceptions in named entity recognition. In the 13th CoNLL, pages [Vandebosch and Cleemput2009] Vandebosch, Heidi and Katrien Van Cleemput. 2009. Cyberbullying among 147–155. youngsters: profiles of bullies and victims. New media find our card
  42. Ritter, Alan, Sam Clark, Mausam, and & society, 11(8):1349–1371. Oren Etzioni. 2011. Named entity recognition in [Wang, Iannotti, and Nansel2009] Wang, Jing, Ronald J. tweets: An experimental study. In EMNLP 2011, Iannotti, and Tonja R. Nansel. 2009. School bully-pages 1524–1534. ing among adolescen… find
  43. Rivers, Ian, V. Paul Poteat, Nathalie verbal, relational, and cyber. J. Adolescent Health, Noret, and Nigel Ashurst. 2009. Observing bullying 45(4):368–375. at school: The mental health implications of witness status. School Psychology Quarterly, 24(4):211–223. find
  44. Salmivalli, Christina. 1999. Participant role approach to school bullying: Implications for intervention. J. of Adolescence, 22(4):453–459. find
  45. Schmidt, Mark W., Glenn Fung, and Rómer Rosales. 2007. Fast optimization methods for l1 regularization: A comparative study and two new approaches. In the 18th ECML, pages 286–297. find
  46. Schwartz, David, Andrea Hop-meyer Gorman, Jonathan Nakamoto, and Robin L. To-blin. 2005. Victimization in the peer group and children’s academic functioning. J. of Educational Psychology, 87:425–435. find
  47. Settles, Burr. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In the EMNLP 2011, pages 1467–1478. find
  48. Smith, Peter K., Kirsten C. Madsen, and Janet C. Moody. 1999. What causes the age decline in reports of being bullied at school? Towards a developmental analysis of risks of being bullied. Educational Research, 41(3):267–285. find
  49. The American Academy of Pediatrics. 2009. Policy statement– role of the pediatrician in youth violence prevention. Pediatrics, 124(1):393–402. find
  50. The White House. 2011. Background on White House conference on bullying prevention. http://www.whitehouse.gov/the-press-office/2011/03/10/background-white-house-conference-bullying-prevention. ↗
  51. Toutanova, Kristina, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2012-learning-bullying-traces-social-media/