Elamur Research⌕
☰
home / papers / 2014 / Developing Age and Gender Predictive Lexica over Social Media

Developing Age and Gender Predictive Lexica over Social Media

Paper · 2014 · Maarten Sap, Gregory Park, Johannes Eichstaedt et al.

Summary

Large-scale social media studies often lack age and gender information, even though these attributes strongly shape behaviour and language. This paper builds openly released predictive lexica for both attributes. The authors use the MyPersonality Facebook dataset (75,394 English-speaking users with self-reported age and gender, over 300M words), plus annotated blogs and gender-labelled Twitter users. They observe that a linear model over relative word frequencies is mathematically the same as a weighted lexicon plus an intercept. So they fit ridge regression for continuous age and an L1-penalized linear SVM for gender on 1gram features, then distribute the learned coefficients as word–weight lists. On held-out Facebook users, the lexica reach close to state-of-the-art results, about r = .83 for age and roughly 91.9% gender accuracy. Lexica trained on several platforms together generalize to blogs and Twitter almost as well as platform-specific models. Packaging the models as lexica gives social scientists, who already rely on hand-built lexica like LIWC, a transparent and portable data-driven tool.

Key points

  • Training data: 72,874 Facebook users (2009–2011), each with at least 1,000 words and under 65 years old. Test data: 1,000 random users plus a stratified sample balanced by gender across 4-year age bins from 13 to 60.
  • Features are 1grams from the Happier Fun Tokenizer, kept only if used by at least 1% of users. Age is modelled with ridge regression and gender with a linear SVM with L1 penalty, which also shrinks the lexicon.
  • In-domain Facebook results: the FB lexicon gives age r = .835 (MAE 3.40 years) and 91.7% gender accuracy. Adding 2- and 3-grams pushes this to r = .85 and 92.7%.
  • Cross-domain results: the Facebook-only lexica drop on blogs (age r = .710, gender 77.4%). The combined FB+BG+T gender lexicon reaches 90.0% on Twitter, beating earlier text-only Twitter results of 75.5–87%. The released models are the FB+BG age lexicon and the FB+BG+T gender lexicon.
  • Accuracy falls as the number of messages per user shrinks. Age r and gender accuracy are .820/.901 at 100 messages, .688/.796 at 20, and only .156/.554 with a single message.

Interesting ideas

  • I1. A linear model is a lexicon. If the features are relative word frequencies, a regression or classifier's coefficients can be shipped as a simple word–weight list with an intercept. This makes a trained model easy to share and inspect, and usable without ML tooling, for example in a Ukrainian sentiment or register lexicon.
  • I2. Multivariate weighting instead of univariate scores. Fitting all words jointly, rather than scoring each in isolation as PMI does, keeps correlated words from counting the same signal twice. This is worth adopting whenever lexicon words co-occur heavily.
  • I3. L1 penalty as lexicon compression. Sparse regularization zeroes out many weights and so produces a smaller, more practical lexicon with little loss in accuracy.
  • I4. Pooling training data across genres. Training on several platforms together gave nearly domain-specific performance everywhere. This is a cheap way to obtain one robust model when target genres vary.
  • I5. Evaluation beyond a random split. Testing on a demographically stratified sample and on out-of-domain corpora reveals biases that a skewed random test set hides. Measuring accuracy as available text per user shrinks shows how much input a model needs in practice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PennsylvaniaPhiladelphia · United States170 works · 60,739 citations🎓University of CambridgeCambridge · United Kingdom155 works · 115,562 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Psychology · 798 works Social Psychology · 129 works

Cite

Cite as Maarten Sap, Gregory Park, Johannes Eichstaedt, Margaret Kern, David Stillwell, Michal Kosinski, Lyle Ungar and Hansen Andrew Schwartz. 2014. Developing Age and Gender Predictive Lexica over Social Media. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/d14-1121

@inproceedings{sap2014developing,
  title = {{Developing Age and Gender Predictive Lexica over Social Media}},
  author = {Maarten Sap, Gregory Park, Johannes Eichstaedt, Margaret Kern, David Stillwell, Michal Kosinski, Lyle Ungar and Hansen Andrew Schwartz},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1121},
  url = {https://doi.org/10.3115/v1/d14-1121}
}

References

39 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Faiyaz Al Zamal, Wendy Liu, and Derek Ruths. 2012. Homophily and latent attribute inference: Inferring latent attributes of twitter users from neighbors. In ICWSM. find
  2. Shlomo Argamon, Moshe Koppel, James W Pennebaker, and Jonathan Schler. 2009. Automatically profiling the author of an anonymous text. Communications of the ACM, 52(2):119–123. find
  3. John D Burger and John C Henderson. 2006. An exploration of observable features related to blogger age. In AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs, pages 15–20. find
  4. John D Burger, John C Henderson, George Kim, and Guido Zarrella. 2011. Discriminating gender on twitter. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1301–1309. Association for Computational Linguistics. 2014. Faststats: How healthy are we. http://www.… ↗ our card
  5. Morgane Ciot, Morgan Sonderegger, and Derek Ruths. 2013. Gender inference of twitter users in non-english contexts. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash, pages 18–21. find
  6. Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Predicting postpartum changes in emotion and behavior via social media. In Proceedings of the 2013 ACM annual conference on Human factors in computing systems, pages 3267– 3276. ACM. find
  7. Peter Sheridan Dodds, Kameron Decker Harris, Isabel M Kloumann, Catherine A Bliss, and Christopher M Danforth. 2011. Temporal patterns of happiness and information in a global social network: Hedonometrics and twitter. PloS one, 6(12):e26752. find
  8. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A library for large linear classification. The Journal of Machine Learning Research, 9:1871–1874. find
  9. Inc. Google. 2014. Google flu trends. http://www.google.org/flutrends. Accessed on March 12, 2014. ↗
  10. Sumit Goswami, Sudeshna Sarkar, and Mayur Rustagi. 2009. Stylometric analysis of bloggers age and gender. In Third International AAAI Conference on Weblogs and Social Media. find
  11. Arthur E Hoerl and Robert W Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67. find
  12. David A Huffaker and Sandra L Calvert. 2005. Gender, identity, and language use in teenage blogs. Journal of Computer-Mediated Communication, 10(2):00–00. find
  13. Rosie Jones, Ravi Kumar, Bo Pang, and Andrew Tomkins. 2007. I know what you did last summer: query logs and user privacy. In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, pages 909–914. ACM. find
  14. Margaret L Kern, Johannes C Eichstaedt, H Andrew Schwartz, Gregory Park, Lyle H Ungar, David J Stillwell, Michal Kosinski, Lukasz Dziurzynski, and Martin EP Seligman. 2014. From sooo excited!!! to so proud: Using language to study development. Developmental psychology, 50(1):178–188. find
  15. Michal Kosinski and David J Stillwell. 2012. mypersonality project. http://www.mypersonality.org/wiki/. ↗
  16. Michal Kosinski, David J Stillwell, and Thore Graepel. 2013. Private traits and attributes are predictable from digital records of human behavior. volume 110, pages 5802–5805. National Acad Sciences. find
  17. David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-Laszlo Barabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. 2009. Computational social science. Science, 323(5915):721– 723. find
  18. Wendy Liu and Derek Ruths. 2013. Whats in a name? using first names as features for gender inference in twitter. In Analyzing Microtext: 2013 AAAI Spring Symposium. find
  19. Alessandra Marengoni, Sara Angleman, René Melis, Francesca Mangialasche, Anita Karp, Annika Gar-men, Bettina Meinow, and Laura Fratiglioni. 2011. Aging with multimorbidity: a systematic review of the literature. Ageing research reviews, 10(4):430– 439. find
  20. Robert R McCrae, Paul T Costa, Margarida Pedroso de Lima, António Simões, Fritz Ostendorf, Alois Angleitner, Iris Marušić, Denis Bratko, Gian Vittorio Caprara, Claudio Barbaranelli, et al. 1999. Age differences in personality across the adult life span: parallels in five cultures. Developmental… find
  21. Alan Mislove, Sune Lehmann, Yong-Yeol Ahn, Jukka-Pekka Onnela, and J Niels Rosenquist. 2011. Understanding the demographics of twitter users. ICWSM, 11:5th. find
  22. Saif M Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu. 2013. Nrc-canada: Building the state-of-the-art in sentiment analysis of tweets. arXiv preprint arXiv:1308.6242. our card
  23. Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and Theo Meder. 2013. how old do you think i am?: A study of language and age in twitter. In Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media. find
  24. Michael J Paul and Mark Dredze. 2011. You are what you tweet: Analyzing twitter for public health. In ICWSM. find
  25. Marco Pennacchiotti and Ana-Maria Popescu. 2011. A machine learning approach to twitter user classification. In ICWSM. find
  26. James W Pennebaker and Lori D Stone. 2003. Words of wisdom: language use over the life span. Journal of Personality and Social Psychology, 85(2):291– 301. find
  27. James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: Liwc 2001. 71:2001. 2014. Social networking fact sheet. http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/. Accessed on August 26, 2014. ↗
  28. Francisco Rangel and Paolo Rosso. 2013. Use of language and author profiling: Identification of gender and age. Natural Language Processing and Cognitive Science, page 177. find
  29. Delip Rao, David Yarowsky, Abhishek Shreevats, and Manaswi Gupta. 2010. Classifying latent user attributes in twitter. In Proceedings of the 2nd international workshop on Search and mining user-generated contents, pages 37–44. ACM. find
  30. Jonathan Schler, Moshe Koppel, Shlomo Argamon, and James W Pennebaker. 2006. Effects of age and gender on blogging. In AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs, volume 6, pages 199–205. find
  31. H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Megha Agrawal, Gregory J Park, Shrinidhi K Lakshmikanth, Sneha Jha, Martin EP Seligman, Lyle Ungar, et al. 2013a. Characterizing geographic variation in well-being using tweets. In ICWSM. find
  32. H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Stephanie M Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David J Stillwell, Martin EP Seligman, et al. 2013b. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one, 8… find
  33. Rong Su, James Rounds, and Patrick Ian Armstrong. 2009. Men and things, women and people: a meta-analysis of sex differences in interests. Psychological Bulletin, 135(6):859–884. find
  34. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll, and Manfred Stede. 2011. Lexicon-based methods for sentiment analysis. Computational linguistics, 37(2):267–307. our card
  35. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54. find
  36. Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288. find
  37. Benjamin Van Durme. 2012. Streaming analysis of discourse participants. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 48–58. Association for Computational Linguistics. find
  38. Svitlana Volkova, Theresa Wilson, and David Yarowsky. 2013. Exploring demographic language variations to improve multilingual sentiment analysis in social media. In Proceedings of the 2013 Conference on Empirical Methods on Natural Language Processing. our card
  39. Theresa Wilson, Zornitsa Kozareva, Preslav Nakov, Sara Rosenthal, Veselin Stoyanov, and Alan Ritter. 2013. Semeval-2013 task 2: Sentiment analysis in twitter. In Proceedings of the International Workshop on Semantic Evaluation, SemEval, volume 13. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sap-2014-developing-age-gender-predictive-lexica/