Elamur Research⌕
☰
home / papers / 2020 / MIND: A Large-scale Dataset for News Recommendation

MIND: A Large-scale Dataset for News Recommendation

Summary

Research on news recommendation has trailed work on product and movie recommendation, mainly because there was no large, high-quality public English benchmark. Existing public sets are small, mostly non-English, or lack the original article text. News also poses its own difficulties: articles expire quickly, which worsens cold-start; their content is textual; and user preferences must be inferred from clicks rather than ratings. To address this, the authors release MIND, built from six weeks of anonymized Microsoft News behavior logs. It is organized as impression logs (the articles shown to a user and which were clicked), each paired with the user's earlier click history. Every article includes its title, abstract, body and category. The authors re-implement several state-of-the-art recommenders originally tested on private data and benchmark them on MIND. Two factors drive accuracy: how well the model understands article content, using NLP text encoders and pre-trained language models, and how well it models user interest. MIND gives the field a shared, reproducible testbed and can also support NLP tasks such as topic classification and headline generation.

Key points

  • Scale: MIND covers 1,000,000 users, 161,013 English articles and 24,155,470 clicks. It is larger or richer than the Plista, Adressa, Globo and Yahoo! datasets, and it is distributed via msnews.github.io.
  • Construction and split:
    • Users were randomly sampled if they had at least 5 clicks between October 12 and November 22, 2019.
    • User IDs were anonymized through hashing with a one-time salt.
    • The fifth week serves as training data and the last week as test data, with the first four weeks used to build click histories.
  • Article text: Bodies proved more informative than titles or abstracts. Combining several text fields, plus category labels and entities, raised accuracy further. Attentive multi-view learning (AMV) outperformed plain concatenation.
  • User modeling: All news encoders were fixed to LSTM for this comparison. Average was the weakest method; Attention, Candidate-Att and GRU did better, and LSTUR better still. Multi-head self-attention (Self-Att) scored best, reaching 66.91 on the first reported metric versus 65.22 for Average.
  • Click history and future work: Accuracy generally rises as users' click histories grow longer, so cold-start users remain difficult. The authors plan to add images, video, other languages and richer engagement signals.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

75 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 72 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.331

@inproceedings{wu2020mind,
  title = {{MIND: A Large-scale Dataset for News Recommendation}},
  author = {Wu, Fangzhao and Qiao, Ying and Chen, Jiun-Hung and Wu, Chuhan and Qi, Tao and Lian, Jianxun and Liu, Danyang and Xie, Xing and Gao, Jianfeng and Wu, Winnie and Zhou, Ming},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.331},
  url = {https://doi.org/10.18653/v1/2020.acl-main.331}
}

References

32 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jon Atle Gulla, Lemei Zhang, Peng Liu, Özlem Özgöbek, and Xiaomeng Su. 2017. The adressa dataset for news recommendation. In Proceedings of the international conference on web intelligence, pages 1042–1048. ACM. find
  2. Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. Deepfm: a factorization-machine based neural network for ctr prediction. In AAAI, pages 1725–1731. AAAI Press. find
  3. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In CIKM, pages 2333–2338. find
  4. Mingxiao An, Fangzhao Wu, Chuhan Wu, Kun Zhang, Zheng Liu, and Xing Xie. 2019. Neural news recommendation with long-and short-term user representations. In ACL, pages 336–345. find
  5. Ilija Ilievski and Sujoy Roy. 2013. Personalized news recommendation based on implicit feedback. In Proceedings of the 2013 international news recommender systems workshop and challenge, pages 10– 15. ACM. find
  6. Trapit Bansal, Mrinal Das, and Chiranjib Bhattacharyya. 2015. Content driven user profiling for comment-worthy recommendations of news and blog articles. In RecSys., pages 195–202. ACM. find
  7. Margarita Karkali, Dimitris Pontikis, and Michalis Vazirgiannis. 2013. Match the news: A firefox extension for real-time news recommendation. In SI-GIR, pages 1117–1118. ACM. find
  8. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787–2795. find
  9. Benjamin Kille, Frank Hopfgartner, Torben Brodt, and Tobias Heintz. 2013. The plista dataset. In Proceedings of the 2013 international news recommender systems workshop and challenge, pages 16– 23. ACM. find
  10. Cheng Chen, Xiangwu Meng, Zhenghua Xu, and Thomas Lukasiewicz. 2017. Location-aware personalized news recommendation with deep semantic analysis. IEEE Access, 5:1624–1638. find
  11. Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In DLRS, pages 7–10. ACM. find
  12. Abhinandan S Das, Mayur Datar, Ashutosh Garg, and Shyam Rajaram. 2007. Google news personalization: scalable online collaborative filtering. In WWW, pages 271–280. ACM. find
  13. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248– 255. find
  14. Michal Kompan and Mária Bieliková. 2010. Content-based news recommendation. In International conference on electronic commerce and web technologies, pages 61–72. Springer. find
  15. Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In KDD, pages 426–434. find
  16. Lei Li, Ding-Ding Wang, Shun-Zhi Zhu, and Tao Li. 2011. Personalized news recommendation: a review and an experimental investigation. Journal of computer science and technology, 26(5):754. find
  17. Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In WWW, pages 661–670. ACM. find
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186. find
  19. Jianxun Lian, Fuzheng Zhang, Xing Xie, and Guangzhong Sun. 2018. Towards better representation learning for personalized news recommendation: a multi-channel deep fusion approach. In IJ-CAI, pages 3805–3811. find
  20. Florent Garcin, Christos Dimitrakakis, and Boi Faltings. 2013. Personalized news recommendation with context trees. In RecSys., pages 105–112. find
  21. Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010. Personalized news recommendation based on click behavior. In IUI, pages 31–40. ACM. find
  22. Suyu Ge, Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2020. Graph enhanced representation learning for news recommendation. In WWW, pages 2863––2869. find
  23. Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017. Embedding-based news recommendation for millions of users. In KDD, pages 1933–1942. ACM. for news recommendation. In WWW, pages 1835– 1844. find
  24. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543. find
  25. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In EMNLP, pages 2383–2392. find our card
  26. Steffen Rendle. 2012. Factorization machines with libfm. TIST, 3(3):57:1–57:22. find
  27. Jeong-Woo Son, A Kim, Seong-Bae Park, et al. 2013. A location-based news article recommendation with explicit localized semantic analysis. In SIGIR, pages 293–302. ACM. find
  28. Gabriel de Souza Pereira Moreira, Felipe Ferreira, and Adilson Marques da Cunha. 2018. News session-based recommendations using deep neural networks. In DLRS, pages 15–23. ACM. find
  29. Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. Dkn: Deep knowledge-aware network Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019a. Neural news recommendation with attentive multi-view learning. In IJCAI-19, pages 3863–3869. find
  30. Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019b. Npa: Neural news recommendation with personalized attention. In KDD, pages 2576–2584. ACM. find
  31. Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019c. Neural news recommendation with multi-head self-attention. In EMNLP-IJCNLP, pages 6390–6395. find
  32. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In NAACL, pages 1480–1489. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2020-mind-large-scale-dataset-news/