Elamur Research
☰
home / papers / 2019 / Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects

Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D19-1018.pdf

Summary

Explainable recommender systems often justify suggestions by generating review- or tip-like text. The authors argue that such text is a poor training target because much of it is generic praise or personal storytelling rather than concrete reasons for a choice. To get better training data, they split reviews into Elementary Discourse Units (EDUs) and filter them with simple rules. A small set is hand-labeled as good or bad justifications, and a fine-tuned BERT classifier then distantly labels whole Yelp and Amazon Clothing corpora. Fine-grained aspects are extracted from each justification to build user personas and item profiles.

On this data they train two generators. Ref2Seq, and its aspect-planning variant AP-Ref2Seq, attends to a user's and an item's past justifications. ACMLM, an aspect-conditional masked language model, fills in templates drawn from earlier justifications. Sampling-based models produce more diverse and informative output, while beam-search models score higher on BLEU. The work matters because it supplies large personalized justification datasets and shows that extracted justifications beat full reviews as a training source.

Key points

  • Annotation: two expert annotators labeled 1,000 + 600 EDU segments (Cohen's kappa 0.927 after alignment). About 24.8% of segments were judged good justifications.
  • Classification: BERT gave the best F1 (~0.75) and precision (0.80) among the classifiers. A BERT sentiment model (BERT-SA) reached only ~0.49 F1, showing that positive sentiment is not the same as justification quality.
  • Transfer: the classifier was trained only on Yelp labels but also worked on Amazon Clothing. Aspects were mined with the sentiment-lexicon method of Zhang et al. (2014).
  • Evaluation: models were compared with BLEU and Distinct-1/2. Human raters scored 100 Yelp outputs per model on Relevance, Informativeness and Diversity (1–5 scale, at least 3 raters each).
  • Main finding: Ref2Seq (Top-k) and ACMLM scored highest on human-rated Diversity and Informativeness. AP-Ref2Seq usually mentioned the aspects it was asked to plan, whereas models trained on whole reviews or tips produced generic phrases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

221 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 219 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1018

@inproceedings{ni2019justifying,
  title = {{Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects}},
  author = {Ni, Jianmo and Li, Jiacheng and McAuley, Julian},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1018},
  url = {https://doi.org/10.18653/v1/d19-1018}
}

References

34 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2017. Guided open vocabulary image captioning with constrained beam search. In EMNLP. find
  2. Stefanos Angelidis and Mirella Lapata. 2018. Summarizing opinions: Aspect extraction meets sentiment prediction and they are both weakly supervised. In EMNLP. find
  3. Ashutosh Baheti, Alan Ritter, Jiwei Li, and William B. Dolan. 2018. Generating more interesting responses in neural conversation models with distributional constraints. In EMNLP. find
  4. Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from rst discourse parsing. In EMNLP. find
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  6. Rose Catherine and William W. Cohen. 2017. Transnets: Learning to transform for recommendation. In RecSys. find
  7. Jianmo Ni, Zachary C. Lipton, Sharad Vikram, and Julian J. McAuley. 2017. Estimating reactions and recommending products with generative models of reviews. In IJCNLP. find
  8. Kyunghyun Cho, Bart van Merrienboer, aglar Gülehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP. find
  9. Jacob Willem Cohen. 1960. A coefficient of agreement for nominal scales. find
  10. Jianmo Ni and Julian McAuley. 2018. Personalized review generation by expanding phrases and attending on aspect-aware representations. In ACL. find
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL. find
  12. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  13. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In NIPS. find
  14. Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. 2017. Learning to generate product reviews from attributes. In EACL. find
  15. Alex Wang and Kyunghyun Cho. 2019. Bert has a mouth, and it must speak: Bert as a markov random field language model. CoRR, abs/1902.04094. find our card
  16. Günes Erkan and Dragomir R. Radev. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res., 22:457–479. find
  17. Yizhong Wang, Sujian Li, and Jingfeng Yang. 2018. Toward fast and accurate neural discourse segmentation. In EMNLP. find
  18. Jun Gao, Wei Bi, Xiaojiang Liu, Junhui Li, and Shuming Shi. 2018. Generating multiple diverse responses for short-text conversation. In AAAI. find
  19. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9:1735– 1780. find
  20. Jason Weston, Emily Dinan, and Alexander H. Miller. 2018. Retrieve and refine: Improved sequence generation models for dialogue. In SCAI@EMNLP. find
  21. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. CoRR, abs/1904.09751. find
  22. Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Mask and infill: Applying masked language model to sentiment transfer. In IJ-CAI. find
  23. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B. Dolan. 2015. A diversity-promoting objective function for neural conversation models. In HLT-NAACL. find our card
  24. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, We… find
  25. Juncen Li, Robin Jia, He He, and Percy S. Liang. 2018. Delete, retrieve, generate: A simple approach to sentiment and style transfer. In NAACL-HLT. find our card
  26. Junyi Li, Wayne Xin Zhao, Ji-Rong Wen, and Yang Song. 2019a. Generating long and informative reviews with aspect-aware coarse-to-fine decoding. In ACL. find
  27. Piji Li, Zihao Wang, Lidong Bing, and Wai Lam. 2019b. Persona-aware tips generation. In WWW. find
  28. Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. 2018. Response generation by context-aware prototype editing. In AAAI. find
  29. Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. 2017. Neural rating regression with abstractive tips generation for recommendation. In SI-GIR. find
  30. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. In AAAI. find
  31. Hui Liu, Qingyu Yin, and William Yang Wang. 2019. Towards explainable nlp: A generative explanation framework for text classification. In ACL. find
  32. Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In SIGIR. find
  33. William C. Mann and Sandra A. Thompson. 1988. Rhetorical structure theory: toward a functional theory of text. find
  34. Elman Mansimov, Alex Wang, and Kyunghyun Cho. 2019. A generalized framework of sequence generation with application to undirected sequence models. ArXiv, abs/1905.12790. — [arXiv:1905.12790](https://arxiv.org/abs/1905.12790) find