📄 Extractive Opinion Summarization in Quantized Transformer Spaces

Summary

Opinion summarization means condensing what many reviewers say about one entity. Training references barely exist, inputs can reach thousands of reviews, and users may want either an overview or a focus on one aspect. Abstractive systems that learn by reconstructing reviews tend to hallucinate and slip into first-person phrasing, and they were tested on ten or fewer reviews.

The authors propose the Quantized Transformer (QT), an unsupervised extractive model adapted from VQ-VAE. A Transformer encoder splits each sentence into several head vectors, which are mapped onto a shared codebook of discrete latent codes. Training uses sentence reconstruction with Soft EM. At inference, the codes act as clusters, and sentences tied to the most populated codes form the summary. Aspect summaries need no retraining: the system automatically picks the head with the lowest aspect entropy and queries it with a few aspect keywords.

The authors also release SPACE, a TripAdvisor hotel benchmark. QT beats strong baselines on SPACE and two other benchmarks, and human judges preferred its output.

Key points

• SPACE contains 1,050 human-written summaries for 50 hotels, both general and aspect-specific. Each summary draws on 100 reviews, about ten times more input than earlier benchmarks.
• Aspect summaries cover six aspects (building, cleanliness, food, location, rooms, service). QT leads the baselines on all of them except building.
• In a human study, 58.7% of QT's aspect summaries stayed exclusively on the requested aspect, and only 8.7% ignored it. For LexRank_ASP, 23.8% of summaries missed the aspect, and MeanSum and Copycat did worse still.
• Trained on just 5% of the data, QT roughly matches Copycat trained on the full corpus (ROUGE-1 on SPACE).
• An eight-head QT learned 1024 latent codes. A t-SNE plot shows one head forming an aspect-separating sub-space without any aspect supervision. Code is at github.com/stangelid/qt.