📄 Machine Learning based English Sentiment Analysis

Summary

The paper tackles three-way polarity classification of English product reviews, labelling each review as positive, negative or neutral so that sellers can gauge customer satisfaction. Despite the English title and abstract, the body is written in Vietnamese. The authors gathered reviews from Amazon for four categories of women's products: dresses, handbags, shoes and rings. From each review they keep only the product type, the title and the body text. They trained a Support Vector Machine using the WEKA library and wrapped it in a Java web application. The training set has 300 comments and the test set has 400. On the test set, F-measure is about 92% for positive, 87% for negative and 81% for neutral reviews. Negative reviews were predicted with very high precision but noticeably lower recall. The work is a small, practical baseline for review sentiment rather than a methodological advance. Its value lies in its clear class definitions and its per-class error accounting.

Key points

• The data are Amazon reviews of four women's product types (dresses, handbags, shoes, rings), split into 300 training and 400 test comments. Gold labels in the test set are 140 positive, 130 negative and 130 neutral.
• The method is an SVM classifier built with WEKA and deployed in a Java web program.
• Per-class results:
  • positive: P 89.3%, R 95.0%, F 92.1%
  • negative: P 97.1%, R 78.5%, F 86.8%
  • neutral: P 76.7%, R 86.2%, F 81.2%
• The system's predicted class counts were 149 positive, 105 negative and 146 neutral. It under-predicts the negative class and over-predicts neutral and positive. From these counts, roughly 87% of test items are labelled correctly.
• The authors list three limitations as future work: they do not yet handle negation, comparative sentences or conditional sentences. They also plan to add product-specific vocabulary and more product types.

Interesting ideas

• I1. Neutral is defined as either balanced praise and criticism or no evaluation at all, with hedge words like "ok" or "alright" as typical cues; this separates "mixed" from "absent" opinion conceptually even though both get one label.
• I2. The surveyed Crystal system abstracts specific party names into role placeholders ("winning party" vs "other party"), copying a sentence once per mentioned party, so the classifier learns patterns about roles rather than names.
• I3. The surveyed appraisal-group approach treats an evaluative phrase such as "not extremely brilliant" as the unit, with slots for appraiser, appraised target, appraisal type and orientation. Words are not the unit.
• I4. The authors report both the gold class distribution and the predicted class distribution alongside per-class P/R/F, which immediately reveals that the model is conservative on negatives.
• I5. Negation, comparatives and conditionals are named as the constructions that defeat the flat classifier.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): the attitude layer in prag could keep "mixed evaluation" and "no evaluation" as distinct speaker stances, and expose hedges like "ok" as an explicit weak-approval act.
• I2 (learn): when inducing repair rules or training on silver data, replacing named entities with their frame role could let one pattern generalize across many entities.
• I3 (frames): an evaluation frame with these four slots, filled from the UD subtree of the adjective and its modifiers and negators, would give explainable sentiment instead of word counting.
• I4 (learn): evaluation reports could always print the predicted-vs-gold label histogram per class, a cheap check that exposes systematic bias before reading individual errors.
• I5 (answer): these three constructions could be turned into targeted test sets for review-style QA, checking that polarity flips or is suspended correctly according to the UD tree.