📄 Tweet Acts: A Speech Act Classifier for Twitter

Summary

Most computational work on tweets looks at literal content. This paper asks a pragmatic question instead: what is the author doing with the tweet? Earlier dialogue-act methods handle Twitter's noisy, informal language poorly. The authors build a six-class taxonomy based on Searle: Assertion, Recommendation, Expression, Question, Request and Miscellaneous. They gather tweets on six topics, two for each of three topic types (entity, event, long-standing), and have three annotators label them. Each tweet is encoded as 3,313 binary features. Some are semantic: opinion, vulgar and abbreviation lexicons, emoticons, speech act verbs and filtered n-grams. Others are syntactic: punctuation, Twitter symbols, POS tags and dependency sub-trees from TweeboParser. Several standard classifiers are compared, and logistic regression does best, reaching a weighted average F1 of about 0.70. That beats the only earlier supervised Twitter system (Zhang, 0.59). Classifiers trained per topic do slightly better than per-type or Twitter-wide ones. The work matters because speech-act labels feed downstream tasks such as rumor detection and gauging public attitude on a topic.

Key points

• Dataset: 7,563 tweets, each labelled by majority vote of three annotators, with inter-annotator kappa of 0.78. Topics include the Boston Marathon bombings, Ferguson, the Red Sox and Ashton Kutcher.
• Features: 3,313 binary features, 1,647 semantic and 1,666 syntactic. Resources include:
  • the Harvard General Inquirer;
  • Wierzbicka's 229 speech act verbs in 37 groups, matched only on verbs tagged by the Owoputi et al. tagger and Porter-stemmed;
  • 1,415 n-grams and 1,655 dependency sub-trees of one or two edges.
• Classifier comparison (20-fold cross-validation): NB, DT, LR, SVM and a majority baseline were tested. LR scored 0.70 on average, versus 0.69 for SVM and 0.24 for the baseline.
• Weak classes: Questions are easy (F1 0.87) and Expressions strong (0.80). Requests (0.16) and Recommendations (0.30) remain hard.
• Granularity and ablation:
  • Topic-specific classifiers averaged 0.74, type-specific 0.71 and Twitter-wide 0.70.
  • Semantic-only features scored 0.64 and syntactic-only 0.62, both below the combined set.

Interesting ideas

• I1. Before matching a list of speech-act verbs, the authors first confirm that each candidate token was tagged as a verb, which avoids false hits on noun or adjective homographs; gating lexicon lookups by part of speech is a cheap precision gain wherever word lists are used.
• I2. Small dependency fragments of one or two edges serve as cues for communicative intent, which shows that local tree shape carries pragmatic signal beyond word sequences.
• I3. Candidate cue phrases and sub-trees are pruned so that ones containing topic-specific terms are dropped, which keeps the classifier from learning the subject matter instead of the act; this is a general guard against topical leakage.
• I4. Training the same model at three scopes (whole platform, topic type, single topic) makes the generalization-versus-overfitting trade-off something you measure rather than assume.
• I5. An ablation that trains on semantic-only and syntactic-only features shows that each family covers classes the other misses; for example, syntax alone nails questions but fails on Miscellaneous.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): prag could detect performative verbs (ask, demand, promise, report) only on UD heads tagged VERB, mapping them to speech-act frames instead of matching surface strings.
• I2 (frames): mine recurring UD sub-tree patterns, such as a modal plus a 2nd-person subject or a wh-word plus aux inversion, as explainable rules for Request, Recommendation and Question.
• I3 (learn): when inducing repair or classification rules from silver data, filter out rules anchored on content entities, and use topic-shifted test sets as negative controls.
• I4 (answer): evaluate readers separately per genre (fables, school tests, word problems) against a global model to see where domain-specific handling is actually worth it.
• I5 (learn): run per-class ablations of lexical versus tree-based evidence in prag's speech-act decisions to confirm that both channels earn their place.