Elamur Research⌕
☰
home / papers / 2019 / Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations

Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations

Summary

This paper presents what the authors describe as the first reported experiments on speech act recognition for Arabic asynchronous conversations, specifically tweets. Earlier Arabic work covered only synchronous settings such as dialogues. The authors used the ArSAS corpus, which contains more than 21K mostly dialectal Arabic tweets labelled with a Twitter-oriented speech act taxonomy derived from Searle. They merged the two smallest categories, leaving five target classes.

They compared two families of models:

  • SVM: hand-engineered lexical, syntactic and structural features, including punctuation, hashtags, mentions, emojis and links.
  • Neural networks: several architectures built on pretrained word embeddings.

All neural models beat the best SVM. A Bi-LSTM did best, reaching 87.5% accuracy and 0.615 macro-F1, against 86.5% and 0.532 for the SVM. Accuracy and micro-F1 look strong, but rare classes remain poorly recognised because the data is heavily imbalanced. The work gives Arabic social-media speech act classification a first baseline. It also suggests that methods developed for English tweets carry over to Arabic dialects.

Key points

  • Data: ArSAS (Elmadany et al., 2018) has more than 21K tweets with six speech act labels, reduced to five classes: Assertion, Expression, Question, Request and Miscellaneous.

  • SVM results: using all feature sets under 5-fold cross-validation, the SVM reached 86.5% accuracy, 0.862 micro-F1 and 0.532 macro-F1. Roughly 90% of Miscellaneous and Request tweets were misclassified.

  • Bi-LSTM results: 87.5% accuracy, 0.860 micro-F1 and 0.615 macro-F1, an 8.3-point macro-F1 gain over the SVM.

  • Per-class Bi-LSTM F1:

    ClassF1
    Assertion0.90
    Expression0.87
    Question0.57
    Miscellaneous0.53
    Request0.20

  • Comparison with English: earlier English tweet work (Zhang et al., 2011; Vosoughi and Roy, 2016b) reported F1 of about 0.70. The authors note that oversampling the minority classes did not improve results.

Interesting ideas

  • I1. Reporting macro-F1 next to accuracy and micro-F1 exposed that strong headline scores concealed near-total failure on small classes, so per-class and macro metrics deserve to be mandatory in imbalanced tasks. — For Mova (learn): evaluation of prag's speech act labels should always include per-class F1 and macro-F1, so rare acts like requests cannot hide behind frequent assertions.
  • I2. Each frequent punctuation mark became its own binary feature, while the remaining marks were pooled into one "other punctuation" flag, a cheap and transparent cue for questions, requests and expressions. — For Mova (frames): prag could record such surface markers as explicit, human-readable evidence attached to each speech act decision rather than burying them in a dense vector.
  • I3. The taxonomy was a Searle-based set adapted to what actually occurs on the platform, and the rarest categories were merged when data was too thin to support them. — For Mova (frames): Mova's speech act inventory could keep a Searle backbone with genre-specific subclasses, collapsing sparse ones until enough examples exist.
  • I4. Oversampling the minority classes did not help, and the authors conclude that genuinely new examples of rare classes are needed. — For Mova (learn): rather than duplicating rare speech acts, the silver-data pipeline or LLM teacher tiers could be targeted at producing fresh, agreed examples of underrepresented acts.
  • I5. The authors suggest training separate classifiers per topic type in ArSAS (long-standing, entity, event), on the idea that mixing topic kinds blurs the decision. — For Mova (frames): speech act recognition could be conditioned on the text's genre or topic type, which the world stage already tracks, instead of one global classifier.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech act theory

Cited by

4 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4620

@inproceedings{algotiml2019arabic,
  title = {{Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations}},
  author = {Algotiml, Bushra and Elmadany, AbdelRahim and Magdy, Walid},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4620},
  url = {https://doi.org/10.18653/v1/w19-4620}
}

References

33 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Libby Hemphill and Andrew J Roback. 2014. Tweet acts: how constituents lobby congress via twitter. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, pages 1200–1210. ACM. find
  2. Hussein Abdul-Raof. 2006. Arabic rhetoric: A pragmatic analysis. Routledge. find
  3. Fareed Hameed Al-Hindawi, Hameed Hasoon Al-Masudi, and Ramia Fuad Mirza. 2014. The speech act theory in english and arabic. Open Journal of Modern Linguistics, 4(01):27. find
  4. Shafiq Joty and Enamul Hoque. 2016. Speech act modeling of written asynchronous conversations with task-specific embeddings and conditional structured models. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1746– 1756. find our card
  5. John Langshaw Austin. 1975. How to do things with words. Oxford university press. find
  6. Berken Bayat, Christopher Krauss, Agathe Merceron, and Stefan Arbanowski. 2016. Supervised speech act classification of messages in german online discussions. In The Twenty-Ninth International Flairs Conference. find
  7. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. Dialogue act classification in domain-independent conversations using a deep recurrent neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2012– 2021. find our card
  8. Sumit Bhatia, Prakhar Biyani, and Prasenjit Mitra. 2014. Summarizing online forum discussions–can dialog acts of individual messages help? In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 2127–2131. find our card
  9. Hark-Soo Kim, Choong-Nyoung Seon, and Jung-Yun Seo. 2011. Review of korean speech act classification: machine learning methods. Journal of Computing Science and Engineering, 5(4):288–293. find
  10. Christophe Cerisara, Pavel Kral, and Ladislav Lenc. 2018. On the effects of using word2vec representations in neural networks for dialogue act recognition. Computer Speech & Language, 47:175–193. find
  11. Minkyoung Kim and Harksoo Kim. 2018. Integrated neural network model for identifying speech acts, predicators, and sentiments of dialogue utterances. Pattern Recognition Letters, 101:1–5. find
  12. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into speech acts. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 309–316. find our card
  13. Youngjoong Ko. 2015. New feature weighting approaches for speech-act classification. Pattern Recognition Letters, 51:107–111. find
  14. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 515–520, San Diego, California. Association for Computational Linguistics. find our card
  15. Kareem Darwish and Walid Magdy. 2014. Arabic information retrieval. Foundations and Trends R in Information Retrieval, 7(4):239–342. find
  16. AbdelRahim Elmadany. 2016. Automatic Act Classification for Arabic Dialogue Context. Ph.D. thesis, Thesis. find
  17. AbdelRahim Elmadany, Sherif M. Abdou, and Mervat Gheith. 2016. Jana: A human-human dialogues corpus for egyptian dialect ldc2016t24. find
  18. James OShea, Zuhair Bandar, and Keeley Crockett. 2010. A machine learning approach to speech act classification using function words. In KES International Symposium on Agent and Multi-Agent Systems: Technologies and Applications, pages 82–91. Springer. find
  19. AbdelRahim Elmadany, Hamdy Mubarak, and Walid Magdy. 2018. Arsas: An arabic speech-act and sentiment corpus of tweets. In OSACT 3: The 3rd Workshop on Open-Source Arabic Corpora and Processing Tools, page 20. find
  20. Sujith Ravi and Jihie Kim. 2007. Profiling student interactions in threaded discussions with speech act classifiers. Frontiers in Artificial Intelligence and Applications, 158:357. find
  21. Björn Gambäck and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify hate-speech. In Proceedings of the first workshop on abusive language online, pages 85–90. find our card
  22. A Saravanaraj, JI Sheeba, and S Pradeep Devaneyan. 2016. Automatic detection of cyberbullying from twitter. International Journal of Computer Science and Information Technology & Security (IJCSITS), ISSN, pages 2249–9555. find
  23. Marwa Graja, Maher Jaoua, and Lamia Hadrich Belguith. 2013. Discriminative framework for spoken tunisian dialect understanding. In International Conference on Statistical Language and Speech Processing, pages 102–110. Springer. find
  24. John R. Searle. 1975. A taxonomy of illocutionary acts. find
  25. Lina Sherkawi, Nada Ghneim, and Oumayma Al Dakkak. 2018. Arabic speech act recognition techniques. ACM Transactions on Asian and Low-Resource Language Information Processing (TAL-LIP), 17(3):18. find
  26. Abu Bakr Soliman, Kareem Eissa, and Samhaa R. El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. find
  27. Maryam Tavafi, Yashar Mehdad, Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2013. Dialogue act recognition in synchronous and asynchronous conversations. In Proceedings of the SIGDIAL 2013 Conference, pages 117–121. find our card
  28. Soroush Vosoughi. 2015. Automatic detection and verification of rumors on Twitter. Ph.D. thesis, Massachusetts Institute of Technology. find
  29. Soroush Vosoughi and Deb Roy. 2016a. A semi-automatic method for efficient detection of stories on social media. In Tenth International AAAI Conference on Web and Social Media. find
  30. Soroush Vosoughi and Deb Roy. 2016b. Tweet acts: A speech act classifier for twitter. In Tenth International AAAI Conference on Web and Social Media. find
  31. Donghyun Yoo, Youngjoong Ko, and Jungyun Seo. 2017. Speech-act classification using a convolutional neural network based on pos tag and dependency-relation bigram embedding. IE-ICE Transactions on Information and Systems, 100(12):3081–3084. find
  32. Renxian Zhang, Dehong Gao, and Wenjie Li. 2011. What are tweeters doing: Recognizing speech acts in twitter. In Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence. find
  33. Renxian Zhang, Wenjie Li, Dehong Gao, and You Ouyang. 2013. Automatic twitter topic summarization with speech acts. IEEE transactions on audio, speech, and language processing, 21(3):649–658. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/algotiml-2019-arabic-tweet-act-speech-act/