Elamur Research⌕
☰
home / papers / 2020 / TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks

TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks

Summary

Previously, Chinese discourse-relation resources covered only written news and were mostly hard to obtain (purchase or owner permission). This paper introduces TED-CDB, a freely released corpus of Chinese TED talks annotated with shallow discourse relations. It provides a spoken-monologue counterpart to news-based treebanks. Annotation follows the PDTB 3.0 sense hierarchy, with changes for Chinese and for speech:

  • a new Expansion subtype, Progression;
  • implicit relations allowed between non-adjacent sentences;
  • verbs counted as explicit connectives, identified through Chinese PropBank modifier roles.

The talks include both Chinese translations of English talks and talks originally delivered in Chinese. Benchmarks with pretrained models such as BERT-wwm show that the data is hard. Level-1 classification F1 stays under 60%, about 35 points below scores on CDTB news text. Transfer experiments show that training on TED-CDB helps other corpora: CDTB in the same language but a different domain, and the multilingual TED-MDB in the same domain but other languages. The resource is therefore useful for spoken-language tasks and for corpora that are small or skewed toward one sense.

Key points

  • TED-CDB covers 72 Chinese TED talks (~268.1K words) with 15,540 discourse relations. That is roughly three times the 5,534 relations in CDTB, and about 7.0K of them are implicit.
  • Data and Chinese annotation guidelines are freely available at github.com/wanqiulong0923/TED-CDB.
  • On 4-way (Level-1) classification, state-of-the-art classifiers score below 60% F1. Level-2 results are also reported as baselines.
  • In zero-shot transfer to CDTB (3-way implicit classification with Expansion removed, BERT-wwm), Contingency and Comparison improved. Temporal did not improve, partly because CDTB has only 7 implicit Temporal test instances.
  • Training multilingual BERT on TED-CDB beat in-corpus cross-validation for all 7 TED-MDB languages, which have about 200 implicit relations each. Comparison showed especially large gains.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.223

@inproceedings{long2020ted,
  title = {{TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks}},
  author = {Long, Wanqiu and Webber, Bonnie and Xiong, Deyi},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.223},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.223}
}

References

35 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, and Guoping Hu. 2019. Pre-training with whole word masking for Chinese BERT. ArXiv, abs/1906.08101. find our card
  2. Wanqiu Long, Xinyi Cai, James Reid, Bonnie Webber, and Deyi Xiong. 2020. Shallow discourse annotation for chinese TED talks. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 1018–1025, Marseille, France. European Language Resources Association. find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  4. Todor Mihaylov and Anette Frank. 2019. Discourse-aware semantic self-attention for narrative reading comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), page… find
  5. D.S.E.U.S. Greenbaum, S. Greenbaum, and Oxford University Press. 1996. Comparing English Worldwide: The International Corpus of English. Clarendon Press. find
  6. Hu Hai and Kübler Sandra. 2020. Investigating translated chinese and its variants using machine learning. Journal of Natural Language Engineering, page 1–34. find
  7. Hen-Hsen Huang and Hsin-Hsi Chen. 2011a. Chinese discourse relation recognition. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1442–1446, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find our card
  8. Hen-Hsen Huang and Hsin-Hsi Chen. 2011b. Chinese discourse relation recognition. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1442–1446, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. find our card
  9. Masaru Isonuma, Junichiro Mori, and Ichiro Sakata. 2019. Unsupervised neural single-document summarization of reviews via learning latent discourse structure and its ranking. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2142–2152, Florence, Italy.… find
  10. Junyi Jessy Li, Marine Carpuat, and Ani Nenkova. 2014. Assessing the discourse factors that influence the quality of machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283–288, Baltimore, Maryland. Associati… find
  11. Tian-Yu Liu. 2009. Easyensemble and feature selection for imbalance data sets. 2009 International Joint Conference on Bioinformatics, Systems Biology and Intelligent Computing, pages 517–520. find
  12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, 1907.11692. find
  13. Karthik Narasimhan and Regina Barzilay. 2015. Machine comprehension with discourse relations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1253–1262,… find our card
  14. Allen Nie, Erin Bennett, and Noah Goodman. 2019. DisSent: Learning sentence representations from explicit discourse relations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4497–4510, Florence, Italy. Association for Computational Linguistics. find our card
  15. Takumi Ohtani, Hidetaka Kamigaito, Masaaki Nagata, and Manabu Okumura. 2019. Context-aware neural machine translation with coreference information. In Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019), pages 45–50, Hong Kong, China. Association for Computational L… find our card
  16. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008a. The Penn discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Ass… find our card
  17. Rashmi Prasad, Katherine Forbes Riley, and Alan Lee. 2017. Towards full text shallow discourse relation annotation: Experiments with cross-paragraph implicit relations in the PDTB. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 7–16, Saarbrücken, Germany. Associa… find our card
  18. Rashmi Prasad, Samar Husain, Dipti Sharma, and Aravind Joshi. 2008b. Towards an annotated corpus of discourse relations in Hindi. In Proceedings of the 6th Workshop on Asian Language Resources. find our card
  19. Kechen Qin, Lu Wang, and Joseph Kim. 2017. Joint modeling of content and discourse relations in dialogues. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 974–984, Vancouver, Canada. Association for Computational Linguistics. find
  20. Nianwen Xue and Martha Palmer. 2009. Adding semantic roles to the chinese treebank. Nat. Lang. Eng., 15:143–172. find
  21. Attapol Rutherford, Vera Demberg, and Nianwen Xue. 2017. A systematic study of neural discourse models for implicit discourse relation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 281–291, Valencia, Spai… find our card
  22. Nianwen Xue and Yaqin Yang. 2011. Chinese sentence segmentation as comma classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 631–635, Portland, Oregon, USA. Association for Computational Linguistics. find
  23. Tatjana Scheffler, Berfin Aktaş, Debopam Das, and Manfred Stede. 2019. Annotating shallow discourse relations in twitter conversations. In Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019, pages 50–55, Minneapolis, MN. Association for Computational Linguistics. find
  24. Deniz Zeyrek, Amália Mendes, and Murathan Kurfalı. 2018. Multilingual extension of PDTB-style annotation: The case of TED multilingual discourse bank. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resourc… find
  25. Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. 2019. ERNIE: Enhanced representation through knowledge integration. ArXiv, abs/1904.09223. find our card
  26. Sara Tonelli, Giuseppe Riccardi, Rashmi Prasad, and Aravind Joshi. 2010. Annotation of discourse relations for conversational spoken dialogs. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Associatio… find our card
  27. Elena Voita, Rico Sennrich, and Ivan Titov. 2019. When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1198–1212, Florence,… find our card
  28. Bonnie Webber, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2019. The Penn Discourse Treebank 3.0 Annotation Manual. find
  29. Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-aware neural extractive text summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5021–5031, Online. Association for Computational Linguistics. find our card
  30. Meishan Zhang, Yue Zhang, Wanxiang Che, and Ting Liu. 2013. Chinese parsing exploiting characters. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 125–134, Sofia, Bulgaria. Association for Computational Linguistics. find
  31. Lanjun Zhou, Binyang Li, Zhongyu Wei, and Kam-Fai Wong. 2014. The CUHK discourse TreeBank for Chinese: Annotating explicit discourse connectives for the Chinese TreeBank. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 942–949, Reykjavik, Ic… find
  32. Yuping Zhou and Nianwen Xue. 2012. PDTB-style discourse annotation of Chinese text. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 69–77, Jeju Island, Korea. Association for Computational Linguistics. find
  33. Yuping Zhou and Nianwen Xue. 2015. The chinese discourse treebank: a chinese corpus annotated with discourse relations. Language Resources and Evaluation, 49:397–431. find
  34. Nianwen Xue. 2006. Semantic role labeling of nominalized predicates in Chinese. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 431–438, New York City, USA. Association for Computational Linguistics. find
  35. Nianwen Xue and Martha Palmer. 2003. Annotating the propositions in the Penn Chinese treebank. In Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, pages 47–54, Sapporo, Japan. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/long-2020-ted-cdb-large-scale-chinese/