Elamur Research⌕
☰
home / papers / 2013 / Annotating the meaning of discourse connectives by looking at their translation: The translation-spotting technique

Annotating the meaning of discourse connectives by looking at their translation: The translation-spotting technique

Summary

Discourse connectives such as since or while can signal several different coherence relations, and machine translation systems often choose the wrong equivalent. Within the Swiss COMTIS project, which aims to improve statistical machine translation between English and French, the authors needed manually labelled reference data to train connective classifiers. Classic sense annotation, where judges choose labels from an inventory like the Penn Discourse Treebank (PDTB) hierarchy, yielded only modest inter-annotator agreement. As an alternative, they propose translation spotting. Instead of reasoning explicitly about abstract senses, annotators identify how each connective was rendered by professional translators in large parallel corpora. When the target language does not disambiguate directly, the observed translations are grouped into sense clusters, based on native speakers' judgements of which target connectives are interchangeable in context. The authors argue that this produces more reliable reference data than sense annotation. They also show that it reveals fine-grained distinctions, for example among causal connectives, and provides empirical material for contrastive studies of discourse relations.

Key points

  • The PDTB, built on the WSJ corpus of about 1M tokens, contains 18,459 explicit and 16,053 implicit annotated relations. Its sense hierarchy has 4 top-level classes, 16 second-level types and 23 third-level subtypes.
  • Agreement in the PDTB is 92% at the top level but falls to 77% at the finest level. For 100 tokens of while, overall agreement was only 67%.
  • Of roughly 100 explicit PDTB connectives, 29 always receive a single sense (412 occurrences). Another 52 carry labels from different top-level classes, and these were the authors' annotation focus.
  • The method has limits. It depends on translators' choices and needs large amounts of data. It also cannot separate senses that the target language expresses with a single connective.
  • Double labels such as temporal-causal for since and background-contrast for alors que proved useful in classification. A three-way classifier nearly matched the two-way one for since and outperformed it for alors que.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 8 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2013.204

@inproceedings{cartoni2013annotating,
  title = {{Annotating the meaning of discourse connectives by looking at their translation: The translation-spotting technique}},
  author = {Cartoni, Bruno and Zufferey, Sandrine and Meyer, Thomas},
  booktitle = {Dialogue \& Discourse Volume 4},
  year = {2013},
  doi = {10.5087/dad.2013.204},
  url = {https://doi.org/10.5087/dad.2013.204}
}

References

50 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ron Artstein and Massimo Poesio (2008). Inter-coder agreement for computational linguistics. Computational Linguistics 34(4):555-596. find
  2. Mona Baker (1993). Corpus linguistics and translation studies: Implications and applications. In M.Baker et al. (Eds), Text and Technology: In honor of John Sinclair. John Benjamins, Amsterdam/Philadelphia. find
  3. Petra Bayerl and Karsten Paul (2011). What Determines Inter-Coder Agreement in Manual Annotations? A Meta-Analytic Investigation. Computational Linguistics 37(4):699-725. find
  4. Bergljot Behrens and Cathrine Fabricius-Hansen (2003). Translation equivalents as empirical data for semantic/pragmatic theory. In Jaszczolt K, Turner Jen (editors), Meaning through Language Contrast. Amsterdam: Benjamins. 463-477. find
  5. Diane Blakemore (2002) Meaning and relevance: the semantics and pragmatics of discourse markers. Cambridge University Press, Cambridge, USA. find
  6. Jean Carletta (1996). Assessing agreement on classification tasks: The Kappa statistic. Computational Linguistics, 22(2):249–254. find our card
  7. Bruno Cartoni and Thomas Meyer (2012). Extracting Directional and Comparable Corpora from a Multilingual Corpus for Translation Studies. In Proceedings of LREC 2012, pages 2132-2137, Istanbul, Turkey. find
  8. Bruno Cartoni, Sandrine Zufferey, Thomas Meyer and Andrei Popescu-Belis (2011). How Comparable are Parallel Corpora? Measuring the Distribution of General Vocabulary and Connectives. In Proceedings of 4th Workshop on Building and Using Comparable Corpora, pages 78-86, Portland, USA. find
  9. Eugene Charniak and Mark Johnson (2005). Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL) pages 173–180. Ann Arbor, MI. find our card
  10. Laurance Danlos and Charlotte Roze (2011). Traduction (automatique) des connecteurs de discours. In Proceedings of TALN 2011, Montpellier, France. find
  11. Liesbeth Degand (2004). Contrastive analyses, translation, and speaker involvement: The case of puisque and aangezien. In M. Achard and S. Kemmer (Eds.), Language, culture and mind, pages 1-20, Stanford: CSLI Publications. find
  12. Liesbeth Degand and Benjamin Fagard (2012). Competing connectives in the causal domain: French car and parce que. Journal of Pragmatics 44(2): 154-168. find
  13. Liesbeth Degand and Henk Pander Maat (2003). A contrastive study of Dutch and French causal connectives on the Speaker Involvement Scale. In A. Verhagen and J. Maarten van de Weijer (editors), Usage-based approaches to Dutch, pages 175-199, LOT, Utrecht. find
  14. Helge Dyvik (1998). A translational basis for semantics. In Johansson, Stig & Signe Okselfjell (eds) Corpora and Crosslinguistic Research: Theory, Method and Case Studies, pages 51-86, Amsterdam: Rodopi. find
  15. K. Anders Ericsson and Herbert Simon (1980) Verbal reports as data. Psychological Review 87(3):215-251. find
  16. Michael Halliday and Ruqaiya Hasan (1976). Cohesion in English. Longman, London, UK. find
  17. Ed Hovy (1990). Parsimonious and profligate approaches to the question of discourse structure relations. In Proceedings of the Fifth International Workshop on Natural Language Generation. Pittsburgh, Pennsylvania. find
  18. Stéphane Huet, Julien Bourdaillet and Philippe Langlais (2009). Intégration de l’alignement de mots dans le concordancier bilingue TransSearch. In Proceedings of TALN’09, Senlis, France. find
  19. Philipp Koehn (2005). Europarl: A parallel corpus for statistical machine translation. In Proceedings of the Tenth Machine Translation Summit, September 13-15, pages 79-86, Phukhet, Thailand. find
  20. Alistair Knott and Robert Dale (1994). Using linguistic phenomena to motivate a set of set of coherence relations. Discourse processes 18(1):35-62. find
  21. Lambda-l, Groupe (1975). Car, parce que, puisque. Revue Romane 10:248-280. find
  22. Richard J. Landis and Gary G. Koch (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. find
  23. William Mann and Sandra Thomson (1992). Relational Discourse Structure: A Comparison of Approaches to Structuring Text by 'Contrast'. In Hwang S. & Merrifield W . (Eds.), Language in Context: Essays for Robert E. Longacre. SIL, pages 19-45, Dallas, USA. find
  24. Christopher Manning and Dan Klein (2003). Optimization, MaxEnt Models, and Conditional Estimation without Magic. Tutorial at HLT-NAACL and 41st ACL conferences. Edmonton, Canada and Sapporo, Japan. find
  25. Thomas Meyer and Andrei Popescu-Belis (2012). Using Sense-labeled Discourse Connectives for Statistical Machine Translation. In Proceedings of the EACL 2012 Workshop on Hybrid Approaches to Machine Translation (HyTra), Avignon, France, pp. 129-138. find our card
  26. Thomas Meyer, Andrei Popescu-Belis, Sandrine Zufferey and Bruno Cartoni (2011). Multilingual Annotation and Disambiguation of Discourse Connectives for Machine Translation. In Proceedings of 12 th SIGdial Meeting on Discourse and Dialog, pages 194-203, Portland, USA. find our card
  27. Eleni Miltsakaki, Nikhil Dinesh, Rashmi Prasad, Aravind Joshi and Bonnie Webber (2005). Experiments on sense annotations and sense disambiguation of discourse connectives. In Proceedings of the TLT 2005 (4th Workshop on Treebanks and Linguistic Theories), Barcelona, Spain. find
  28. Eleni Miltsakaki, Livio Robaldo, Alan Lee and Aravind Joshi (2008). Sense Annotation in the Penn Discourse Treebank. In Alexander Gelbukh (editor), Computational Linguistics and Intelligent Text Processing. Lecture Notes in Computer Science, pages 275-286, Springer Berlin / Heidelberg. find
  29. Jacques Moeschler (2002). Connecteurs, encodage conceptuel et encodage procédural. Cahiers de linguistique française 24:265-292. find
  30. Dick Noël (2003). Translations as evidence for semantics: An illustration. Linguistics 41(4):757-785. find
  31. The PDTB Research Group (2007). The Penn Discourse Treebank 2.0 Annotation Manual. IRCS Technical Reports Series, 99p. find
  32. Mirna Pit (2007). Cross-linguistic analyses of backward causal connectives in Dutch, German and French. Languages in Contrast, 7(1), 53–82. find
  33. Andrei Popescu-Belis, Bruno Cartoni, Andrea Gesmundo, James Henderson, Cristina Hulea, Paola Merlo, Thomas Meyer, Jacques Moeschler and Sandrine Zufferey (2011). Improving MT coherence through text-level processing of input texts: the COMTIS project. In Proceedings of Tralogy 2011 (Translation Caree… find
  34. Andrei Popescu-Belis, Thomas Meyer, Jeevanthi Liyanapathirana, Bruno Cartoni and Sandrine Zufferey (2012). Discourse-level Annotation over Europarl for Machine Translation: Connectives and Pronouns. In Proceedings of LREC 2012, pages 2716-2720, Istanbul, Turkey. find
  35. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi and Bonnie Webber (2008). The Penn Discourse Treebank 2.0. In Proceedings of 6th International Conference on Language Resources and Evaluation (LREC), pages 2961–2968, Marrakech, Morocco. find our card
  36. Eddy Roulet, Antoine Auchlin, Jacques Moeschler, Christian Rubattel and Marianne Schelling, (1985). L'articulation du discours en français contemporain. Peter Lang, Berne, Switzerland. find
  37. Charlotte Roze, Laurance Danlos and Philippe Muller (2010). LEXCONN: a French Lexicon of Discourse Connectives. In Proceedings of Multidisciplinary Approaches to Discourse (MAD 2010), Moissac, France. find our card
  38. Ted Sanders (1997). Semantic and pragmatic sources of coherence: On the categorization of coherence relations in context. Discourse Processes 24:119-147. find
  39. Ted Sanders, Wilbert Spooren and Leo Noordman (1992). Towards a taxonomy of coherence relations. Discourse Processes 15, 1-36. find
  40. Ted Sanders and Ninke Stukker, (2012). Causal connectives in discourse: a cross-linguistic perspective. Special issue of Journal of Pragmatics 44 (2):131-137. find
  41. Ted Sanders T. and Eve Sweetser (2009). Causal categories in discourse and cognition. Walter de Gruyter, Berlin, Germany. find
  42. Michel Simard, (2003). Translation spotting for translation memories. find
  43. HLT-NAACL 2003, Workshop: Building and Using Parallel Texts Data Driven Machine Translation and Beyond. find
  44. Wilbert Spooren and Liesbeth Degand (2010). Coding coherence relations: Reliability and validity. Corpus Linguistics and Linguistic Theory 6 (2): 241-266. find
  45. Marc Verhagen, Inderjeet Mani, Roser Sauri, Jessica Littman, Robert Knippen, Seok Bae Jang, Anna Rumshisky, John Phillips, James Pustejovsky (2005). Automating Temporal Annotation with {TARSQI}. Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics (ACL), Demo Sessi… find
  46. Jean Véronis and Philippe Langlais (2000). Evaluation of parallel text alignment systems: The arcade project. In Parallel Text Processing. Kluwer Academic Publishers, Text Speech and Language Technology Series: 369-388. find
  47. Bonnie Webber and Aravind Joshi (2012). Discourse Structure and Computation: Past, Present and Future. Proceedings of the ACL-2012 Special Workshop on Rediscovering 50 Years of Discoveries. Jeju, Republic of Korea. 42–54, Deirdre Wilson (2011). The conceptual-procedural distinction: Past, present an… find
  48. Sandrine Zufferey (2012). Car, parce que, puisque revisited. Three empirical studies on French connectives. Journal of Pragmatics 44(2), 138-153. find
  49. Sandrine Zufferey and Bruno Cartoni (2012). English and French causal connectives in contrast. Languages in Contrast 12(2):232-250. find
  50. Sandrine Zufferey, Liesbeth Degand, Andrei Popescu-Belis and Ted Sanders (2012). Empirical validations of multilingual annotation schemes for discourse relations. Eighth Joint ISO-ACL/SIGSEM Workshop on Interoperable Semantic Annotation, pages 77-84, Pisa, Italy. find

↑ back to top