Elamur Research⌕
☰
home / papers / 2016 / Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network

Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network

Summary

The paper addresses intra-sentential subject zero anaphora resolution in Japanese. This is the task of linking an omitted subject of a predicate to an antecedent in the same sentence, a frequent phenomenon in pro-drop languages. The authors use a Multi-column Convolutional Neural Network (MCNN). It scores every predicate–candidate antecedent pair, with separate columns fed by word sequences taken from the surface sentence and from its dependency tree. This design is inspired by Centering Theory and earlier feature-based work, but it largely replaces manual feature engineering. Final decisions are made greedily: pairs are ranked by probability, and each empty subject slot takes the best candidate above a threshold.

The approach was compared with Ouchi et al.'s (2015) global-optimization system, which was the state of the art. The MCNN had a lower F-score but higher precision at every recall level. The authors argue that this reliability matters more than F-score for practical tools such as information extraction and machine translation.

Key points

  • Motivation figures: about 60% of subjects in Japanese newspaper text are omitted, and subjects make up 85% of intra-sentential zero anaphora in the authors' data.
  • The MCNN combines four column sets: BASE, SURFSEQ, DEPTREE and PREDCONTEXT. Using all four gave the best average precision.
  • At 0.25 recall, the MCNN reached roughly 0.8 precision versus about 0.65 for Ouchi's method. At 0.4 recall, the MCNN stayed near 0.7 precision.
  • Ouchi's system overall reached only 0.61 precision, and its precision rose to 0.80 only at recall below 0.01.
  • In an instance-wise evaluation, average precision was 0.602 vs 0.341 for anaphoric cases and 0.337 vs 0.163 for cataphoric cases, comparing the full MCNN with a single-column CNN. Cataphora remains the weak spot.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 12 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d16-1132

@inproceedings{iida2016intra,
  title = {{Intra-Sentential Subject Zero Anaphora Resolution using Multi-Column Convolutional Neural Network}},
  author = {Iida, Ryu and Torisawa, Kentaro and Oh, Jong-Hoon and Kruengkrai, Canasai and Kloetzer, Julien},
  booktitle = {Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing},
  year = {2016},
  doi = {10.18653/v1/d16-1132},
  url = {https://doi.org/10.18653/v1/d16-1132}
}

References

36 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Berg-eron, Nicolas Bouchard, and Yoshua Bengio. 2012. Theano: new features and speed improvements. In Proceedings of the NIPS 2012 Workshop: Deep Learning and Unsupervised Feature Learning. find
  2. Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005. Learning a similarity metric discriminatively, with application to face verification. In Proceedings of Computer Vision and Pattern Recognition Conference, pages 539–546. Dan Claudiu Cireşan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column d… find
  3. Li Dong, Furu Wei, Ming Zhou, and Ke Xu. 2015. Question answering over freebase with multi-column convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages… find
  4. Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 626–63… find
  5. Barbara J. Grosz, Scott Weinstein, and Aravind K. Joshi. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225. find our card
  6. Yuta Hayashibe, Mamoru Komachi, and Yuji Matsumoto. 2011. Japanese predicate argument structure analysis exploiting argument position and type. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 201–209. Geoffrey E . Hinton, Nitish Srivastava, Alex Krizhevsky,… find
  7. Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional neural network architectures for matching natural language sentences. In Processings of Advances in Neural Information Processing Systems 27 (NIPS 2014), pages 2042–2050. find
  8. Ryu Iida and Massimo Poesio. 2011. A cross-lingual ILP solution to zero anaphora resolution. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 804–813. find our card
  9. Ryu Iida, Kentaro Inui, Hiroya Takamura, and Yuji Matsumoto. 2003. Incorporating contextual cues in trainable models for coreference resolution. In Proceedings of the 2003 EACL Workshop on The Computational Treatment of Anaphora, pages 23–30. find
  10. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2006. Exploiting syntactic patterns as clues in zero-anaphora resolution. In Processings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 625–632. find our card
  11. Ryu Iida, Mamoru Komachi, Kentaro Inui, and Yuji Matsumoto. 2007. Annotating a Japanese text corpus with predicate-argument and coreference relations. In Proceedings of the ACL Workshop: ‘Linguistic Annotation Workshop’, pages 132–139. find our card
  12. Ryu Iida, Kentaro Torisawa, Chikara Hashimoto, Jong-Hoon Oh, and Julien Kloetzer. 2015. Intra-sentential zero anaphora resolution using subject sharing recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2179–2189. find our card
  13. Kenji Imamura, Kuniko Saito, and Tomoko Izumi. 2009. Discriminative approach to predicate-argument structure analysis with zero-anaphora resolution. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing,… find our card
  14. Hideki Isozaki and Tsutomu Hirao. 2003. Japanese zero pronoun resolution based on ranking rules and machine learning. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 184–191. find our card
  15. Rie Johnson and Tong Zhang. 2015. Effective use of word order for text categorization with convolutional neural networks. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 103–112. find
  16. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 655–665. find our card
  17. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1746–1751. find our card
  18. Taku Kudo, Kaoru Yamamoto, and Yuji Matsumoto. 2004. Applying conditional random fields to Japanese morphological analysis. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 230–237. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gr… find our card
  19. Chunyang Liu, Wenbo Sun, Wenhan Chao, and Wanxiang Che. 2013. Convolution neural network for relation extraction. In Proceedings of the 9th International Conference of Advanced Data Mining and Applications, pages 231–242. find
  20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 3111–3119. find
  21. Shigeko Nariyama. 2002. Grammar for ellipsis resolution in Japanese. In Proceedings of the 9th International Conference on Theoretical and Methodological Issues in Machine Translation, pages 135–145. find
  22. Vincent Ng. 2004. Learning noun phrase anaphoricity to improve conference resolution: Issues in representation and optimization. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics, pages 151–158. find our card
  23. Thien Huu Nguyen and Ralph Grishman. 2015. Relation extraction: Perspective from convolutional neural networks. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 39–48. find
  24. Manabu Okumura and Kouji Tamura. 1996. Zero pronoun resolution in Japanese discourse based on centering theory. In Proceedings of The 16th International Conference on Computational Linguistics, pages 871– 876. find our card
  25. Hiroki Ouchi, Hiroyuki Shindo, Kevin Duh, and Yuji Matsumoto. 2015. Joint case argument identification for Japanese predicate argument structure analysis. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural… find
  26. Ryohei Sasano and Sadao Kurohashi. 2011. A discriminative approach to Japanese zero anaphora resolution with large-scale lexicalized case frames. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 758–766. find our card
  27. Ryohei Sasano, Daisuke Kawahara, and Sadao Kurohashi. 2008. A fully-lexicalized probabilistic model for Japanese zero anaphora resolution. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 769–776. find our card
  28. Dominik Scherer, Andreas Müller, and Sven Behnke. 2010. Evaluation of pooling operations in convolutional architectures for object recognition. In Proceedings of the 20th International Conference on Artificial Neural Networks, pages 92–101. find
  29. Kazuhiro Seki, Atsushi Fujii, and Tetsuya Ishikawa. 2002. A probabilistic method for analyzing Japanese anaphora integrating zero pronoun detection and resolution. In Processings of the 19th International Conference on Computational Linguistics, pages 1–7. find our card
  30. Hirotoshi Taira, Sanae Fujita, and Masaaki Nagata. 2008. A Japanese predicate argument structure analysis using decision lists. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 523–532. find
  31. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… find our card
  32. Katsumasa Yoshikawa, Masayuki Asahara, and Yuji Matsumoto. 2011. Jointly extracting Japanese predicate-argument relation with Markov Logic. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1125–1133. find
  33. Naoki Yoshinaga and Masaru Kitsuregawa. 2009. Polynomial to linear: Efficient classification with conjunctive features. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1542–1551. find
  34. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. In arXiv:1212.5701 (Dec 27, 2012). find our card
  35. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of the 25th International Conference on Computational Linguistics, pages 2335–2344. find
  36. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753–1762. find

↑ back to top