Elamur Research
☰
home / papers / 2008 / The Importance of Syntactic Parsing and Inference in Semantic Role Labeling

The Importance of Syntactic Parsing and Inference in Semantic Role Labeling

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J08-2005.pdf

Summary

Semantic Role Labeling (SRL) finds, for each verb predicate, the spans that fill roles such as A0, A1 or AM-LOC. Most SRL systems depend on full syntactic parse trees. It was unclear how much this matters compared with shallow syntax (POS tags, chunks, clauses), and at which processing step. The authors build a pipeline with four parts: candidate pruning, argument identification, role classification, and a final global step. That last step is cast as Integer Linear Programming (ILP), which enforces constraints such as no overlapping spans and no repeated core labels. Using PropBank data, they compare two otherwise identical systems that differ only in access to full-parse features. Full parses help most during pruning. The shallow components score well in isolation, but the candidates they pass on make later steps harder, so a stage's precision and recall alone do not tell its real value. Finally, ILP-based joint inference merges systems built on different parsers, giving the best F1 at CoNLL-2005. The study shows where syntax pays off and offers a simple way to combine systems.

Key points

  • The system runs in four steps: candidates are first pruned, arguments are then identified, their roles are next classified, and ILP inference comes last. The inference stage applies hard constraints, for example R-arg/C-arg consistency and illegal argument types per verb.
  • The combined system merges SRL outputs based on the Collins (1999) and Charniak (2001) parsers. It achieved the highest F1 among 19 participants in the CoNLL-2005 shared task.
  • In an earlier chunk-only approach (Gildea and Palmer 2002), overly strict pruning discarded over 60% of true arguments. At CoNLL-2004, the best shallow-syntax system trailed the best full-parse system by roughly 10 F1 points.
  • In PropBank I, core arguments (A0–A5, AA) account for 71.26% of labels, with A0 at 25.39% and A1 at 35.19%. Adjuncts make up 24.90%, C-arg 1.22% and R-arg 2.63%.
  • Future directions include learning constraints from data instead of hand-writing them, and domain adaptation, since performance dropped sharply on out-of-domain test data at CoNLL-2005.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Semantic role labeling

Cited by

147 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 134 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.2008.34.2.257

@article{punyakanok2008importance,
  title = {{The Importance of Syntactic Parsing and Inference in Semantic Role Labeling}},
  author = {Punyakanok, Vasin and Roth, Dan and Yih, Wen-tau},
  journal = {Computational Linguistics},
  year = {2008},
  doi = {10.1162/coli.2008.34.2.257},
  url = {https://doi.org/10.1162/coli.2008.34.2.257}
}

References

47 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baker, Collin F., Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley Framenet project. In Proceedings of COLING-ACL, pages 86–90, Montreal, Canada. find our card
  2. Bikel, Daniel M. 2004. Intricacies of Collins’ parsing model. Computational Linguistics, 30(4):479–511. find
  3. Bishop, Christopher M., 1995. Neural Networks for Pattern Recognition, chapter 6.4: Modelling conditional distributions, page 215. Oxford University Press, Oxford, UK. find
  4. Carlson, Andrew J., Chad M. Cumby, Jeff L. Rosen, and Dan Roth. 1999. The SNoW learning architecture. Technical Report UIUCDCS-R-99-2101, UIUC Computer Science Department. find
  5. Carreras, Xavier and Llúis Màrquez. 2004. Introduction to the CoNLL-2004 shared tasks: Semantic role labeling. In Proceedings of CoNLL-2004, pages 89–97, Boston, MA. find
  6. Carreras, Xavier and Llúis Màrquez. 2005. Introduction to the CoNLL-2005 shared task: Semantic role labeling. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 152–164, Ann Arbor, MI. find our card
  7. Carreras, Xavier, Llúis Màrquez, and Jorge Castro. 2005. Filtering–ranking perceptron learning for partial parsing. Machine Learning, 60:41–71. find
  8. Carreras, Xavier, Llúis Màrquez, Vasin Punyakanok, and Dan Roth. 2002. Learning and inference for clause identification. In Proceedings of the 13th European Conference on Machine Learning (ECML-2002), pages 35–47, Helsinki, Finland. find
  9. Charniak, Eugene. 2001. Immediate-head parsing for language models. In Proceedings of the 39th Annual Meeting of the Association of Computational Linguistics (ACL-2001), pages 116–123, Toulouse, France. find
  10. Chen, John and Owen Rambow. 2003. Use of deep linguistic features for the recognition and labeling of semantic arguments. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing (EMNLP-2003), pages 41–48, Sapporo, Japan. find
  11. Collins, Michael. 1999. Head-driven Statistical Models for Natural Language Parsing. Ph.D. thesis, Computer Science Department, University of Pennsylvania, Philadelphia, PA. find
  12. Dagan, Ido, Yael Karov, and Dan Roth. 1997. Mistake-driven learning in text categorization. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-1997), pages 55–63, Providence, RI. find
  13. Even-Zohar, Yair and Dan Roth. 2001. A sequential model for multi-class classification. In Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing (EMNLP-2001), pages 10–19, Pittsburgh, PA. find
  14. Freund, Yoav and Robert E. Schapire. 1999. Large margin classification using the Perceptron algorithm. Machine Learning, 37(3):277–296. find
  15. Gildea, Daniel and Julia Hockenmaier. 2003. Identifying semantic roles using combinatory categorial grammar. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing (EMNLP-2003), pages 57–64, Sapporo, Japan. find
  16. Gildea, Daniel and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. find our card
  17. Gildea, Daniel and Martha Palmer. 2002. The necessity of parsing for predicate argument recognition. In Proceedings of the 40th Annual Meeting of the Association of Computational Linguistics (ACL-2002), pages 239–246, Philadelphia, PA. find
  18. Golding, Andrew R. and Dan Roth. 1999. A Winnow based approach to context-sensitive spelling correction. Machine Learning, 34(1-3):107–130. find
  19. Grove, Adam J. and Dan Roth. 2001. Linear concepts and hidden variables. Machine Learning, 42(1–2):123–141. find
  20. Guéret, Christelle, Christian Prins, and Marc Sevaux. 2002. Applications of Optimization with Xpress-MP. Dash Optimization. Translated and revised by Susanne Heipcke. http://www.dashoptimization. com/home/downloads/book/booka4.pdf. ↗
  21. Hacioglu, Kadri. 2004. Semantic role labeling using dependency trees. In Proceedings of the 20th International Conference on Computational Linguistics (COLING), Geneva, Switzerland. find
  22. Hacioglu, Kadri, Sameer Pradhan, Wayne Ward, James H. Martin, and Daniel Jurafsky. 2004. Semantic role labeling by tagging syntactic chunks. In Proceedings of CoNLL-2004, pages 110–113, Boston, MA. find
  23. Haghighi, Aria, Kristina Toutanova, and Christopher D. Manning. 2005. A joint model for semantic role labeling. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 173–176, Ann Arbor, MI. find
  24. Kingsbury, Paul and Martha Palmer. 2002. From Treebank to PropBank. In Proceedings of LREC-2002, Las Palmas, Canary Islands, Spain. find our card
  25. Kipper, Karin, Martha Palmer, and Owen Rambow. 2002. Extending PropBank with VerbNet semantic predicates. In Proceedings of Workshop on Applied Interlinguas, Tiburon, CA. find
  26. Koomen, Peter, Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2005. Generalized inference with multiple semantic role labeling systems. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 181–184, Ann Arbor, MI. find
  27. Levin, Beth. 1993. English Verb Classes and Alternations: A Preliminary Investigation. University of Chicago Press, Chicago. find
  28. Levin, Beth and Malka R. Hovav. 1996. From lexical semantics to argument realization. Unpublished manuscript. find
  29. Li, Xin and Dan Roth. 2001. Exploring evidence for shallow parsing. In Proceedings of CoNLL-2001, pages 107–110, Toulouse, France. find
  30. Marcus, Mitchell P., Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  31. Màrquez, Llúis, Jesus Giménez Pere Comas, and Neus Català. 2005. Semantic role labeling as sequential tagging. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 193–196, Ann Arbor, MI. find
  32. Noreen, Eric W. 1989. Computer-Intensive Methods for Testing Hypotheses. New York: John Wiley & Sons. find
  33. Palmer, Martha, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. find our card
  34. Pradhan, Sameer, Kadri Hacioglu, Valerie Krugler, Wayne Ward, James H. Martin, and Daniel Jurafsky. 2005. Support vector learning for semantic argument classification. Machine Learning, 60:11–39. find
  35. Pradhan, Sameer, Kadri Hacioglu, Wayne Ward, James H. Martin, and Daniel Jurafsky. 2003. Semantic role parsing adding semantic structure to unstructured text. In Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM 2003), pages 629–632, Melbourne, FL. find
  36. Pradhan, Sameer, Kadri Hacioglu, Wayne Ward, James H. Martin, and Daniel Jurafsky. 2005. Semantic role chunking combining complementary syntactic views. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 217–220, Ann Arbor, MI. find
  37. Pradhan, Sameer, Wayne Ward, Kadri Hacioglu, James H. Martin, and Daniel Jurafsky. 2004. Shallow semantic parsing using support vector machines. In Proceedings of NAACL-HLT 2004, pages 233–240, Boston, MA. find
  38. Punyakanok, Vasin, Dan Roth, Wen-tau Yih, and Dav Zimak. 2004. Semantic role labeling via integer linear programming inference. In Proceedings the 20th International Conference on Computational Linguistics (COLING), pages 1346–1352, Geneva, Switzerland. find
  39. Punyakanok, Vasin and Dan Roth. 2001. The use of classifiers in sequential inference. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13, pages 995–1001. MIT Press. find
  40. Roth, Dan. 1998. Learning to resolve natural language ambiguities: A unified approach. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pages 806–813, Madison, WI. find
  41. Roth, Dan and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of CoNLL-2004, pages 1–8, Boston, MA. find our card
  42. Roth, Dan and Wen-tau Yih. 2005. Integer linear programming inference for conditional random fields. In Proceedings of the 22nd International Conference on Machine Learning (ICML-2005), pages 737–744, Bonn, Germany. find
  43. Surdeanu, Mihai, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, pages 8–15, Sapporo, Japan. find
  44. Tjong Kim Sang, Erik F. and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of CoNLL-2000 and LLL-2000, pages 127–132, Lisbon, Portugal. find
  45. Xpress-MP. 2004. Dash Optimization. Xpress-MP. http://www. dashoptimization.com. find
  46. Xue, Nianwen and Martha Palmer. 2004. Calibrating features for semantic role labeling. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP-2004), pages 88–94, Barcelona, Spain. find
  47. Zhang, Tong, Fred Damerau, and David Johnson. 2002. Text chunking based on a generalization of Winnow. Journal of Machine Learning Research, 2:615–637. find