Elamur Research⌕
☰
home / papers / 2016 / CAMR at SemEval-2016 Task 8: An Extended Transition-based AMR Parser

CAMR at SemEval-2016 Task 8: An Extended Transition-based AMR Parser

Summary

The paper presents the authors' entry to SemEval-2016 Task 8 (Meaning Representation Parsing). AMR parsers still lag well behind syntactic parsers, which limits their use downstream. The team starts from CAMR, their transition-based parser. CAMR first produces a dependency tree, using a Charniak parser retrained on OntoNotes v5.0. It then turns that tree into an AMR graph through a small set of edge, node and concept-inference actions.

The work adds new feature sources:

  • the 18 fine-grained OntoNotes named entity types;
  • the ISI verbalization list, used to repair concept alignments that the JAMR aligner misses;
  • semantic role labels and frameset predictions from ASSERT, with IMS used for frameset disambiguation.

As a post-processing step, the RPI AMR-based wikifier links entities to Wikipedia. Brown cluster features were also tested but lowered accuracy, so they were dropped.

On the LDC2015E86 development set, the combined extensions raise Smatch F1 by about two points. The official blind test score is 62%. The paper shows that external semantic resources and wikification can improve transition-based AMR parsing.

Key points

  • Development set (LDC2015E86) Smatch F1:
    • baseline: 65.27
    • +VERB: 65.82
    • +VERB+RNE: 66.01
    • +SRL: 66.56
    • +WIKI: 67.33
    • Adding BROWN features to VERB lowered F1 to 65.62.
  • The OntoNotes-trained Charniak parser gave slightly better AMR results than the WSJ-trained one (65.27 vs. 64.70 F1).
  • On the SemEval test set the system reached 66.54 F1. On the blind test set it reached 62.01, a drop of roughly 4 points that the authors attribute to the blind data being harder.
  • LDC2015E86 adds wikification, which Smatch takes into account. Its scores are therefore not directly comparable with those reported on the earlier releases LDC2013E117 and LDC2014T12.
  • On the full LDC2014T12 test set, without wikification, the extended parser scored 66.51 F1, compared with 65.89 for the Wang et al. (2015a) version.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation

Topics: Dependency parsing

Cited by

32 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 28 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s16-1181

@inproceedings{wang2016camr,
  title = {{CAMR at SemEval-2016 Task 8: An Extended Transition-based AMR Parser}},
  author = {Wang, Chuan and Pradhan, Sameer and Pan, Xiaoman and Ji, Heng and Xue, Nianwen},
  booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
  year = {2016},
  doi = {10.18653/v1/s16-1181},
  url = {https://doi.org/10.18653/v1/s16-1181}
}

References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yoav Artzi, Kenton Lee, and Luke Zettlemoyer. 2015. Broad-coverage ccg semantic parsing with amr. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1699–1710, Lisbon, Portugal, September. Association for Computational Linguistics. find
  2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186. Association for Computational Linguistics. find our card
  3. Shu Cai and Kevin Knight. 2013. Smatch: an evaluation metric for semantic feature structures. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 748–752. Association for Computational Linguistics. find
  4. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and maxent discriminative reranking. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ’05, pages 173–180, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  5. Jeffrey Flanigan, Sam Thomson, Jaime Carbonell, Chris Dyer, and Noah A. Smith. 2014. A discriminative graph-based parser for the abstract meaning representation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1426–1436, Baltimore, Maryland, June. Association for Computational Linguistics. find
  6. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. find our card
  7. Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajič. 2005. Non-projective dependency parsing using spanning tree algorithms. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, pages 523–530, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  8. Olena Medelyan and Ian H Witten. 2008. Domain-independent automatic keyphrase indexing with small training sets. Journal of the American Society for Information Science and Technology, 59(7):1026–1040. find
  9. Xiaoman Pan, Taylor Cassidy, Ulf Hermjakob, Heng Ji, and Kevin Knight. 2015. Unsupervised entity linking with abstract meaning representation. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1130–1139, Denver, Colorado, May–June. Association for Computational Linguistics. find
  10. Xiaochang Peng, Linfeng Song, and Daniel Gildea. 2015. A synchronous hyperedge replacement grammar based approach for AMR parsing. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning. find
  11. Sameer Pradhan, Wayne Ward, Kadri Hacioglu, James Martin, and Dan Jurafsky. 2004. Shallow semantic parsing using support vector machines. In Proceedings of the Human Language Technology Conference/North American chapter of the Association of Computational Linguistics (HLT/NAACL), Boston, MA, May. find our card
  12. Sameer Pradhan, Kadri Hacioglu, Valerie Krugler, Wayne Ward, James Martin, and Dan Jurafsky. 2005. Support vector learning for semantic argument classification. Machine Learning Journal, 60(1):11–39. find
  13. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using OntoNotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 143–152, Sofia, Bulgaria, August. find
  14. Michael Pust, Ulf Hermjakob, Kevin Knight, Daniel Marcu, and Jonathan May. 2015. Parsing english into abstract meaning representation using syntax-based machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1143–1154, Lisbon, Portugal, September. Association for Computational Linguistics. find our card
  15. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384–394. Association for Computational Linguistics. find our card
  16. Chuan Wang, Nianwen Xue, and Sameer Pradhan. 2015a. Boosting transition-based amr parsing with refined actions and auxiliary analyzers. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 857–862, Beijing, China, July. Association for Computational Linguistics. find our card
  17. Chuan Wang, Nianwen Xue, and Sameer Pradhan. 2015b. A transition-based algorithm for amr parsing. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 366–375, Denver, Colorado, May–June. Association for Computational Linguistics. find
  18. Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011. OntoNotes: A large training corpus for enhanced processing. Springer. find
  19. Zhi Zhong and Hwee Tou Ng. 2010. It makes sense: A wide-coverage word sense disambiguation system for free text. In Proceedings of the ACL 2010 System Demonstrations, pages 78–83, Uppsala, Sweden. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2016-camr-semeval-2016-task-8/