Elamur Research⌕
☰
home / papers / 2005 / Exploring Various Knowledge in Relation Extraction

Exploring Various Knowledge in Relation Extraction

Summary

The paper addresses relation extraction as defined in the ACE RDC task: given two entity mentions, decide whether a semantic relation links them and, if so, assign one of 5 types or 24 subtypes. The authors build a feature-based classifier with SVMLight (linear kernel, one-vs-others decomposition). They systematically add lexical cues (words, head words, entity types, mention levels, overlap counts), syntactic cues (base phrase chunks, full parse trees, dependency trees) and semantic resources (WordNet and a Name List). A central design choice is to separate chunk-derived features from full-parse features so that each source's contribution can be measured on its own. Chunking turns out to supply most of the syntactic benefit, and full parsing adds little. The authors explain this by noting that most ACE relations join mentions that sit very close together. On the 24 subtypes the system beats the previous best feature-based system by roughly 3 F. On the 5 types it beats tree-kernel methods by over 20 F. The findings suggest that cheap shallow features can stand in for costly deep parsing.

Key points

  • Setup: ACE corpus with 5 relation types and 24 subtypes, classified using SVMLight with a linear kernel and one-vs-others multi-class decomposition. Experiments assume perfect entity detection and tracking (EDT).
  • Results on the test set:
    • Relation detection reaches 84.8 / 66.7 / 74.7 (P / R / F).
    • Classification into the 5 types reaches 68.0 F.
    • Classification into the 24 subtypes reaches 55.5 F.
  • Baselines: Kambhatla (2004) scored 52.8 F on subtypes. The Culotta et al. (2004) tree kernel scored 63.2 F on detection and 45.8 F on types.
  • Relations are mostly local. Of 9,830 training relations, 4,163 have no words between the two mentions, and 7,694 have no other mention in between.
  • Error analysis: 73% of errors (627/864) come from detection rather than characterization, so detection is the main bottleneck. "AT" is singled out as a particularly hard type.

Interesting ideas

  • I1. Separate feature families to attribute their value. Splitting chunking features from full-parse features showed that the expensive component added little. The same ablation design could tell a low-resource language project, such as Ukrainian, whether a parser is worth the cost or a chunker is enough.
  • I2. Check the distance distribution before choosing deep syntax. Counting the words and mentions between argument pairs explains why shallow features work. This simple corpus statistic can guide architecture decisions for any pairwise extraction task.
  • I3. Positional binning of context words. Context words are split into slots: the first and last word between the mentions, the remaining in-between words, and the two words on each outer side. This keeps word order information within a linear model and transfers easily to other span-pair classification problems.
  • I4. Resolve pronouns before extracting features. A pronominal mention is replaced by its nearest non-pronominal antecedent from the coreference chain, so uninformative words like "it" do not dilute the features. This is useful wherever arguments are often pronouns.
  • I5. Trim mention spans to reduce noise. Each mention is cut from the start of its extent to the end of its head, which drops trailing modifiers such as "of McDonald". The idea is a cheap way to clean noisy argument spans in other annotation schemes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

173 papers of our library cite it (of the 140884 whose bibliographies we read); 13 of them have a page here, 160 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1219840.1219893

@inproceedings{zhou2005exploring,
  title = {{Exploring Various Knowledge in Relation Extraction}},
  author = {Zhou, GuoDong and Su, Jian and Zhang, Jie and Zhang, Min},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219893},
  url = {https://doi.org/10.3115/1219840.1219893}
}

References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agichtein E. and Gravano L. (2000). Snowball: Extracting relations from large plain text collections. In Proceedings of 5 th ACM International Conference on Digital Libraries. 4-7 June 2000. San Antonio, TX. find
  2. Brin S. (1998). Extracting patterns and relations from the World Wide Web. In Proceedings of WebDB workshop at 6 th International Conference on Extending DataBase Technology (EDBT’1998).23-27 March 1998, Valencia, Spain Collins M. (1999). Head-driven statistical models for natural language parsing.… find
  3. Collins M. and Duffy N. (2002). Covolution kernels for natural language. In Dietterich T.G., Becker S. and Ghahramani Z. editors. Advances in Neural Information Processing Systems 14. Cambridge, MA. find
  4. Culotta A. and Sorensen J. (2004). Dependency tree kernels for relation extraction. In Proceedings of 42 th Annual Meeting of the Association for Computational Linguistics. 21-26 July 2004. find our card
  5. Barcelona, Spain Cumby C.M. and Roth D. (2003). On kernel methods for relation learning. In Fawcett T. and Mishra N. editors. In Proceedings of 20 th International Conference on Machine Learning (ICML’2003). 21-24 Aug 2003. Washington D.C. USA. AAAI Press. find
  6. Haussler D. (1999). Covention kernels on discrete structures. Technical Report UCS-CRL-99-10. University of California, Santa Cruz. find
  7. Joachims T. (1998). Text categorization with Support Vector Machines: Learning with many relevant features. In Proceedings of European Conference on Machine Learning(ECML’1998). 21-23 April 1998. find our card
  8. Chemnitz, Germany Miller G.A. (1990). WordNet: An online lexical database. International Journal of Lexicography. 3(4):235-312. find
  9. Miller S., Fox H., Ramshaw L. and Weischedel R. (2000). A novel use of statistical parsing to extract information from text. In Proceedings of 6 th Applied Natural Language Processing Conference. 29 April - 4 May 2000, Seattle, USA MUC-7. (1998). Proceedings of the 7 th Message Understanding Confere… find
  10. Kambhatla N. (2004). Combining lexical, syntactic and semantic features with Maximum Entropy models for extracting relations. In Proceedings of 42 th Annual Meeting of the Association for Computational Linguistics. 21-26 July 2004. Barcelona, Spain. find
  11. Roth D. and Yih W.T. (2002). Probabilistic reasoning for entities and relation recognition. In Proceedings of 19 th International Conference on Computational Linguistics(CoLING’2002). Taiwan. find
  12. Vapnik V. (1998). Statistical Learning Theory. Whiley, Chichester, GB. find
  13. Zelenko D., Aone C. and Richardella. (2003). Kernel methods for relation extraction. Journal of Machine Learning Research. pp1083-1106. find our card
  14. Zhang Z. (2004). Weekly-supervised relation classification for Information Extraction. In Proceedings of ACM 13 th Conference on Information and Knowledge Management (CIKM’2004). 8-13 Nov 2004. Washington D.C., USA. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2005-exploring-various-knowledge-relation-extraction/