Using Lexical Dependency and Ontological Knowledge to Improve a Detailed Syntactic and Semantic Tagger of English

What it is about

The paper tags each word with a syntactic tag and a broad semantic class at once, using a maximum-entropy tagger with a beam search from left to right. The data is the ATR General English Treebank, about 518,000 words with a tagset of roughly 165 syntactic tags and 42 plus 29 semantic classes for nouns, adjectives, adverbs and verbs. Two extra knowledge sources are added as binary features: lexical dependencies from a parser (a link-grammar parser and Collins' phrase-structure parser), and the position of a word in a hierarchy, either WordNet hypernyms or an automatic clustering of 50 million words. The best model reaches 84.90% accuracy, against an 82.58% baseline, and raises accuracy on words unseen in training (OOV) from 30.58% to 37.02%. Dependency features help most on verbs, and the ontology features help most on OOV words; the two gains add up.

Ideas

• I1. en/src/ptag.rs and en/src/tag.rs: add a feature from the head and dependents of each word, taken from a UD parse of the same sentence, as a second pass. Keep the feature order in BTreeMap form, and measure UPOS and feature accuracy on UD EWT dev, run twice.
• I2. Use a hypernym chain from world/src/dictionary.rs as back-off features for words the tagger has not seen. Report OOV accuracy separately, because the paper shows that this is where the ontology helps.
• I3. Negative control: shuffle the dependency labels in the features. If the gain stays, the feature is not carrying the signal the paper claims.

For Mova

The chapters are languages/en/10-word-classes.md (tag classes) and languages/en/19-lexicon-mwe-register.md (lexicon and senses). The code is en/src/ptag.rs, en/src/tag.rs and world/src/dictionary.rs. The measures: tag and feature accuracy on UD EWT dev, the OOV split, and the same run twice with identical output. The paper's maximum-entropy model is not an idea for Mova; the feature design and the OOV back-off are.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1273073.1273101

@inproceedings{finch2006using,
  title = {{Using Lexical Dependency and Ontological Knowledge to Improve a Detailed Syntactic and Semantic Tagger of English}},
  author = {Finch, Andrew and Black, Ezra and Hwang, Young-Sook and Sumita, Eiichiro},
  booktitle = {Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions},
  year = {2006},
  doi = {10.3115/1273073.1273101},
  url = {https://doi.org/10.3115/1273073.1273101}
}

References

21 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Black and A. Finch. 2001. Developing and proving effective broad-coverage semantic-and-syntactic tagsets for natural language: The atr approach. In Proceedings of ICCPOL-2001. find
2. E. Black, S. Eubank, H. Kashioka, R. Garside, G. Leech, and D. Magerman. 1996a. Beyond skeleton parsing: producing a comprehensive large– scale general–english treebank with full grammatical analysis. In Proceedings of the 16th Annual Conference on Computational Linguistics, pages 107– 112, Copenhagen. find
3. E. Black, S. Eubank, H. Kashioka, and J. Saia. 1996b. Reinventing part-of-speech tagging. Journal of Natural Language Processing (Japan), 5:1. find
4. Ezra Black, Andrew Finch, and Hideki Kashioka. 1998. Trigger-pair predictors in parsing and tagging. In Proceedings, 36th Annual Meeting of the Association for Computational Linguistics, 17th Annual Conference on Computational Linguistics, Montreal, Canada. find
5. Michael John Collins. 1996. A new statistical parser based on bigram lexical dependencies. In Arivind Joshi and Martha Palmer, editors, Proceedings of the Thirty-Fourth Annual Meeting of the Association for Computational Linguistics, pages 184–191, San Francisco. Morgan Kaufmann Publishers. find
6. C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
7. Dennis Grinberg, John Lafferty, and Daniel Sleator. 1995. A robust parsing algorithm for LINK grammars. Technical Report CMU-CS-TR-95-125, CMU, Pittsburgh, PA. find
8. J. Kupiec. 1992. Robust part-of-speech tagging using a hidden markov model. Computer Speech and Language, 6:225–242. find
9. A. K. Lamjiri, O. El Demerdash, and L.Kosseim. 2004. Simple features for statistical word sense disambiguation. In Proc. ACL 2004 – Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text (Senseval-3), Barcelona, Spain, July. ACL-2004. OOV’s 30.58 30.92 30.92 31.24 35.08 36.18 37.02 Nouns 68.47 68.18 69.16 69.36 71.89 72.28 72.80 Verbs 74.32 74.96 77.21 75.78 75.83 76.29 78.36 Adj/Adv 70.99 73.02 73.52 72.62 75.34 74.47 76.16 find
10. C. Li and H. Li. 2002. Word translation disambiguation using bilingual bootstrapping. find
11. Xiaobin Li, Stan Szpakowicz, and Stan Matwin. 1995. A wordnet-based algorithm for word sense disambiguation. In IJCAI, pages 1368–1374. find
12. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1994. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330. find our card
13. B. Merialdo. 1994. Tagging english text with a probabilistic model. Computational Linguistics, 20(2):155–172. find our card
14. Rada Mihalcea and Dan I. Moldovan. 1998. Word sense disambiguation based on semantic density. In Sanda Harabagiu, editor, Use of WordNet in Natural Language Processing Systems: Proceedings of the Conference, pages 16–22. Association for Computational Linguistics, Somerset, New Jersey. find
15. I. Nancy and V. Jean. 1998. Word sense disambiguation: The state of the art. Computational Linguistics, 24:1:1–40. find
16. G. Ramakrishnan and B. Prithviraj. 2004. Soft word sense disambiguation. In International Conference on Global Wordnet (GWC 04), Brno, Czeck Republic. find
17. A. Ratnaparkhi. 1996. A maximum entropy part-of-speech tagger. In Proceedings of the Empirical Methods in Natural Language Processing Conference. find
18. R. Rosenfeld. 1996. A maximum entropy approach to adaptive statistical language modelling. Computer Speech and Language, 10:187–228. find
19. A. Suarez. 2002. A maximum entropy-based word sense disambiguation system. In Proc. International Conference on Computational Linguistics. find
20. A. Ushioda. 1996. Hierarchical clustering of words. In In Proceedings of COLING 96, pages 1159–1162. find
21. D. Yarowsky. 1993. One sense per collocation. In In the Proceedings of ARPA Human Language Technology Workshop. find