Elamur Research⌕
☰
home / papers / 2004 / A Statistical Model for Multilingual Entity Detection and Tracking

A Statistical Model for Multilingual Entity Detection and Tracking

Paper · 2004 · R. Florian, H. Hassan, A. Ittycheriah et al.

Summary

The paper addresses Entity Detection and Tracking (EDT): finding every named, nominal and pronominal reference to an entity in free text and grouping references to the same real-world object into clusters. The authors, from IBM T.J. Watson, split EDT into two stages. In mention detection, each token gets a label that marks the start, inside or outside of a typed mention, using Maximum Entropy (MaxEnt) and Robust Risk Minimization (RRM) classifiers with Viterbi-style decoding. In entity tracking, a new MaxEnt model decides whether a mention should join an existing entity. Both models accept many lexical, syntactic and semantic features. Mention detection also draws on outputs of pre-existing named entity taggers that were trained on other corpora with different label schemes. The same framework was applied to Arabic, Chinese and English on the ACE September 2003 data, and the resulting system reached top-tier ACE results in all three languages. This shows that one statistical architecture can carry over to very different languages with only small, language-specific feature changes.

Key points

  • Evaluated on Arabic, Chinese and English within the Automatic Content Extraction (ACE) program, September 2003 evaluation. The system ranked among the best submissions for every language.
  • End-to-end ACE value on ACE'03 evaluation data: Arabic 54.5, Chinese 58.8, and English 69.7 (RRM) / 73.4 (MaxEnt).
  • Entity tracking on gold ("true") mentions gave ACE values of roughly 83.2 (Arabic) and 89.4 (Chinese), and 90.9 / 88.0 on two English test sets (Feb02, Sept02). The authors describe this as close to the inter-annotator agreement reported by LDC.
  • Training data was very unbalanced: English had 340k words, while Arabic had only around 60k words and Chinese around 90k characters. The authors note this limits feature-rich models for the newer languages.
  • In the feature ablations, dictionary/gazetteer features gave the largest single gain in ACE value. RRM used gazetteers better than MaxEnt and closed its initial gap to MaxEnt.
  • Stated limitations:
    • Genericity of mentions is not modelled; the feature window is only 5 tokens.
    • Pronoun coreference is relatively weak.

Interesting ideas

  • I1. Combination hypothesis. Outputs of existing taggers become extra feature streams, even when their categories and annotation guidelines differ from the target scheme. The learner then works out how those foreign labels map to the desired types, which lets mismatched legacy NER tools be reused for a new tagset or language.
  • I2. Rule-based systems as feature sources. Instead of choosing between hand-written rules and statistical learning, the authors suggest treating a rule system's output as one more input feature. This keeps expert knowledge without having to design feature interactions by hand.
  • I3. One architecture, swappable features. Keeping the classifier identical across languages and changing only the feature streams (e.g., Arabic segmentation handling) makes porting to a low-resource language mostly a matter of feature engineering, not redesign.
  • I4. Coreference as a link-or-not decision. Framing entity tracking as a series of binary decisions about attaching each mention to an existing entity allows arbitrary features in a simple MaxEnt classifier. It is an easy baseline to rebuild for other clustering-style tasks.
  • I5. Watch for metric mismatch. The paper shows that token-level F-measure and the task-level ACE value can move quite differently, because entity types are weighted differently. When a project's final score is a weighted, task-specific metric, it is worth reporting both and not assuming that optimizing one optimizes the other.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬IBM Research - Thomas J. Watson Research CenterYorktown Heights · United States41 works · 46,197 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 467 works

Cite

Cite as R. Florian, H. Hassan, A. Ittycheriah, H. Jing, N. Kambhatla, X. Luo, N. Nicolov and S. Roukos. 2004. A Statistical Model for Multilingual Entity Detection and Tracking. Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{florian2004statistical,
  title = {{A Statistical Model for Multilingual Entity Detection and Tracking}},
  author = {R. Florian, H. Hassan, A. Ittycheriah, H. Jing, N. Kambhatla, X. Luo, N. Nicolov and S. Roukos},
  booktitle = {Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004},
  year = {2004},
  doi = {10.21236/ada460245},
  url = {https://doi.org/10.21236/ada460245}
}

References

21 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. Aberdeen, D. Day, L. Hirschman, P. Robinson, and M. Vilain. 1995. Mitre: Description of the Alembic system used for MUC-6. In Proceedings of MUC-6, pages 141–155. find
  2. E. T. Bell. 1934. Exponential numbers. American Math. Monthly, 41:411–419. find
  3. A. Berger, S. Della Pietra, and V. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71.
  4. D. M. Bikel, S. Miller, R. Schwartz, and R. Weischedel. 1997. Nymble: a high-performance learning name-finder. In Proceedings of ANLP-97, pages 194–201.
  5. A. Borthwick, J. Sterling, E. Agichtein, and R. Grishman. 1998. Exploiting diverse knowledge sources via maximum entropy in named entity recognition. find
  6. A. Ittycheriah, L. Lita, N. Kambhatla, N. Nicolov, S. Roukos, and M. Stys. 2003. Identifying and tracking entity mentions in a maximum entropy framework. In HLT-NAACL 2003: Short Papers, May 27 - June 1. find
  7. H. Jing, R. Florian, X. Luo, T. Zhang, and A. Ittycheriah. 2003. HowtogetaChineseName(Entity): Segmentation and combination issues. In Proceedings of EMNLP’03, pages 200–207. find
  8. Y.-S. Lee, K. Papineni, S. Roukos, O. Emam, and H. Hassan. 2003. Language model based Arabic word segmentation. In Proceedings of the ACL’03, pages 399–406. find
  9. A. Mikheev, M. Moens, and C. Grover. 1999. Named entity recognition without gazetteers. In Proceedings of EACL’99.
  10. S. Miller, M. Crystal, H. Fox, L. Ramshaw, R. Schwarz, R. Stone, and R. Weischedel. 1998. Bbn: Description of the SIFT system as used for MUC-7. In MUC-7. find
  11. G. A. Miller. 1995. WordNet: A lexical database. Communications of the ACM, 38(11). find
  12. V. Ng and C. Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the ACL’02, pages 104–111.
  13. NIST. 2003a. The ACE evaluation plan. www.nist.gov/speech/tests/ace/index.htm. find
  14. NIST. 2003b. Proceedings of ACE’03. Booklet, Alexandria, VA, September. find
  15. L. Ramshaw and M. Marcus. 1994. Exploring the statistical derivation of transformational rule sequences for part-of-speech tagging. In Proceedings of the ACL Workshop on Combining Symbolic and Statistical Approaches to Language, pages 128–135. find
  16. L. Ramshaw and M. Marcus. 1995. Text chunking using transformation-based learning. In Proceedings of WVLC’95, pages 82–94.
  17. W. M. Soon, H. T. Ng, and C. Y. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544.
  18. E. F. Tjong Kim Sang and J. Veenstra. 1999. Representing text chunks. In Proceedings of EACL’99.
  19. E. F. Tjong Kim Sang. 2002. Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition. In Proceedings of CoNLL-2002, pages 155–158.
  20. R. J. Williams and J. Peng. 1990. An efficient gradient–based algorithm for on–line training of recurrent neural networks trajectories. Neural Computation, 2(4):490–501. find
  21. T. Zhang, F. Damerau, and D. E. Johnson. 2002. Text chunking based on a generalization of Winnow. Journal of Machine Learning Research, 2:615–637. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/florian-2004-statistical-model-multilingual-entity-detection/