Elamur Research⌕
☰
home / papers / 2007 / Arabic Cross-Document Person Name Normalization

Arabic Cross-Document Person Name Normalization

Paper · 2007 · Walid Magdy, Kareem Darwish, Ossama Emam et al.

Summary

The paper addresses cross-document person-name normalization for Arabic news. The goal is to decide which person entities, each already resolved inside its own document, refer to the same individual across documents. The task is hard for three reasons: transliterated foreign names have many spellings, common Arabic first names are highly ambiguous, and one person may carry several names. The authors built a new dataset from Arabic news articles about Gaza and Lebanon in July 2006. To extract the entities, they relied on the NERT tool developed by Florian et al. (2004). Their method has two steps. First, heuristic rules clean errant entities left by NER and tracking errors. Second, an SVM decides for each pair of entities whether they should be merged, using lexical, orthographic, phonetic and morphological features. The resulting clusters reach a cluster F-measure of 0.93. Two baselines score much lower: no merging gives 0.62, and merging only exact name matches gives 0.74. The work shows that a small, mostly language-light feature set can resolve person identity across documents without dictionaries or WordNet-style resources, which Arabic largely lacks.

Key points

  • Data: 7,184 person entities from 2,931 Arabic news documents found via the Arabic news.google.com. The source NERT system scores F 0.77 on NER and 0.88 on intra-document tracking.
  • Cleaning rules:
    • A name mention is dropped if its corpus frequency is under 1/30 of the entity's most frequent mention.
    • Single-token names shorter than 3 characters are removed.
    • Entities with 10 or more distinct name mentions are discarded.
    • Nominals are stemmed with the Lee et al. (2003) stemmer.
  • Training pairs: 19,825 pairs were manually judged, with 18,503 positive and 1,322 negative. Pairs were picked by shared names, shared nominals, substring matches or small edit distance.
  • Test set: 470 entities grouped into 253 gold classes, giving 110,215 pairs. The classifier was SVMLight with a linear kernel.
  • Results: with all features, purity is 97.2% and CF is 93.1%. Removing edit distance drops CF to 85.5%, and removing the tokens-per-name feature drops it to 77.8%. Training on only 2k pairs still yields a CF of 90.3%.

Interesting ideas

  • I1. Before running the expensive pairwise classifier, cheap filters (shared mention, substring, small edit distance) pick candidate pairs, which also concentrates training data near the decision boundary.
  • I2. Upstream extraction noise is repaired by corpus-level statistics, for example by dropping a mention whose frequency is tiny compared with its entity's dominant name.
  • I3. An entity that collects too many different names is treated as over-merged, because real people are referred to by only a few names.
  • I4. Nominal descriptors such as "the president" are stemmed and then compared as features, so role words help tell apart people who share a name.
  • I5. Evaluation combines purity, entropy and cluster F with a tuned threshold, and compares against both a no-merge baseline and an exact-match baseline.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): candidate generation like this could choose which ambiguous cases to send to the LLM teacher tiers, rather than spending labels on easy ones.
  • I2 (world): when entities from many texts are merged, global mention frequencies could flag and detach mentions that a local coreference step attached by mistake.
  • I3 (frames): a cap on distinct name forms per coreference chain gives a simple, explainable sanity check that can trigger splitting or review.
  • I4 (world): lemmatized role nominals attached to an entity could act as attributes when deciding whether two entities in the world model are the same person.
  • I5 (learn): reporting both trivial baselines and a purity/recall trade-off curve would make entity-merging evaluations harder to game.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBM (Egypt)Giza · Egypt8 works · 301 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksDecision Sciences · 209 worksManagement Science and Operations Research · 108 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Walid Magdy, Kareem Darwish, Ossama Emam and Hany Hassan. 2007. Arabic Cross-Document Person Name Normalization. Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{magdy2007arabic,
  title = {{Arabic Cross-Document Person Name Normalization}},
  author = {Walid Magdy, Kareem Darwish, Ossama Emam and Hany Hassan},
  booktitle = {Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources},
  year = {2007},
  doi = {10.3115/1654576.1654582},
  url = {https://doi.org/10.3115/1654576.1654582}
}

References

19 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bhattacharya I. and Getoor L. “A Latent Dirichlet Allocation Model for Entity Resolution.” 6 th SIAM Conference on Data Mining (SDM), Bethesda, USA, April 2006. find
  2. Chinchor N., Brown E., Ferro L., and Robinson P. “Named Entity Recognition Task Definition.” MITRE, 1999. find
  3. Cohen W., Ravikumar P., and Fienberg S. E. “A Comparison of String Distance Metrics for Name-Matching Tasks.” In Proceedings of the International Joint Conference on Artificial Intelligence, 2003. find
  4. Dozier C. and Zielund T. “Cross-document Co-Reference Resolution Applications for People in the Legal Domain.” In 42nd Annual Meeting of the Association for Computational Linguistics, Reference Resolution Workshop, Barcelona, Spain. July 2004. find
  5. Fleischman M. B. and Hovy E. “Multi-Document Person Name Resolution.” In 42nd Annual Meeting of the Association for Computational Linguistics, Reference Resolution Workshop, Barcelona, Spain. July 2004. find
  6. Ji H. and Grishman R. “Applying Coreference to Improve Name Recognition”. In 42nd Annual Meeting of the Association for Computational Linguistics, Reference Resolution Barcelona, Spain. July (2004). find
  7. Workshop, Ji H. and Grishman R. "Improving Name Tagging by Reference Resolution and Relation Detection." ACL 2005 find
  8. Joachims T. “Learning to Classify Text Using Support Vector Machines.” Ph.D. Dissertation, Kluwer, (2002). ↗
  9. Joachims T. “Optimizing Search Engines Using Click-through Data.” Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD), (2002). find
  10. Lee Y. S., Papineni K., Roukos S., Emam O., Hassan H. “Language Model Based Arabic Word Segmentation.” In ACL 2003, pp. 399-406, (2003).
  11. Li H., Srihari R. K., Niu C., and Li W. “Location Normalization for Information Extraction.” Proceedings of the 19th international conference on Computational linguistics, pp. 1-7, 2002 find
  12. Li H., Srihari R. K., Niu C., and Li W. “Location Normalization for Information Extraction.” Proceedings of the sixth conference on applied natural language processing, 2000. pp. 247 – 254. find
  13. Mann G. S. and Yarowsky D. “Unsupervised Personal Name Disambiguation.” Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003. pp. 33-40. find
  14. Maynard D., Tablan V., Ursu C., Cunningham H., and Wilks Y. “Named Entity Recognition from Diverse Text Types.” Recent Advances in Natural Language Processing Conference, (2001). find
  15. Palmer D. D. and Day D. S. “A statistical Profile of the Named Entity Task”. Proceedings of the fifth conference on Applied natural language processing, pp. 190-193, (1997). find
  16. R. Florian R., Hassan H., Ittycheriah A., Jing H., Kambhatla N., Luo X., Nicolov N., and Roukos S. “A Statistical Model for Multilingual Entity Detection and Tracking.” In HLT-NAACL, 2004. ↗
  17. Rosell M., Kann V., and Litton J. E. “Comparing Comparisons: Document Clustering Evaluation Using Two Manual Classifications.” In ICON 2004 find
  18. Sekine S. “Named Entity: History and Future”. Project notes, New York University, (2004). find
  19. Solorio T. “Improvement of Named Entity Tagging by Machine Learning.” Ph.D. thesis, National Institute of Astrophysics, Optics and Electronics, Puebla, Mexico, September 2005. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/magdy-2007-arabic-cross-document-person-name/