Elamur Research⌕
☰
home / papers / 2020 / WN-Salience: A Corpus of News Articles with Entity Salience Annotations

WN-Salience: A Corpus of News Articles with Entity Salience Annotations

Summary

Research on entity salience, which asks which of the many entities in a text are actually central to it, has been held back by a shortage of suitable data. Existing resources have several weaknesses. Some rely on automatic NLP pipelines for entity annotation, which introduces errors. Others derive salience from click logs or abstract-matching heuristics, which are noisy. The rest are crowdsourced, which makes them small and costly, or are not fully public. The authors address this with WN-Salience, built automatically from English Wikinews. In Wikinews articles, the writers themselves link entity mentions. When an entity is strongly tied to the story, they also assign a matching Wikinews category. The paper treats these category-linked entities as salient, so the labels reflect the author's judgement rather than readers' or heuristics'. The authors compare WN-Salience with earlier datasets and run baseline salience detection experiments. The results show the new corpus is challenging and differs in character from NYT-Salience. It offers a scalable, freely licensed benchmark for salience detection and salient entity linking.

Key points

  • Size: WN-Salience contains about 7,000 English news articles with roughly 90,000 human-made in-text entity annotations. It is released on GitHub under CC BY 2.5.
  • Label source: Entities linked to Wikinews categories count as salient. Entities linked only to Wikipedia pages count as non-salient.
  • Compared datasets:
    • MDA: about 50,000 items, labels from click-based soft labelling, proprietary.
    • NYT-Salience: 100,976 items, heuristic labels, text not distributed.
    • Reuters-128 Salience: 128 items, crowdsourced.
    • Trani et al.'s Wikinews set: 604 items, graded crowdsourced labels.
  • Experiments: The baseline follows Dunietz and Gillick (2014), using hand-crafted features with a Naive Bayes classifier.
  • Findings:
    • Positional features proved effective, consistent with earlier work.
    • Frequency-based features (head count, mentions) worked poorly on WN-Salience, likely because its articles are short.
    • The centrality feature lowered recall noticeably on WN-Salience.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{wu2020wn,
  title = {{WN-Salience: A Corpus of News Articles with Entity Salience Annotations}},
  author = {Wu, Chuan and Kanoulas, Evangelos and Rijke, Maarten de and Lu, Wei},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.257/}
}

References

15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. (2008). An experimental comparison of click position-bias models. In WSDM, pages 87–94. find
  2. ACM. Dalton, J., Dietz, L., and Allan, J. (2014). Entity query feature expansion using knowledge base links. In SIGIR, pages 365–374. find
  3. ACM. Dojchinovski, M., Reddy, D., Kliegr, T., Vitvar, T., and Sack, H. (2016). Crowdsourced corpus with entity salience annotations. In LREC, Paris, France. European Language Resources Association (ELRA). find our card
  4. Dunietz, J. and Gillick, D. (2014). A new entity salience task with millions of training examples. In EACL, volume 14, page 205. find our card
  5. Fetahu, B., Markert, K., and Anand, A. (2015). Automated news suggestions for populating Wikipedia entity pages. In CIKM, pages 323–332. find
  6. ACM. Gabrilovich, E., Ringgaard, M., and Subramanya, A. (2013). Facc1: Freebase annotation of clueweb corpora, version 1 (release date 2013-06-26, format version 1, correction level 0). http://lemurproject.org/clueweb09/FACC1/. ↗
  7. Gamon, M., Yano, T., Song, X., Apacible, J., and Pantel, P. (2013). Identifying salient entities in web pages. In CIKM, pages 2375–2380. find
  8. ACM. Ieong, S., Mishra, N., Sadikov, E., and Zhang, L. (2012). Domain bias in web search. In WSDM, pages 413–422. find
  9. ACM. Ponza, M., Ferragina, P., and Piccinno, F. (2019). Swat: A system for detecting salient Wikipedia entities in texts. Computational Intelligence, 35(4):858–890. find
  10. Raviv, H., Kurland, O., and Carmel, D. (2016). Document retrieval using entity-based language models. In SIGIR, pages 65–74. ACM. Röder, M., Usbeck, R., Hellmann, S., Gerber, D., and Both, A. (2014). N 3 -a collection of datasets for named entity recognition and disambiguation in the NLP interchang… find
  11. Sandhaus, E. (2008). The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6(12):e26752. find
  12. Shen, W., Wang, J., and Han, J. (2015). Entity linking with a knowledge base: Issues, techniques, and solutions. TKDE, 27(2):443–460. Tran, T. A., Niederée, C., Kanhabua, N., Gadiraju, U., and Anand, A. (2015). Balancing novelty and salience: Adaptive learning to rank entities for timeline summariz… find
  13. ACM. Trani, S., Lucchese, C., Perego, R., Losada, D. E., Cec-carelli, D., and Orlando, S. (2018). SEL: A unified algorithm for salient entity linking. find
  14. Computational Intelligence. Wu, C., Kanoulas, E., and de Rijke, M. (2019). It all starts with entities: A salient entity topic model. Natural Language Engineering, pages 1–19. find
  15. Xiong, C., Liu, Z., Callan, J., and Liu, T.-Y. (2018). Towards better text understanding and retrieval through kernel entity salience modeling. In SIGIR, pages 575–584. ACM. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/wu-2020-wn-salience-corpus-news-articles/