Elamur Research⌕
☰
home / papers / 2016 / Improving event prediction by representing script participants

Improving event prediction by representing script participants

Summary

The paper addresses the automatic acquisition of script knowledge, i.e., knowledge about the typical course of events in everyday situations such as visiting a restaurant. Earlier unsupervised models were built on coreference chains and barely outperformed the simplest frequency baseline. Their weak point is entities that occur only once in a text: all of them fall into one shared category, so information about them is lost. The authors propose representing each event as a verb with labels for the scenario participants (participant-labeled events, PLE) in the subject, direct object and indirect object positions. Arguments are extracted with Stanford CoreNLP. Participant labels are assigned by the heuristic of Kampmann et al. (2015), which relies on WordNet hypernyms. The next event is predicted by a bigram model that counts event pairs with any number of intervening events (skip-all). The model was evaluated on the narrative cloze task on the Dinners from Hell and InScript corpora. It substantially outperforms the approaches of Pichotta & Mooney and Jans et al., as well as the frequency baseline, especially when the correct event has to be ranked near the top. Participant knowledge thus makes it possible to generalize from specific words to their roles in the scenario. The approach also combines crowdsourced script acquisition with unsupervised acquisition.

Key points

  • On Dinners from Hell, the model reaches Recall@10 = 0.41. For comparison, P&M scores 0.26, Jans 0.25 and MostFreq 0.27. Its coverage is lower (0.757 vs. 0.809–0.957) because it makes no prediction for an unseen context.
  • When predicting the full PLE, Recall@10 is 0.18 vs. 0.13. The authors present this result as a baseline for future work.
  • Data: Dinners from Hell contains 143 blog stories about restaurant visits, 10 of which form the dev set. InScript has 910 stories collected via Mechanical Turk across 10 scenarios, about 90 per scenario. Testing uses document-level leave-one-out.
  • Replacing automatic coreference chains with gold ones barely changes the result. Gold participant annotation, by contrast, gives a noticeable gain. The current automatic tagger reaches only 0.59 micro F.
  • Across all 10 InScript scenarios, the model consistently outperforms previous approaches on Recall@1 and Recall@10.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1067

@inproceedings{ahrendt2016improving,
  title = {{Improving event prediction by representing script participants}},
  author = {Ahrendt, Simon and Demberg, Vera},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1067},
  url = {https://doi.org/10.18653/v1/n16-1067}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chambers, N. and Jurafsky, D. (2008). Unsupervised learning of narrative event chains. In Proceedings of ACL-08: HLT, pages 789–797, Columbus, Ohio. Association for Computational Linguistics. find
  2. Chambers, N. and Jurafsky, D. (2009). Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 602– 610, Suntec, Singap… find
  3. Jans, B., Bethard, S., Vulić, I., and Moens, M.-F. (2012). Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 336–344, Avignon, France. Association for Computational L… find
  4. Kampmann, A., Thater, S., and Pinkal, M. (2015). A case-study of automatic participant labeling. In Proceedings of the International Conference of the German Society for Computational Linguistics and Language Technology, pages 97–105. find
  5. Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J., and McClosky, D. (2014). The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. find our card
  6. Modi, A., Anikina, T., Ostermann, S., and Pinkal, M. (2016). Inscript: Narrative texts annotated with script information. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference. find
  7. Pichotta, K. and Mooney, R. (2014). Statistical script learning with multi-argument events. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 220–229, Gothenburg, Sweden. Association for Computational Linguistics. find
  8. Rahman, A. and Ng, V. (2012). Resolving complex cases of definite pronouns: the winograd schema challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 777–789. Association for Computational Linguis… find our card
  9. Regneri, M., Koller, A., and Pinkal, M. (2010). Learning script knowledge with web experiments. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 979–988, Uppsala, Sweden. Association for Computational Linguistics. find
  10. Rudinger, R., Demberg, V., Modi, A., Van Durme, B., and Pinkal, M. (2015). Learning to predict script events from domain-specific text. Lexical and Computational Semantics (* SEM 2015), pages 205–210. find
  11. Schank, R. and Abelson, R. (1977). Scripts, plans, goals and understanding: An inquiry into human knowledge structures. Lawrence Erlbaum Associates, Hillsdale, NJ. find
  12. Schütz-Bosbach, S. and Prinz, W. (2007). Prospective coding in event representation. Cognitive processing, 8(2):93–102. find
  13. Zwaan, R. A., Langston, M. C., and Graesser, A. C. (1995). The construction of situation models in narrative comprehension: An event-indexing model. Psychological science, pages 292–297. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ahrendt-2016-improving-event-prediction-representing-script/