🧑‍🔬 Marco Marelli

Marco Marelli works on how to evaluate compositional distributional semantic models. These are systems that build a vector for the meaning of a whole sentence by combining the vectors of its individual words. The papers here ask how such models can be tested so that the results reflect composition itself rather than unrelated abilities. The answer they give is purpose-built English sentence data, judged by crowdworkers for semantic relatedness and for entailment. The work therefore sits under Evaluation & benchmarks and Reasoning.

The most cited of these papers is A SICK cure for the evaluation of compositional distributional semantic models, which introduced the SICK benchmark. It responds to a gap in earlier test sets. Some were small and syntactically uniform, while others, such as STS and RTE, required handling named entities, dates and encyclopedic facts that have little to do with combining word meanings. SICK started from image and video captions, simplified them, and rewrote them into paraphrases, contradictions and scrambled variants. The companion paper, SemEval-2014 Task 1, used the same resource as the basis of a shared task. In it, participating systems were scored on relatedness and on entailment between full sentences.

Both papers here date from 2014, so the newest work on the site continues this benchmarking effort rather than marking a change of direction. Marelli is currently at the University of Milano-Bicocca. Earlier bylines include the University of Trento and Ghent University. Co-authors on this work include Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, Roberto Zamparelli and Stefano Menini.

Marco Marelli has 2 papers in our library, in 2014, most of them on Evaluation & benchmarks, Reasoning; often with Luisa Bentivogli, Marco Baroni, Raffaella Bernardi.