Elamur Research⌕
☰
home / papers / 2017 / SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation

SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation

Summary

The paper summarizes the SemEval-2017 shared task on semantic textual similarity (STS). Systems must determine how close in meaning two sentences are on a scale from 0 to 5. Performance is measured by the Pearson correlation between system scores and human judgments. Previously the task focused mainly on English, but this year the main focus was on multilingual and cross-lingual pairs involving Arabic, Spanish and Turkish. Most of the evaluation data was drawn from the SNLI corpus. The authors constructed the sentence pairs themselves, selecting them by the similarity of GloVe vectors, obtained annotations via Amazon Mechanical Turk, and then had the sentences translated into other languages by humans. An additional subtask examines the relationship between STS and machine translation quality estimation using WMT data. 31 teams participated, submitting 84 runs. For English and Spanish, the best systems agree well with human judgments. For Arabic, Arabic-English and Turkish-English pairs, results are noticeably weaker, so models that transfer more easily to new languages with little training data are needed. In addition, the authors introduced the STS Benchmark, a curated set of English data for comparing methods.

Key points

  • Seven tracks of 250 pairs each, 1750 pairs in total: ar-ar, ar-en, es-es, es-en (4a based on SNLI, 4b based on WMT QE), en-en and tr-en. The Turkish-English track was a "surprise language": no training data was provided for it, and the pair itself was announced only with the release of the test set.
  • Annotation: each pair was rated by 5 MTurk workers with master qualification, and the gold score is the mean of these five. Pairs in track 4b were annotated by a single expert, a native Spanish speaker. Annotators were paid $1 per batch of 20 pairs.
  • The SNLI pairs were constructed anew rather than taken as is, so that the existing entailment labels would not hint at the STS score. Selection used the cosine similarity of sums of 50-dimensional GloVe vectors (Gigaword 5 + Wikipedia).
  • Participants: 31 teams, 84 runs. 17 teams submitted a total of 44 runs covering all languages. For English, the best systems typically reach 70–80% correlation with human judgments.
  • The STS Benchmark is a publicly available selection of English STS data from 2012–2017. It is accompanied by baseline results (averaged word embeddings, SIF, InferSent, Paragram, FastText, etc.) and a leaderboard on the STS wiki.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Multilingual & cross-lingual NLP

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s17-2001 · arXiv 1708.00055

@inproceedings{cer2017semeval,
  title = {{SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation}},
  author = {Cer, Daniel and Diab, Mona and Agirre, Eneko and Lopez-Gazpio, Iñigo and Specia, Lucia},
  booktitle = {Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)},
  year = {2017},
  doi = {10.18653/v1/s17-2001},
  eprint = {1708.00055},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/s17-2001}
}

References

80 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretab… find our card
  2. Alexandre Berard, Christophe Servan, Olivier Pietquin, and Laurent Besacier. 2016. MultiVec: a multilingual and multilevel representation learning toolkit for NLP. In Proceedings of LREC 2016. http://www.lrec-conf.org/proceedings/lrec2016/pdf/666 Paper.pdf. ↗
  3. Ergun Biçici. 2017. RTM at SemEval-2017 Task 1: Referential translation machines for predicting semantic similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2030. ↗
  4. Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. SemEval-2014 Task 10: Multilingual semantic textual similarity. In Proceedings of SemEval 2014. http://www.aclweb.org/anthology/S14-2010. ↗ our card
  5. Johannes Bjerva and Robert Östling. 2017. ResSim at SemEval-2017 Task 1: Multilingual word representations for semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2021. ↗
  6. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proceedings of the SemEval-2016. http://www.aclweb.org/anthology/S16-1081. ↗ our card
  7. Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of WMT… find our card
  8. Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. SemEval-2012 Task 6: A pilot on semantic textual similarity. In Proceedings of *SEM 2012/SemEval 2012. http://www.aclweb.org/anthology/S12-1051. ↗ our card
  9. Ondřej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of WMT 2013. http://www.aclweb.org/anthology/W13-220… ↗ our card
  10. Jérémy Ferrero, Laurent Besacier, Didier Schwab, and Frédéric Agnès. 2017. CompiLIG at SemEval-2017 Task 1: Cross-language plagiarism detection methods for semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2012. ↗
  11. Pedro Fialho, Hugo Patinho Rodrigues, Luı́sa Coheur, and Paulo Quaresma. 2017. L2f/inesc-id at semeval-2017 tasks 1 and 2: Lexical and semantic features in word and textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2032. ↗
  12. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of EMNLP 2015. http://aclweb.org/anthology/D/D15/D15-1075.pdf. ↗
  13. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of NAACL/HLT 2013. http://cs.jhu.edu/ ccb/publications/ppdb.pdf. ↗ our card
  14. Tomas Brychcin and Lukas Svoboda. 2016. UWB at SemEval-2016 Task 1: Semantic textual similarity using lexical, syntactic, and semantic information. In Proceedings of SemEval 2016. https://www.aclweb.org/anthology/S/S16/S16-1089.pdf. ↗
  15. Basma Hassan, Samir AbdelRahman, Reem Bahgat, and Ibrahim Farag. 2017. FCICU at SemEval-2017 Task 1: Sense-based language independent semantic textual similarity approach. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2015. ↗
  16. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, and Hui Jiang. 2016. Enhancing and combining sequential and tree LSTM for natural language inference. CoRR abs/1609.06038. http://arxiv.org/abs/1609.06038. ↗
  17. Hua He, Kevin Gimpel, and Jimmy Lin. 2015. Multi-perspective sentence similarity modeling with convolutional neural networks. In Proceedings of EMNLP. pages 1576–1586. http://aclweb.org/anthology/D15-1181. ↗ our card
  18. Yun-Nung Chen, Dilek Hakkani-Tür, and Xiaodong He. 2015. Learning bidirectional intent embeddings by convolutional deep structured semantic models for spoken language understanding. In Proceedings of NIPS-SLU, 2015. https://www.microsoft.com/en-us/research/publication/learning-bidirectional-intent-… ↗
  19. Hua He and Jimmy Lin. 2016. Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In Proceedings of NAACL/HLT. http://www.aclweb.org/anthology/N16-1108. ↗ our card
  20. Hua He, John Wieting, Kevin Gimpel, Jinfeng Rao, and Jimmy Lin. 2016. UMD-TTIC-UW at SemEval-2016 Task 1: Attention-based multi-perspective convolutional neural networks for textual similarity measurement. In Proceedings of SemEval 2016. http://www.anthology.aclweb.org/S/S16/S16-1170.pdf. ↗
  21. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. CoRR abs/1705.02364. http://arxiv.org/abs/1705.02364. ↗ our card
  22. John Henderson, Elizabeth Merkhofer, Laura Strickhart, and Guido Zarrella. 2017. MITRE at SemEval-2017 Task 1: Simple semantic similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2027. ↗
  23. Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2010. Recognizing textual entailment: Rational, evaluation and approaches. J. Nat. Language Eng. 16:105–105. https://doi.org/10.1017/S1351324909990234. — [doi:10.1017/s1351324909990234](https://doi.org/10.1017/s1351324909990234) ↗
  24. Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In Proceedings of NAACL/HLT. http://www.aclweb.org/anthology/N16-1162. ↗ our card
  25. Birk Diedenhofen and Jochen Musch. 2015. cocor: A comprehensive solution for the statistical comparison of correlations. PLoS ONE 10(4). http://dx.doi.org/10.1371/journal.pone.0121945. — [doi:10.1371/journal.pone.0121945](https://doi.org/10.1371/journal.pone.0121945) ↗
  26. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9(8):1735–1780. http://dx.doi.org/10.1162/neco.1997.9.8.1735. ↗
  27. Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In Proceedings of COLING 04. http://aclweb.org/anthology/C/C04/C04-1051.pdf. ↗ our card
  28. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of NAACL/HLT 2006. http://aclweb.org/anthology/N/N06/N06-2015.pdf. ↗
  29. Mirela-Stefania Duma and Wolfgang Menzel. 2017. SEF@UHH at SemEval-2017 Task 1: Unsupervised knowledge-free semantic textual similarity via paragraph vector. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2024. ↗
  30. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of CIKM. https://www.microsoft.com/en-us/research/publication/learning-deep-structured-semantic-models-for-web-search-us… ↗
  31. Cristina España Bonet and Alberto Barrón-Cedeño. 2017. Lump at SemEval-2017 Task 1: Towards an interlingua semantic similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2019. ↗
  32. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of ACL/IJCNLP. http://www.aclweb.org/anthology/P15-1162. ↗ our card
  33. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. https://books.google.com/books?id=Rehu8OOzMIMC. ↗
  34. Sergio Jimenez, Claudia Becerra, and Alexander Gelbukh. 2012a. Soft cardinality: A parameterized similarity function for text comparison. In Proceedings of *SEM 2012/SemEval 2012. http://www.aclweb.org/anthology/S12-1061. ↗
  35. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of LREC 14. http://www.lrec-conf.org/proceedings/lrec2014/pdf/363 Paper.pdf. ↗ our card
  36. Sergio Jimenez, Claudia Becerra, and Alexander Gelbukh. 2012b. Soft Cardinality: A parameterized similarity function for text comparison. In Proceedings of *SEM 2012/SemEval 2012. http://aclweb.org/anthology/S/S12/S12-1061.pdf. ↗
  37. Yuan Cao Gaurav Kumar Matt Post. 2015. Joshua 6: A phrase-based and hierarchical statistical machine translation. The Prague Bulletin of Mathematical Linguistics 104:516. https://ufal.mff.cuni.cz/pbml/104/art-post-cao-kumar.pdf. ↗
  38. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. CoRR abs/1607.01759. http://arxiv.org/abs/1607.01759. ↗
  39. Fanqing Meng, Wenpeng Lu, Yuteng Zhang, Jinyong Cheng, Yuehan Du, and Shuwang Han. 2017. QLUT at SemEval-2017 Task 1: Semantic textual similarity based on word embeddings. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2020. ↗
  40. Tom Kenter, Alexey Borisov, and Maarten de Rijke. 2016. Siamese cbow: Optimizing word embeddings for sentence representations. In Proceedings of ACL. http://www.aclweb.org/anthology/P16-1089. ↗
  41. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. CoRR abs/1301.3781. http://arxiv.org/abs/1301.3781. ↗ our card
  42. Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2015. Skip-thought vectors. CoRR abs/1506.06726. http://arxiv.org/abs/1506.06726. — [arXiv:1506.06726](https://arxiv.org/abs/1506.06726) ↗
  43. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS 2013. http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf. ↗ our card
  44. Sarah Kohail, Amr Rekaby Salama, and Chris Biemann. 2017. STS-UHH at SemEval-2017 Task 1: Scoring semantic textual similarity using supervised and unsupervised ensemble. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2025. ↗
  45. George A. Miller. 1995. WordNet: A lexical database for english. Commun. ACM 38(11):39–41. https://doi.org/10.1145/219717.219748. ↗ our card
  46. Jey Han Lau and Timothy Baldwin. 2016. An empirical evaluation of doc2vec with practical insights into document embedding generation. In Proceedings of ACL Workshop on Representation Learning for NLP. http://www.aclweb.org/anthology/W/W16/W16-1609.pdf. ↗
  47. Alessandro Moschitti. 2006. Efficient convolution kernels for dependency and constituent syntactic trees. In Proceedings of ECML’06. http://dx.doi.org/10.1007/11871842 32. — [doi:10.1007/11871842](https://doi.org/10.1007/11871842) ↗
  48. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. CoRR abs/1405.4053. http://arxiv.org/abs/1405.4053. ↗ our card
  49. Jiaqi Mu, Suma Bhat, and Pramod Viswanath. 2017. Representing sentences as low-rank subspaces. CoRR abs/1704.05358. http://arxiv.org/abs/1704.05358. ↗
  50. I-Ta Lee, Mahak Goindani, Chang Li, Di Jin, Kristen Marie Johnson, Xiao Zhang, Maria Leonor Pacheco, and Dan Goldwasser. 2017. PurdueNLP at SemEval-2017 Task 1: Predicting semantic textual similarity with paraphrase and event embeddings. In Proceedings of SemEval-2017. http://www.aclweb.org/antholog… ↗
  51. El Moatez Billah Nagoudi, Jérémy Ferrero, and Didier Schwab. 2017. LIM-LIG at SemEval-2017 Task1: Enhancing the semantic similarity for arabic sentences with vectors weighting. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2017. ↗
  52. Roberto Navigli and Simone Paolo Ponzetto. 2010. BabelNet: Building a very large multilingual semantic network. In Proceedings of ACL 2010. http://aclweb.org/anthology/P/P10/P10-1023.pdf. ↗ our card
  53. Wenjie Liu, Chengjie Sun, Lei Lin, and Bingquan Liu. 2017. ITNLP-AiKF at SemEval-2017 Task 1: Rich features based svr for semantic textual similarity computing. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2022. ↗
  54. Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. 2017. Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features. arXiv https://arxiv.org/pdf/1703.02507.pdf. ↗ our card
  55. Nabin Maharjan, Rajendra Banjade, Dipesh Gautam, Lasang J. Tamang, and Vasile Rus. 2017. Dt team at semeval-2017 task 1: Semantic similarity using alignments, sentence-level embeddings and gaussian mixture model output. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2014. ↗
  56. Robert Parker, David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2011. Gigaword Fifth Edition LDC2011T07. Linguistic Data Consortium. https://catalog.ldc.upenn.edu/ldc2011t07. ↗
  57. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of ACL 2014 Demonstrations. http://www.aclweb.org/anthology/P/P14/P14-5010. ↗ our card
  58. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of EMNLP 2014. http://www.aclweb.org/anthology/D14-1162. ↗ our card
  59. Nghia The Pham, Germán Kruszewski, Angeliki Lazaridou, and Marco Baroni. 2015. Jointly optimizing word representations for lexical and sentential tasks with the c-phrase model. In Proceedings of ACL/IJCNLP. http://www.aclweb.org/anthology/P15-1094. ↗
  60. Frane Šarić, Goran Glavaš, Mladen Karan, Jan Šnajder, and Bojana Dalbelo Bašić. 2012a. Takelab: Systems for measuring semantic text similarity. In Proceedings of *SEM 2012/SemEval 2012. http://www.aclweb.org/anthology/S12-1060. ↗
  61. Nils Reimers, Philip Beyer, and Iryna Gurevych. 2016. Task-oriented intrinsic evaluation of semantic textual similarity. In Proceedings of COLING 2016. http://aclweb.org/anthology/C16-1009. ↗
  62. Frane Šarić, Goran Glavaš, Mladen Karan, Jan Šnajder, and Bojana Dalbelo Bašić. 2012b. TakeLab: Systems for measuring semantic text similarity. In Proceedings of SemEval 2012. http://www.aclweb.org/anthology/S12-1060. ↗
  63. Barbara Rychalska, Katarzyna Pakulska, Krystyna Chodor-owska, Wojciech Walczak, and Piotr Andruszkiewicz. 2016. Samsung Poland NLP Team at SemEval-2016 Task 1: Necessity for diversity; combining recursive autoencoders, wordnet and ensemble methods to measure semantic similarity. In Proceedings of Se… find
  64. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. From paraphrase database to compositional paraphrase model and back. Transactions of the ACL (TACL) 3:345–358. http://aclweb.org/anthology/Q/Q15/Q15-1025.pdf. ↗ our card
  65. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2016a. Charagram: Embedding words and sentences via character n-grams. In Proceedings of EMNLP. https://aclweb.org/anthology/D16-1157. ↗
  66. Alexandre Salle, Marco Idiart, and Aline Villavicencio. 2016a. Enhancing the lexvec distributed word representation model using positional contexts and external memory. CoRR abs/1606.01283. http://arxiv.org/abs/1606.01283. — [arXiv:1606.01283](https://arxiv.org/abs/1606.01283) ↗
  67. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2016b. Towards universal paraphrastic sentence embeddings. In Proceedings of ICLR 2016. http://arxiv.org/abs/1511.08198. ↗ our card
  68. Alexandre Salle, Marco Idiart, and Aline Villavicencio. 2016b. Matrix factorization using window sampling and negative sampling for improved word representations. In Proceedings of ACL. http://aclweb.org/anthology/P16-2068. ↗
  69. John Wieting and Kevin Gimpel. 2017. Revisiting recurrent networks for paraphrastic sentence embeddings. CoRR abs/1705.00364. http://arxiv.org/abs/1705.00364. ↗
  70. Gilles Serasset. 2015. DBnary: Wiktionary as a lemon-based multilingual lexical resource in RDF. Semantic Web Journal (special issue on Multilingual Linked Open Data) 6:355–361. https://doi.org/10.3233/SW-140147. ↗
  71. Hao Wu, Heyan Huang, Ping Jian, Yuhang Guo, and Chao Su. 2017. BIT at SemEval-2017 Task 1: Using semantic information space to evaluate semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2007. ↗
  72. Yang Shao. 2017. HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2016. ↗
  73. Wei Xu, Chris Callison-Burch, and William B. Dolan. 2015. SemEval-2015 Task 1: Paraphrase and semantic similarity in Twitter (PIT). In Proceedings of SemEval 2015. http://www.aclweb.org/anthology/S15-2001. ↗ our card
  74. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of CIKM ’14. https://www.microsoft.com/en-us/research/publication/a-latent-semantic-model-with-convolutional-pooling-stru… ↗
  75. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL 2:67–78. http://aclweb.org/anthology/Q14-1006. ↗
  76. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of AMTA 2006. http://mt-archive.info/AMTA-2006-Snover.pdf. ↗ our card
  77. WenLi Zhuang and Ernie Chang. 2017. Neobility at SemEval-2017 Task 1: An attention-based sentence similarity model. In Proceedings SemEval-2017. http://www.aclweb.org/anthology/S17-2023. ↗
  78. Martyna Śpiewak, Piotr Sobecki, and Daniel Karaś. 2017. OPI-JSA at SemEval-2017 Task 1: Application of ensemble learning for computing semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/anthology/S17-2018. ↗
  79. Md Arafat Sultan, Steven Bethard, and Tamara Sumner. 2015. DLS@CU: Sentence similarity from word alignment and semantic vector composition. In Proceedings of SemEval 2015. http://aclweb.org/anthology/S/S15/S15-2027.pdf. ↗
  80. Junfeng Tian, Zhiheng Zhou, Man Lan, and Yuanbin Wu. 2017. ECNU at SemEval-2017 Task 1: Leverage kernel-based traditional nlp features and neural networks to build a universal model for multilingual and cross-lingual semantic textual similarity. In Proceedings of SemEval-2017. http://www.aclweb.org/… ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cer-2017-semeval-2017-task-1-semantic/