Elamur Research⌕
☰
home / papers / 2024 / The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations

The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations

Paper · 2024 · Aina Garí Soler, Matthieu Labeau and Chloé Clavel

Summary

Most pre-trained language models tokenize input into subwords. Rare or unknown words are therefore broken into several pieces and have no dedicated embedding. This paper asks two questions. First, how should those pieces be pooled into a single contextualized word vector? Second, do the resulting vectors capture meaning as well as those of in-vocabulary words? The authors evaluate English models (BERT, CharacterBERT, ELECTRA, XLNet, and the FLOTA segmentation method) on intrinsic similarity tasks in two settings:

  • Inter-word: a newly built WordNet-based dataset, SPLIT-SIM, whose pairs contain zero, one or two split words.
  • Within-word: instance pairs taken from WiC.

Several pooling strategies are compared, including averaging subword vectors (AVG) and using only the longest subword (LNG). Averaging is the most reliable choice overall. Split words usually receive weaker similarity estimates than full words, but polysemous nouns are a notable exception. Segmentations that respect morphology yield better estimates. Similarities between two split words tend to be inflated, so raw cosine values should not be compared across split-types. These findings matter for anyone applying contextual embeddings to rare vocabulary.

Key points

  • SPLIT-SIM (released at github.com/ainagari/splitsim) has four subsets: monosemous nouns (M-N), monosemous verbs (M-V), polysemous nouns (P-N) and polysemous verbs (P-V). Each subset is balanced across 0-SPLIT, 1-SPLIT and 2-SPLIT pairs.
  • Existing benchmarks contain few split words under BERT's default tokenizer: 0-SPLIT pairs make up 97% of CoSimLex, 85% of SCWS, 96% of Simlex-999 and 95% of WS353.
  • 1-SPLIT pairs perform worst in ordinary conditions, mainly because the two words differ widely in frequency. The split-word disadvantage persists when the analysis controls for frequency.
  • In the within-word WiC setting, BERT with AVG correctly labeled only 18 of 97 "different-meaning" 2-SPLIT pairs. CharacterBERT (CBERT) performed equally well across split-types and was the best option for 2-SPLIT pairs.
  • Using fewer subwords does not guarantee better estimates. For BERT, the first subword (often the longest) is generally the least informative piece.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Institut national de recherche en sciences et technologies du numériqueLe Chesnay-Rocquencourt · France84 works · 25,414 citations🎓Institut Polytechnique de ParisPalaiseau · France15 works · 1,612 citations🔬Laboratoire Traitement et Communication de l’InformationPalaiseau · France8 works · 378 citations🎓Télécom ParisPalaiseau · France11 works · 826 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Aina Garí Soler, Matthieu Labeau and Chloé Clavel. 2024. The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00647

@article{soler2024impact,
  title = {{The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations}},
  author = {Aina Garí Soler, Matthieu Labeau and Chloé Clavel},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00647},
  url = {https://doi.org/10.1162/tacl_a_00647}
}

References

75 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Paşca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and WordNet-based approaches. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the A…
  2. Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. FLAIR: An easy-to-use framework for state-of-the-art NLP. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages…
  3. Carlos S. Armendariz, Matthew Purver, Matej Ulčar, Senja Pollak, Nikola Ljubešić, and Mark Granroth-Wilding. 2020. CoSimLex: A resource for evaluating graded word similarity in context. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 5878–5886, Marseille, France. Eu… find
  4. Alexandra Benamar, Cyril Grouin, Meryl Bothua, and Anne Vilnat. 2022. Evaluating tokenizers impact on OOVs representation with transformers models. In Proceedings of the Language Resources and Evaluation Conference, pages 4193–4204, Marseille, France. European Language Resources Association. \#\# Ackn… find
  5. We thank the anonymous reviewers and the TACL action editor for their thorough reviews and useful remarks, which helped improve this paper. This research has been supported by the Télécom Paris research chair on Data Science and Artificial Intelligence for Digitalized Industry and Services (DSAIDI… find
  6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. https://doi.org/10.1162/tacl_a_00051 ↗
  7. Rishi Bommasani, Kelly Davis, and Claire Cardie. 2020. Interpreting pretrained contextualized representations via reductions to static embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4758–4781, Online. Association for Computational Lingui…
  8. Kaj Bostrom and Greg Durrett. 2020. Byte pair encoding is suboptimal for language model pretraining. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4617–4624, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.findings-emnlp.414 ↗
  9. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chr… — Language Models are Few-Shot Learners
  10. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. https://doi.org/10 .18653/v1/N19-1423
  11. Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Yonatan Belinkov, and Preslav Nakov. 2019. One size does not fit all: Comparing NMT representations of different granularities. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu… find
  12. Hicham El Boukkouri, Olivier Ferret, Thomas Lavergne, Hiroshi Noji, Pierre Zweigenbaum, and Jun’ichi Tsujii. 2020. CharacterBERT: Reconciling ELMo and BERT for word-level open-vocabulary representations from characters. In Proceedings of the 28th International Conference on Computational Linguistics…
  13. Katrin Erk, Diana McCarthy, and Nicholas Gaylord. 2009. Investigations on word senses and word usages. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 10–18, Suntec, Singapore.… find
  14. Kenneth Ward Church. 2020. Emerging trends: Subwords, seriously? Natural Language Engineering, 26(3):375–382. https://doi.org/10.1017/S1351324920000145 ↗
  15. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR. find
  16. Katrin Erk, Diana McCarthy, and Nicholas Gaylord. 2013. Measuring word meaning in context. Computational Linguistics, 39(3):511–554. https://doi.org/10.1162/COLI a 00142 ↗
  17. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. find
  18. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Language, Speech, and Communication. MIT Press, Cambridge, MA. https://doi.org/10.7551/mitpress/7287.001.0001 — [doi:10.7551/mitpress](https://doi.org/10.7551/mitpress) ↗
  19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019
  20. Philip Gage. 1994. A new algorithm for data compression. C Users Journal, 12(2):23–38. find
  21. Aaron Gokaslan and Vanya Cohen. 2019. OpenWebText corpus. find
  22. Christina L. Gagné, Thomas L. Spalding, and Daniel Schmidtke. 2019. LADEC: The large database of English compounds. Behavior Research Methods, 51(5):2152–2179. https://doi.org/10.3758/s13428-019-01282-6, PubMed: 31347038 — [doi:10.3758/s13428-019-01282-6](https://doi.org/10.3758/s13428-019-01282-6) ↗
  23. Aurélie Herbelot and Marco Baroni. 2017. High-risk learning: Acquiring new word vectors from tiny data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 304–309, Copenhagen, Denmark. Association for Computational Linguistics. https://doi.org /10.18653… ↗
  24. Matthias Gallé. 2019. Investigating the effectiveness of BPE: The power of shorter sequences. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1375–1381, Hong Kon… find
  25. Felix Hill, Roi Reichart, and Anna Korhonen. 2015. SimLex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 41(4):665–695. https://doi.org/10.1162/COLI a 00237 ↗
  26. Valentin Hofmann, Janet Pierrehumbert, and Hinrich Schütze. 2021. Superbizarre is not superb: Derivational morphology improves BERT’s interpretation of complex words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen…
  27. Aina Garı́ Soler and Marianna Apidianaki. 2021. Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. Transactions of the Association for Computational Linguistics, 9:825–844. https://doi.org/10.1162/tacl_a_00400 ↗
  28. Aina Garı́ Soler, Marianna Apidianaki, and Alexandre Allauzen. 2019. Word usage similarity estimation with sentence representations and automatic substitutes. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 9–21, Minneapolis, Minnesota. Associa… find
  29. Valentin Hofmann, Hinrich Schütze, and Janet Pierrehumbert. 2022. An embarrassingly simple method to mitigate undesirable properties of pretrained language model tokenizers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 38… find
  30. Aina Garı́ Soler, Matthieu Labeau, and Chloé Clavel. 2022. One word, two sides: Traces of stance in contextualized word representations. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3950–3959, Gyeongju, Republic of Korea. International Committee on Computa… find
  31. Mario Giulianelli, Marco Del Tredici, and Raquel Fernández. 2020. Analysing lexical semantic change with contextualised word representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3960–3973, Online. Association for Computational Linguistic…
  32. Jimin Hong, TaeHee Kim, Hyesu Lim, and Jaegul Choo. 2021. AVocaDo: Strategy for adapting vocabulary to downstream domain. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4692–4700, Online and Punta Cana, Dominican Republic. Association for Computation… find
  33. Eric Huang, Richard Socher, Christopher Manning, and Andrew Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 873–882, Jeju Island, Korea.…
  34. Claudia Leacock, Martin Chodorow, and George A. Miller. 1998. Using corpus statistics and WordNet relations for sense identification. Computational Linguistics, 24(1):147–165.
  35. Matthias Huck, Simon Riess, and Alexander Fraser. 2017. Target-side word segmentation strategies for neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 56–67, Copenhagen, Denmark. Association for Computational Linguistics. https://doi.org/10.18653/v1 /W… find
  36. Jiahuan Li, Yutong Shen, Shujian Huang, Xinyu Dai, and Jiajun Chen. 2021a. When is char better than subword: A systematic study of segmentation algorithms for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internati… find
  37. Omri Keren, Tal Avinari, Reut Tsarfaty, and Omer Levy. 2022. Breaking character: Are subwords good enough for MRLs after all? ArXiv, abs/2204.04748v1. find
  38. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational…
  39. Xiaotao Li, Shujuan You, Yawen Niu, and Wai Chen. 2021b. Learning embeddings for rare words leveraging Internet search engine and spatial location relationships. In Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pages 278–287, Online. Association for Com… find
  40. Taku Kudo and John Richardson. 2018. Sentence-Piece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Assoc…
  41. Qianchu Liu, Diana McCarthy, and Anna Korhonen. 2020. Towards better context-aware lexical semantics: Adjusting contextualized representations through static anchors. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4066–4075, Online. Associati… find
  42. Severin Laicher, Sinan Kurtyigit, Dominik Schlechtweg, Jonas Kuhn, and Sabine Schulte im Walde. 2021. Explaining and improving BERT performance on lexical semantic change detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Studen… find
  43. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  44. Thang Luong, Richard Socher, and Christopher Manning. 2013. Better word representations with recursive neural networks for morphology. In Proceedings of the Seventeenth Conference on Computational Natural Language Thomas K. Landauer and Susan T. Dumais. 1997. A solution to Plato’s problem: The laten…
  45. Manuel Mager, Arturo Oncevay, Elisabeth Mager, Katharina Kann, and Thang Vu. 2022. BPE vs. morphological segmentation: A case study on machine translation of four polysynthetic languages. In Findings of the Association for Computational Linguistics: ACL 2022, pages 961–971, Dublin, Ireland. Associat… find
  46. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. WiC: The word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum…
  47. Syrielle Montariol and Alexandre Allauzen. 2021. Measure and evaluation of semantic divergence across two languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long P… find
  48. Mohammad Taher Pilehvar, Dimitri Kartsaklis, Victor Prokhorov, and Nigel Collier. 2018. Card-660: Cambridge rare word dataset - A reliable benchmark for infrequent word representation models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1391–1401,… find
  49. Stephen Mutuvi, Emanuela Boros, Antoine Doucet, Adam Jatowt, Gaël Lejeune, and Moses Odeo. 2022. Fine-tuning de modèles de langues pour la veille épidémiologique multilingue avec peu de ressources (Fine-tuning Language Models for Low-resource Multilingual Epidemic Surveillance). In Actes de la 2… find
  50. Tiago Pimentel, Ryan Cotterell, and Brian Roark. 2021. Disambiguatory signals are stronger in word-initial positions. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 31–41, Online. Association for Computational Lingui… find
  51. Victor Prokhorov, Mohammad Taher Pilehvar, Dimitri Kartsaklis, Pietro Lio, and Nigel Collier. 2019. Unseen word representation by aligning heterogeneous lexical semantic spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6900–6907. https://doi.org/10.1609/aaai.v33i01 .3… — [doi:10.1609/aaai.v33i01](https://doi.org/10.1609/aaai.v33i01) ↗
  52. Anmol Nayak, Hariprasad Timmapathini, Karthikeyan Ponnalagu, and Vijendran Gopalan Venkoparao. 2020. Domain adaptation challenges of BERT in tokenization and sub-word representations of out-of-vocabulary words. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pages 1–5,… find
  53. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. arXiv preprint arXiv:2003.07082. https://arxiv.org/abs/2003.07082 ↗
  54. Dat Quoc Nguyen, Dai Quoc Nguyen, Dang Duc Pham, and Son Bao Pham. 2014. RDRPOSTagger: A ripple down rules-based part-of-speech tagger. In Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 17–20, Gothenburg, Swede… find
  55. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  56. Mike Schuster and Kaisuke Nakajima. 2012. Japanese and Korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5149–5152. IEEE. https://doi.org/10.1109/ICASSP.2012.6289079 — [doi:10.1109/icassp.2012.6289079](https://doi.org/10.1109/icassp.2012.6289079) ↗
  57. Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, and Iryna Gurevych. 2021. How good is your tokenizer? On the monolingual performance of multilingual language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Join…
  58. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin…
  59. David J. Sheskin. 2003. Handbook of parametric and nonparametric statistical procedures. Chapman and Hall/CRC. https://doi.org/10.1201/9781420036268 — [doi:10.1201/9781420036268](https://doi.org/10.1201/9781420036268) ↗
  60. Claudia H. Sánchez-Gutiérrez, Hugo Mailhot, S. Hélène Deacon, and Maximiliano A. Wilson. 2018. MorphoLex: A derivational morphological database for 70,000 English words. Behavior Research Methods, 50:1568–1580. https://doi.org/10.3758/s13428-017 -0981-8 PubMed: 29124719 ↗
  61. Robyn Speer. 2022. rspeer/wordfreq: v3.0 (v3.0.2). Zenodo. https://doi.org/10 .5281/zenodo.7199437 find
  62. Timo Schick and Hinrich Schütze. 2020a. BERTRAM: Improved word embeddings have big impact on contextualized model performance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3996–4007, Online. Association for Computational Linguistics. https://doi… find
  63. Timo Schick and Hinrich Schütze. 2020b. Rare words: A major problem for contextualized embeddings and how to fix it by attentive mimicking. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8766–8774. https://doi.org/10.1609/aaai.v34i05 .6403 — [doi:10.1609/aaai.v34i05](https://doi.org/10.1609/aaai.v34i05) ↗
  64. Dominik Schlechtweg, Nina Tahmasebi, Simon Hengchen, Haim Dubossarsky, and Barbara McGillivray. 2021. DWUG: A large resource of diachronic word usage graphs in four languages. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7079–7091, Online and Punta… find
  65. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7222–7240, Online. Association for Computation…
  66. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,…
  67. Gregor Wiedemann, Steffen Remus, Avi Chawla, and Chris Biemann. 2019. Does BERT make any sense? Interpretable word sense disambiguation with contextualized embeddings. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019): Long Papers, pages 161–170, Erlangen, Germany.… find
  68. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — Transformers: State-of-the-Art Natural Language Processing
  69. Processing Systems, volume 32. Curran Associates, Inc. find
  70. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERT-Score: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. find
  71. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV…
  72. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… find
  73. A Results with FastText Zhibiao Wu and Martha Palmer. 1994. Verb semantics and lexical selection. In 32nd Annual Meeting of the Association for Computational Linguistics, pages 133–138, Las Cruces, New Mexico, USA. Association for Computational Linguistics. https://doi.org/10.3115/981732.981751 ↗
  74. Chaoqun Yang, Yuanyuan Zhu, Ming Zhong, and Rongrong Li. 2019a. Semantic similarity computation in knowledge graphs: Comparisons and improvements. In 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW), pages 249–252. https://doi.org/10.1109/ICDEW.2019 .000-5 — [doi:10.1109/icdew.2019](https://doi.org/10.1109/icdew.2019) ↗
  75. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019b. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/soler-2024-impact-word-splitting-semantic-content/