Elamur Research⌕
☰
home / papers / 2026 / SmartMatch: Real-Time Semantic Retrieval for Translation Memory Systems

SmartMatch: Real-Time Semantic Retrieval for Translation Memory Systems

Paper · 2026 · Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Alicia Picazo-Izquierdo et al.

Summary

Translation memory (TM) tools in CAT software usually find matches with edit-distance fuzzy matching. This approach misses useful segments that mean the same thing but share few words. Modern LLM-era sentence encoders have rarely been tested for TM because of latency concerns. The authors present SmartMatch, an open-source toolkit and web demo. It puts three retrieval approaches side by side over the same TM data: Okapi/Pensieve fuzzy matching, BM25 in Weaviate, and dense retrieval with encoders over a Weaviate HNSW index. In the interface, users can type or pick a query, switch backend and model, set k, see matches with scores and per-query latency, and record which result they prefer. The authors benchmark ten encoders on DGT-TM using COMET, BERTScore, METEOR and chrF, plus coverage and latency. Dense retrieval returned candidates for every query and gave the best semantic scores, with LaBSE strongest overall, while running in milliseconds. The fuzzy baseline had poorer coverage and very slow worst-case responses. This shows that semantic TM retrieval is practical for real-time translation work.

Key points

  • Stack: Python 3.11+, Weaviate (HNSW ANN plus a BM25 inverted index over identical segments), LangChain and Typer, with a Java Okapi/Pensieve service as the legacy CAT baseline. Code is at github.com/EEstevanell/SmartMatch.
  • Encoders compared: BGE-M3, Multilingual E5, LaBSE, LASER, Llama-3.2-1B, M3E, MiniLM-L6-v2, MPNet-base-v2, DistilRoBERTa and T5 base.
  • Coverage: every encoder found a match for 100% of queries, compared with 85% for BM25 and 81% for Okapi Pensieve.
  • Quality: BERTScore reached up to 0.91 at k=10. LaBSE, LASER and T5 base beat Pensieve significantly with large effect size (|δ| ≥ 0.474) on COMET and BERTScore at k=5 and k=10. The lexical baselines stayed competitive on METEOR and chrF.
  • Latency (p50/p90): BM25 was 3/13 ms, MiniLM-L6-v2 6/8 ms and LASER 7/14 ms. Okapi Pensieve was 44/1116 ms. Stated limitations are that the system lacks CAT-integration features and does not adapt automatically from user feedback.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,105 worksInformation retrieval & search · 552 works

Cite

Cite as Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Alicia Picazo-Izquierdo, Tharindu Ranasinghe, Ruslan Mitkov and Rafael Muñoz. 2026. SmartMatch: Real-Time Semantic Retrieval for Translation Memory Systems. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-demo.36

@inproceedings{estevanellvalladares2026smartmatch,
  title = {{SmartMatch: Real-Time Semantic Retrieval for Translation Memory Systems}},
  author = {Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Alicia Picazo-Izquierdo, Tharindu Ranasinghe, Ruslan Mitkov and Rafael Muñoz},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-demo.36},
  url = {https://doi.org/10.18653/v1/2026.eacl-demo.36}
}

References

39 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
  2. Andrea Silvestre Baquero and Ruslan Mitkov. 2017. Translation memory systems have a long way to go. In Proceedings of the Workshop Human-Informed Translation and Interpreting Technology, pages 44– 51. find
  3. Eduard Barbu, Carla Parra Escartín, Luisa Bentivogli, Matteo Negri, Marco Turchi, Constantin Orasan, and Marcello Federico. 2016. The first automatic translation memory cleaning shared task. Machine Translation, 30(3):145–166. find
  4. Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1):289–300. find
  5. Qian Cao and Deyi Xiong. 2018. Encoding gated translation memory into neural machine translation. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 3042– 3047. find
  6. Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, and Rui Yan. 2022. Neural machine translation with contrastive translation memories. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3591–3601, Abu Dhabi, United Arab Emirates. Association for Computationa… find
  7. Norman Cliff. 1993. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological bulletin, 114(3):494. find
  8. Spence Green, Jason Chuang, Jeffrey Heer, and Christopher D Manning. 2014. Predictive translation memory: A mixed-initiative system for human language translation. In Proceedings of the 27th annual ACM symposium on User interface software and technology, pages 177–187. find
  9. Rohit Gupta and Constantin Orasan. 2014. Incorporating paraphrasing in translation memory matching and retrieval. In Proceedings of the 17th Annual Conference of the European Association for Machine Translation, pages 3–10. find
  10. Rohit Gupta, Constantin Orăsan, Marcos Zampieri, Mihaela Vela, Josef van Genabith, and Ruslan Mitkov. 2016. Improving translation memory matching and retrieval using paraphrases. Machine Translation, 30(1):19–40. find
  11. Qiuxiang He, Guoping Huang, Lemao Liu, and Li Li. 2019. Word position aware translation memory for neural machine translation. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 367–379. Springer. find
  12. Miguel A Jiménez-Crespo. 2010. The effect of translation memory tools in translated web texts: Evidence from a comparative product-based study. Linguistica antverpiensia, 8:213–232. find
  13. Liangyou Li, Andy Way, and Qun Liu. 2014. A discriminative framework of integrating translation memory features into smt. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: MT Researchers Track, pages 249– 260. find
  14. Ruslan Mitkov. 2022. Translation memory systems. In The Routledge Handbook of Translation and Memory, pages 364–380. Routledge. find
  15. Yongyu Mu, Abudurexiti Reheman, Zhiquan Cao, Yuchun Fan, Bei Li, Yinqiao Li, Tong Xiao, Chun-liang Zhang, and Jingbo Zhu. 2023. Augmenting large language model translators via translation memories. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10287–10299, Toronto, Ca… find
  16. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
  17. Bernard Rosner, Robert J Glynn, and Mei-Ling T Lee. 2006. The wilcoxon signed rank test for paired comparisons of clustered data. Biometrics, 62(1):185– 192. find
  18. Matteo Negri, Duygu Ataman, Masoud Jalili Sabet, Marco Turchi, and Marcello Federico. 2017. Automatic translation memory cleaning. Machine Translation, 31(3):93–115. find
  19. Masoud Jalili Sabet, Matteo Negri, Marco Turchi, and Eduard Barbu. 2016. An unsupervised method for automatic translation memory cleaning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 287–292. find
  20. Jiyoung Park. 2024. Ethical approach to translation memory reuse: discussions from copyright and business ethics perspectives. Translation Studies, 17(1):37–52. find
  21. Pilar Sánchez-Gijón, Joss Moorkens, and Andy Way. 2019. Post-editing neural machine translation versus translation memory segments. Machine Translation, 33(1):31–59. find
  22. Raul Pas , calau, Laura-Rebeca Stiegelbauer, and Dumi-tra Mădălina Pantea. 2025. The importance of translation workflow in global business. Studii de S , tiint , ă s , i Cultură, 21(2). find
  23. Satoshi Sato and Makoto Nagao. 1990. Toward memory-based translation. In COLING 1990 Volume 3: Papers Presented to the 13th International Conference on Computational Linguistics. find
  24. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  25. Anne Gram Schjoldager and Tina Paulsen Christensen. 2010. Translation-memory (tm) research: what do we know and how do we know it? Hermes, 44:89– 101. find
  26. Ming Qian and Chuiqing Kong. 2024. Enabling human-centered machine translation using concept-based large language model prompting and translation memory. In International Conference on Human-Computer Interaction, pages 118–134. Springer. find
  27. Tharindu Ranasinghe, Hansi Hettiarachchi, Constantin Orasan, and Ruslan Mitkov. 2025. MUSTS: MUltilingual semantic textual similarity benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 331–353, Vienna, Austria. Associ… find
  28. Ralf Steinberger, Andreas Eisele, Szymon Klocek, Spyridon Pilos, and Patrick Schlüter. 2012. DGT-TM: A freely available translation memory in 22 languages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 454–459, Istanbul, Turkey. European…
  29. Takuya Tamura, Xiaotian Wang, Takehito Utsuro, and Masaaki Nagata. 2023. Target language monolingual translation memory based nmt by cross-lingual retrieval of similar translations and reranking. In Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 313–323. find
  30. Tharindu Ranasinghe, Ruslan Mitkov, Constantin Orăsan, and Rocío Caro Quintana. 2021. Semantic textual similarity based on deep learning. Corpora in Translation and Contrastive Research in the Digital Age: Recent advances and explorations, 158:101. find
  31. Katerina Raisa Timonera and Ruslan Mitkov. 2015. Improving translation memory matching through clause splitting. In Proceedings of the Workshop Natural Language Processing for Translation Memories, pages 17–23. find
  32. Tharindu Ranasinghe, Constantin Orasan, and Ruslan Mitkov. 2020. Intelligent translation memory matching and retrieval with sentence encoders. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 175–184, Lisboa, Portugal. European Association for M… find
  33. Kun Wang, Chengqing Zong, and Keh-Yih Su. 2013. Integrating translation memory into phrase-based machine translation during decoding. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11–21, Sofia, Bulgaria. Association for Comp… find
  34. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  35. Friedel Wolff. 2016. Combining off-the-shelf components to clean a translation memory. Machine Translation, 30(3):167–181. find
  36. Uwe Reinke. 2018. State of the art in translation memory technology. Language Technologies for a Multilingual Europe; Rehm, G., Stein, D., Sasaki, F., Witt, A., Eds, pages 55–84. find
  37. Jitao Xu, Josep Crego, and François Yvon. 2023. Integrating translation memories into non-autoregressive machine translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1326–1338, Dubrovnik, Croatia. Association for Computati… find
  38. Tianfu Zhang, Heyan Huang, Chong Feng, and Xiaochi Wei. 2020. Similarity-aware neural machine translation: reducing human translator efforts by leveraging high-potential sentences with translation memory. Neural Computing and Applications, 32(23):17623– 17635. find
  39. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/valladares-2026-smartmatch-real-time-semantic-retrieval/