Elamur Research⌕
☰
home / papers / 2020 / Bifixer and Bicleaner: two open-source tools to clean your parallel data

Bifixer and Bicleaner: two open-source tools to clean your parallel data

Summary

Neural MT is more sensitive to noisy training data than older statistical systems. Yet widely used public parallel corpora are often fed to training without any cleaning. The authors test two open-source tools from the ParaCrawl project on four English–Portuguese corpora: Europarl, OpenSubtitles, JW300 and WikiMatrix. The tools were originally built for raw web-crawled text.

  • Bifixer repairs data before filtering. It fixes encoding and Mojibake, HTML entities and common typos, re-splits glued sentences, and marks exact and near duplicates.
  • Bicleaner then filters the data. It applies 37 hand-written rules and scores each remaining sentence pair with a random-forest classifier. To keep the evaluation fair, the authors trained new classifiers that excluded the tested corpora.

The authors trained Transformer systems in both translation directions, before and after cleaning, using two classifier thresholds (0.5 and 0.7). Cleaning shrank the noisier corpora sharply and shortened training time. BLEU was equal or slightly higher in every setting, and the strictest filtering gave the best scores. This shows that cleaning even well-known corpora is a cheap step that can save disk space and compute without hurting translation quality.

Key points

  • Corpus sizes (EN–PT): Europarl v7 has 2.2M sentence pairs, OpenSubtitles 2018 has over 33M, JW300 has 2.1M and WikiMatrix has 4.4M.
  • Bifixer results (averages): re-splitting recovers about 1.1% new sentences, about 4.6% of sentences get fixed, and about 9.8% are removed as duplicates or near duplicates. OpenSubtitles alone loses 8.1M pairs (24.5%) at this stage.
  • Manual check of 100 sample pairs per corpus: problems appeared in 30 pairs from WikiMatrix, 11 from OpenSubtitles and 2 from Europarl.
  • MT setup: 24 Transformer-base systems built with Marian and SentencePiece (32k vocabulary). They were evaluated on TED Talks dev/test sets with sacreBLEU.
  • Main result: the largest gain, +1 to +2.2 BLEU, came from OpenSubtitles after about 22M pairs (two-thirds of the corpus) were removed. WikiMatrix shrank by about half. Europarl EN→PT stayed unchanged, and the authors suggest some rules may be too strict for already clean data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cited by

54 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 50 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ramirezsanchez2020bifixer,
  title = {{Bifixer and Bicleaner: two open-source tools to clean your parallel data}},
  author = {Ramírez-Sánchez, Gema and Zaragoza-Bernabeu, Jaume and Bañón, Marta and Rojas, Sergio Ortiz},
  booktitle = {Proceedings of the 22nd Annual Conference of the European Association for Machine Translation},
  year = {2020},
  url = {https://aclanthology.org/2020.eamt-1.31/}
}

References

15 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agic, Zeljko and Ivan Vulic. 2019. Jw300: A wide-coverage parallel corpus for low-resource languages. In Korhonen, Anna, David R. Traum, and Lluı́s Màrquez, editors, ACL (1), pages 3204–3210. Association for Computational Linguistics. find our card
  2. Chaudhary, Vishrav, Yuqing Tang, Francisco Guzmán, Holger Schwenk, and Philipp Koehn. 2019. Low-resource corpus filtering using multilingual sentence embeddings. In Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2), pages 261–266, Florence, Italy, Aug… find our card
  3. Defauw, Arne, Tom Vanallemeersch, Sara Szoc, Frederic Everaert, Koen Van Winckel, Kim Scholte, Joris Brabers, and Joachim Van den Bogaert. 2019. Collecting domain specific data for mt: an evaluation of the paracrawl pipeline. In Proceedings of Machine Translation Summit XVII Volume 2: Translator, Pr… find
  4. Junczys-Dowmunt, Marcin, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, Syst… find our card
  5. Khayrallah, Huda and Philipp Koehn. 2018. On the impact of various types of noise on neural machine translation. In Proceedings of the Second Workshop on Neural Machine Translation and Generation, Melbourne. Association for Computational Linguistics. find our card
  6. Koehn, Philipp, Huda Khayrallah, Kenneth Heafield, and Mikel L. Forcada. 2018. Findings of the WMT 2018 shared task on parallel corpus filtering. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, Belgium, Brussels, October. Association for Computational Linguistics. find our card
  7. Koehn, Philipp, Francisco Guzmán, Vishrav Chaudhary, and Juan Pino. 2019. Findings of the WMT 2019 shared task on parallel corpus filtering for low-resource conditions. In Proceedings of the Fourth Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 3: Shared Tas… find our card
  8. Koehn, Philipp. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Conference Proceedings: the tenth Machine Translation Summit, pages 79–86, Phuket, Thailand. AAMT, AAMT. find
  9. Lison, Pierre and Jörg Tiedemann. 2016. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles. In Calzolari, Nicoletta, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios P… find our card
  10. Morishita, Makoto, Jun Suzuki, and Masaaki Nagata. 2019. Jparacrawl: A large scale web-based english-japanese parallel corpus. arXiv preprint arXiv:1911.10668. find
  11. Rikters, Matı̄ss. 2018. Impact of Corpora Quality on Neural Machine Translation. In In Proceedings of the 8th Conference Human Language Technologies - The Baltic Perspective (Baltic HLT 2018), Tartu, Estonia. find
  12. Sánchez-Cartagena, Vı́ctor M., Marta Bañón, Sergio Ortiz-Rojas, and Gema Ramı́rez-Sánchez. 2018. Prompsit’s submission to WMT 2018 Parallel Corpus Filtering shared task. In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, Brussels, Belgium, October. Assoc… find our card
  13. Schwenk, Holger, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2019. Wikimatrix: Mining 135m parallel sentences in 1620 language pairs from wikipedia. CoRR, abs/1907.05791. find our card
  14. Ye, Qi, Sachan Devendra, Felix Matthieu, Padmanabhan Sarguna, and Neubig Graham. 2018. When and why are pre-trained word embeddings useful for neural machine translation. In HLT-NAACL. find our card
  15. Bicleaner Corpora DGT EuBookshop JRC Capes Tilde SCielo NewsCommentary Global Voices Europarl JW300 WikiMatrix OpenSubtitles Training: probabilistic dictionaries Training: classifier model Cleaning: pre-filtering and classifying Subset (\#sentences) 3.5M 3.5M 1M 10M 1M 1M 50k 25k 100k 25k 2.2M 2.1M 4… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/sanchez-2020-bifixer-bicleaner-two-open-source/