Elamur Research⌕
☰
home / authors / Pedro Ortiz Suarez

Pedro Ortiz Suarez

Pedro Ortiz Suarez works in natural language processing. The papers here focus on building large text collections from the web and using them to pretrain models for languages other than English. A recurring question is whether noisy Common Crawl text can replace smaller, curated sources such as Wikipedia once it has been filtered and sorted by language. The work ties corpus construction, especially the OSCAR corpus, to tests on tasks such as part-of-speech tagging, dependency parsing and named entity recognition, within Language models & pretraining and Multilingual & cross-lingual NLP.

The most cited paper here is CamemBERT: a Tasty French Language Model (2019). It trained a monolingual French Transformer with the RoBERTa masked-language-modelling recipe on the French part of OSCAR. It also compared variants built from smaller or differently assembled corpora to see how corpus size and variety affect results. In A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020), five languages (Bulgarian, Catalan, Danish, Finnish and Indonesian) each received two sets of ELMo vectors. One set came from full Wikipedia dumps and the other from the corresponding slices of OSCAR. Each version was then plugged into UDPipe 2.0 for tagging and parsing.

The newest work here, Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022), turns back to the corpus itself. It revises the Ungoliant pipeline so that each whole document, rather than each line, receives a language label. It also strips boilerplate such as menus and footers, and it adds metadata annotations that users can apply for quality filtering.

Bylines place Ortiz Suarez at Sorbonne Université and the ALMANACH team in Paris from 2019 to 2022, alongside Inria. Later bylines list the University of Mannheim (2022–2023) and most recently the German Research Centre for Artificial Intelligence in Kaiserslautern (2023–2025). The most frequent co-authors on the papers here are Benoît Sagot and Laurent Romary. Other collaborators include Louis Martin, Djamé Seddah, Benjamin Muller and Éric Villemonte De La Clergerie.

Pedro Ortiz Suarez has 3 papers in our library, in 2019–2022, most of them on Language models & pretraining, Multilingual & cross-lingual NLP; often with Benoît Sagot, Laurent Romary, Benjamin Muller.

3 papers in our library · 2019–2022

OpenAlex · ORCID · h-index 11 · 52 works · 1 763 citations OpenAlex, 2026-10-14

3 works in our library · 1 274 citations in our library · 6 more works in our library · ⭐ Paper Star rank #1407

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: Google Scholar · Semantic Scholar · dblp · ACL Anthology

Affiliations

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 2 more institutions on single bylines are not listed.

Topics

Language models & pretraining1Multilingual & cross-lingual NLP1

Works by year

2020s

20221 paper20201 paper

2010s

20191 paper

Papers

The most cited first (OpenAlex and our library).

  1. CamemBERT: a Tasty French Language Model (2019 · 1 262 citations)
    Large pretrained Transformer models mostly exist for English or as multilingual models. Multilingual models are bulky and can underperform for well-resourced languages. This work asks whether a competitive monolingual model can be built for another language, using French as the test case. The authors pretrain…
  2. A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020 · 219 citations)
    Contextualized language models need large amounts of raw text. For many mid-resource languages, the usual source, Wikipedia, is small and covers only one encyclopedic genre. The authors ask whether a much larger but noisier web corpus can make up for its lower quality. They train ELMo embeddings for Bulgarian…
  3. Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022 · 154 citations)
    Earlier OSCAR releases (OSCAR 2019, OSCAR 21.09) built with the Ungoliant pipeline assigned a language to each line of Common Crawl text separately.

Also in our library

6 more works of theirs are in our library without a page here yet; they will appear as we publish them.

Co-authors here

7 co-authors with a page, the most papers together first.

Cited by

Papers of our library cite their works 1 274 times (of the 140 884 papers whose bibliographies we read); 126 citing papers with a page here →.

This page in Markdown → https://md.research.elamur.ai/free/en/authors/pedro-ortiz-suarez/