Konstantin Dobler
Konstantin Dobler has 2 papers in our library, in 2023–2024, most of them on Language models & pretraining, Multilingual & cross-lingual NLP, Semitic languages; often with Gerard de Melo.
OpenAlex · h-index 2 · 15 works · 16 citations OpenAlex, 2026-10-14
📄 2 works in our library · 🔗 Cited 58 times in our library · ⭐ Paper Stars ranking: #8728
Affiliations
Fields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Artificial Intelligence · 2 worksArts and Humanities · 2 worksComputer Science · 2 worksLanguage and Linguistics · 2 works
🧭 Topics 4Research subjects and languages the work is about.Language models & pretraining · 1 workMultilingual & cross-lingual NLP · 1 workSemitic languages · 1 workTokenization & segmentation · 1 work
Works by year
2020s
Papers
- FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models (2023 · 62 citations)
Adapting a multilingual model to one language often calls for a new tokenizer tailored to that language. The pretrained embedding matrix then no longer fits the new vocabulary. The authors propose FOCUS (Fast Overlapping Token Combinations Using Sparsemax) to build a sensible starting embedding matrix for the new… - Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough (2024)
The paper asks whether an academic lab with only a handful of server-grade GPUs and a few days of access can usefully adapt a 7B LLM to a new language through continued pretraining.
Co-authors here
Venues
- Annual Meeting of the Association for Computational Linguistics (ACL)
- Conference on Empirical Methods in Natural Language Processing (EMNLP): EMNLP 2023
Cited by
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/authors/konstantin-dobler/