Elamur Research
☰
home / papers / 2019 / FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP

FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-4010.pdf

Summary

Modern word representations, such as character-level features and contextualized language-model embeddings, usually cannot be plugged into a model the way classic vectors can. Each type tends to require its own changes to the network architecture. FLAIR, a PyTorch-based Python library from Zalando Research, addresses this by putting every embedding type behind one shared interface. A single .embed() call attaches vectors to Sentence/Token objects, and any combination of embeddings can be merged through StackedEmbeddings. As a result, one model architecture can work with GloVe, ELMo, BERT, Flair embeddings and others without extra engineering. The library also provides:

  • a downloader that fetches public benchmark corpora and converts them into ready-made data splits,
  • a generic ModelTrainer with Hyperopt-based hyperparameter search,
  • a model zoo of pre-trained taggers and classifiers.

This is a demonstration paper, so its contribution is the software itself rather than new benchmark scores. Its value lies in letting researchers compare and combine embeddings, and apply strong sequence labeling or text classification models, with only a few lines of code.

Key points

  • Supported embeddings: WordEmbeddings, CharacterEmbeddings, BytePairEmbeddings, FlairEmbeddings, PooledFlairEmbeddings, ELMoEmbeddings, ELMoTransformerEmbeddings and BertEmbeddings, plus DocumentPoolEmbeddings and DocumentLSTMEmbeddings for whole documents. The recommended default stacks WordEmbeddings with FlairEmbeddings.
  • Dataset downloader: covers CoNLL 2000, CoNLL 2003 (Dutch/Spanish), EIEC (Basque), IMDB, TREC-6, TREC-50, Universal Dependencies (30 languages), WikiNER (9 languages) and WNUT-17. Licensed data such as the English and German CoNLL-03 sets cannot be fetched automatically.
  • Training tools: ModelTrainer handles mini-batching, checkpointing, learning-rate annealing, evaluation and logging. Hyperparameter tuning uses Hyperopt (TPE) against the dev split.
  • Model zoo:
    • 4-class NER for en/de/nl/es;
    • 12-class Ontonotes NER;
    • GermEval 2018 offensive-language detection;
    • IMDB sentiment analysis;
    • PropBank semantic frame detection;
    • a multilingual universal PoS tagger covering 12 languages.
  • Model variants: "default" models are GPU-oriented and use LMs with 2048 hidden states. "Fast" models are CPU-oriented and use LMs with about 1024 hidden states. Installation requires Python ≥3.6 and is done with pip install flair.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

301 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 292 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-4010

@inproceedings{akbik2019flair,
  title = {{FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP}},
  author = {Akbik, Alan and Bergmann, Tanja and Blythe, Duncan and Rasul, Kashif and Schweter, Stefan and Vollgraf, Roland},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)},
  year = {2019},
  doi = {10.18653/v1/n19-4010},
  url = {https://doi.org/10.18653/v1/n19-4010}
}

References

28 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alan Akbik, Tanja Bergmann, and Roland Vollgraf. 2019a. Multilingual sequence labeling with one model. In NLDL 2019, Northern Lights Deep Learning Workshop. find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  3. Alan Akbik, Tanja Bergmann, and Roland Vollgraf. 2019b. Pooled contextualized embeddings for named entity recognition. In NAACL, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics, page to appear. find
  4. Benjamin Heinzerling and Michael Strube. 2018. BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  5. Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In COLING 2018, 27th International Conference on Computational Linguistics, pages 1638– 1649. find our card
  6. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% solution. In Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers, pages 57–60. Association for Computational Linguistics. find
  7. Inaki Alegria, Olatz Arregi, Irene Balza, Nerea Ezeiza, Izaskun Fernandez, and Ruben Urizar. Design and development of a named entity recognizer for an agglutinative language. find
  8. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 427–431. find
  9. Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464. find our card
  10. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. find
  11. James Bergstra, Daniel Yamins, and David Daniel Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. find
  12. Xin Li and Dan Roth. 2002. Learning question classifiers. In Proceedings of the 19th international conference on Computational linguistics-Volume 1, pages 1–7. Association for Computational Linguistics. find
  13. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. pages 142–147. Association for Computational Linguistics. find
  14. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional LSTMs-CNNs-CRF. arXiv preprint arXiv:1603.01354. find our card
  15. Ellen M Voorhees and Donna Harman. 2000. Overview of the sixth text retrieval conference (trec-6). Information Processing & Management, 36(1):3–35. find
  16. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon… find our card
  17. Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the germeval 2018 shared task on the identification of offensive language. Austrian Academy of Sciences, Vienna September 21, 2018. find
  18. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  19. Daniel Zeman, Jan Haji, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. find
  20. Conll 2018 shared task: Multilingual parsing from raw text to universal dependencies. Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21. find our card
  21. Joel Nothman, Nicky Ringland, Will Radford, Tara Murphy, and James R. Curran. 2012. Learning multilingual named entity recognition from Wikipedia. Artificial Intelligence, 194:151–175. find
  22. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. find
  23. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  24. Matthew Peters, Mark Neumann, and Christopher Clark Kenton Lee Luke Zettlemoyer Mohit Iyyer, Matt Gardner. 2018a. Deep contextualized word representations. 6th International Conference on Learning Representations. find
  25. Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018b. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509. find our card
  26. Alec Radford, Karthik Narasimhan, Time Salimans, and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, Technical report, OpenAI. find
  27. Erik F Tjong Kim Sang and Sabine Buchholz. 2000. Introduction to the CoNLL-2000 shared task: Chunking. In Proceedings of the 2nd workshop on Learning language in logic and the 4th conference on Computational natural language learning-Volume 7, pages 127–132. Association for Computational Linguistics… find
  28. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, find