📰 BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models 2022 — accepted papers

BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models 2022 is the 2022 edition of the BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models, held in virtual+Dublin, May 2022 (publisher: Association for Computational Linguistics). The ACL Anthology lists 12 papers of it; our library holds 2 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. GPT-NeoX-20B: An Open-Source Autoregressive Language ModelSidney Black, Stella Biderman, Eric Hallahan et al.404 citations · our summary
2. You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual SettingsZeerak Talat, Aurélie Névéol, Stella Biderman et al.59 citations · our summary
3. Pipelines for Social Bias Testing of Large Language Models ↗Debora Nozza, Federico Bianchi, Dirk Hovy30 citations
4. Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ↗Xisen Jin, Dejiao Zhang, Henghui Zhu et al.26 citations
5. UNIREX: A Unified Learning Framework for Language Model Rationale Extraction ↗Aaron Chan, Maziar Sanjabi, Lambert Mathias et al.19 citations
6. A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model ↗Imad Lakim, Ebtesam Almazrouei, Ibrahim Abualhaol et al.18 citations
7. Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned ↗Sameera Horawalavithana, Ellyn Ayton, Shivam Sharma et al.18 citations
8. Emergent Structures and Training Dynamics in Large Language Models ↗Ryan Teehan, Miruna Clinciu, Oleg Serikov et al.7 citations
9. Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0 ↗Francesco De Toni, Christopher Akiki, Javier De La Rosa et al.7 citations
10. Dataset Debt in Biomedical Language Modeling ↗Jason Fries, Natasha Seelam, Gabriel Altay et al.5 citations

All accepted papers (12)

The most cited first; 2 with our summary link to their page here, the others to the original.

1. GPT-NeoX-20B: An Open-Source Autoregressive Language ModelSidney Black, Stella Biderman, Eric Hallahan et al.404 citations · our summary
2. You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual SettingsZeerak Talat, Aurélie Névéol, Stella Biderman et al.59 citations · our summary
3. Pipelines for Social Bias Testing of Large Language Models ↗Debora Nozza, Federico Bianchi, Dirk Hovy30 citations
4. Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ↗Xisen Jin, Dejiao Zhang, Henghui Zhu et al.26 citations
5. UNIREX: A Unified Learning Framework for Language Model Rationale Extraction ↗Aaron Chan, Maziar Sanjabi, Lambert Mathias et al.19 citations
6. A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model ↗Imad Lakim, Ebtesam Almazrouei, Ibrahim Abualhaol et al.18 citations
7. Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned ↗Sameera Horawalavithana, Ellyn Ayton, Shivam Sharma et al.18 citations
8. Emergent Structures and Training Dynamics in Large Language Models ↗Ryan Teehan, Miruna Clinciu, Oleg Serikov et al.7 citations
9. Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0 ↗Francesco De Toni, Christopher Akiki, Javier De La Rosa et al.7 citations
10. Dataset Debt in Biomedical Language Modeling ↗Jason Fries, Natasha Seelam, Gabriel Altay et al.5 citations
11. Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model ↗Sosuke Kobayashi, Shun Kiyono, Jun Suzuki et al.4 citations
12. Using ASR-Generated Text for Spoken Language Modeling ↗Nicolas Hervé, Valentin Pelloin, Benoit Favre et al.2 citations

📄 Papers with our summary in our library (2)

The most cited first (OpenAlex citations plus our library's).

1. GPT-NeoX-20B: An Open-Source Autoregressive Language ModelBigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language ModelsSidney Black, Stella Biderman, Eric Hallahan, et al.2022 · 821 citations
2. You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual SettingsBigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language ModelsZeerak Talat, Aurélie Névéol, Stella Biderman, et al.2022 · 130 citations

🧭 Top topics

Topics of its works in our library, with the number of works on each.

🧠Language models & pretraining (1)🌍Multilingual & cross-lingual NLP (1)

📚 Fields

🎭Arts and Humanities (2)🖥️Computer Science (2)🤝Social Sciences (1)

🧑‍🔬 Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

🏛️ Top institutions

Institutions on the bylines of its works here, by the number of works.

🏢GoogleMountain View · United States1 work here🔬Centre National de la Recherche ScientifiqueParis · France1 work here🏢AmazonSeattle · United States1 work here🎓University of California, Los AngelesLos Angeles · United States1 work here🎓National University of SingaporeSingapore · Singapore1 work here🎓Simon Fraser UniversityBurnaby · Canada1 work here🎓Trinity College DublinDublin · Ireland1 work here🎓Université Paris-SaclayGif-sur-Yvette · France1 work here🎓Heriot-Watt UniversityEdinburgh · United Kingdom1 work here🎓Yale UniversityNew Haven · United States1 work here🔬Beijing Academy of Artificial IntelligenceBeijing · China1 work here🏢NvidiaSanta Clara · United States1 work here

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/bigscience/2022/