Elamur Research⌕
☰
home / papers / 2023 / BioASQ-QA: A manually curated corpus for Biomedical Question Answering

BioASQ-QA: A manually curated corpus for Biomedical Question Answering

Paper · 2023 · Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis et al.

Summary

Biomedical literature grows faster than experts can follow, with PubMed now holding tens of millions of records. Systems that answer precise research questions are therefore needed, but most QA benchmarks are less realistic and provide only short answers. This data descriptor presents BioASQ-QA, the corpus built for Task B of the BioASQ challenge, which has run every year since 2012. European biomedical specialists write English questions based on needs from their own work. They use a dedicated annotation tool to find supporting MEDLINE articles and snippets and to link concepts from resources such as MeSH, Gene Ontology, Disease Ontology, Jochem and UniProt. Each question receives a paragraph-length "ideal" answer that works as a summary and, where the type allows, an exact answer as well. After each challenge round, the experts grade system outputs and use them to correct and expand the gold data. The resource now contains 4721 questions and continues to grow. It supports work on retrieval, QA, multi-document summarization and text generation. It has also served as a common benchmark showing steady progress, with top systems' ideal answers scored above 4 out of 5.

Key points

  • Size and growth: 4721 expert-written questions so far, with about 500 added each year. In total, 21 experts contributed, and 7 of them were the most active.
  • Question types: factoid, yes/no, list and summary. Each question comes with relevant documents, snippets, ontology concepts, an ideal answer and, where applicable, an exact answer. RDF triples from Linked Life Data were used early on but later dropped.
  • Ideal answer grading: experts score ideal answers on four criteria (including precision, repetition and readability) using a 1–5 scale. Averages for the best systems exceed 80%.
  • Inter-annotator agreement: experts chose quite different documents and snippets (mean F1 below 50%). Their ideal answers nevertheless agreed closely, meaning they reached the same answers through different evidence.
  • Availability: the data is released as JSON on Zenodo, and the BioASQ software is open source on GitHub. Around 100 universities and companies have taken part in the challenge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬National Centre of Scientific Research "Demokritos"Athens · Greece4 works · 1,826 citations🔬Institute of Informatics & TelecommunicationsAgia Paraskevi · Greece2 works · 1,638 citations🎓Aristotle University of ThessalonikiThessaloniki · Greece2 works · 193 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 487 works

Cite

Cite as Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras. 2023. BioASQ-QA: A manually curated corpus for Biomedical Question Answering. Scientific Data.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1038/s41597-023-02068-4

@inproceedings{krithara2023bioasq,
  title = {{BioASQ-QA: A manually curated corpus for Biomedical Question Answering}},
  author = {Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis and Georgios Paliouras},
  booktitle = {Scientific Data},
  year = {2023},
  doi = {10.1038/s41597-023-02068-4},
  url = {https://doi.org/10.1038/s41597-023-02068-4}
}

References

18 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. National Library of Medicine. Medline pubmed production statistics. https://www.nlm.nih.gov/bsd/pmresources.html (2022). ↗
  2. Chen, Q., Allot, A. & Lu, Z. LitCovid: an open database of COVID-19 literature. Nucleic Acids Research 49, D1534–D1540, https://doi.org/10.1093/nar/gkaa952 (2020). — [doi:10.1093/nar/gkaa952](https://doi.org/10.1093/nar/gkaa952) ↗
  3. Nentidis, A., Krithara, A. & Paliouras, G. BioASQ website. www.BioASQ.org (2022). find
  4. National Library of Medicine. The Medical Subject Headings (MeSH) thesaurus. https://www.nlm.nih.gov/mesh/meshhome.html (2022). ↗
  5. Linked Life Data (LLD). http://linkedlifedata.com/ (2012). ↗
  6. Wasim, M., Mahmood, D. W. & Khan, D. U. G. A survey of datasets for biomedical question answering systems. International Journal of Advanced Computer Science and Applications 8, https://doi.org/10.14569/IJACSA.2017.080767 (2017). — [doi:10.14569/ijacsa.2017.080767](https://doi.org/10.14569/ijacsa.2017.080767) ↗
  7. Jin, Q. et al. Biomedical question answering: A survey of approaches and challenges. ACM Comput. Surv. 55, https://doi.org/10.1145/3490238 (2022). ↗
  8. Hettne, K. M. et al. A dictionary to identify small molecules and drugs in free text. Bioinformatics 25, 2983–2991, https://doi.org/10.1093/bioinformatics/btp535 (2009). — [doi:10.1093/bioinformatics/btp535](https://doi.org/10.1093/bioinformatics/btp535) ↗
  9. Ashburner, M. et al. Gene ontology: tool for the unification of biology. the gene ontology consortium. Nat Genet 25, 25–29, https://doi.org/10.1038/75556 (2000). — [doi:10.1038/75556](https://doi.org/10.1038/75556) ↗
  10. The Gene Ontology Consortium. The Gene Ontology Resource: 20 years and still GOing strong. Nucleic Acids Research 47, D330–D338, https://doi.org/10.1093/nar/gky1055 (2018). — [doi:10.1093/nar/gky1055](https://doi.org/10.1093/nar/gky1055) ↗
  11. The UniProt Consortium. UniProt: the Universal Protein Knowledgebase. Nucleic Acids Research 51, D523–D531, https://doi.org/10.1093/nar/gkac1052 (2022). — [doi:10.1093/nar/gkac1052](https://doi.org/10.1093/nar/gkac1052) ↗
  12. Schriml, L. M. et al. Human Disease Ontology 2018 update: classification, content and workflow expansion. Nucleic Acids Research 47, D955–D962, https://doi.org/10.1093/nar/gky1032 (2018). — [doi:10.1093/nar/gky1032](https://doi.org/10.1093/nar/gky1032) ↗
  13. Doms, A. & Schroeder, M. GoPubMed: exploring PubMed with the Gene Ontology. Nucleic Acids Research 33, W783–W786, https://doi.org/10.1093/nar/gki470 (2005). — [doi:10.1093/nar/gki470](https://doi.org/10.1093/nar/gki470) ↗
  14. Malakasiotis, P., Androutsopoulos, I., Almirantis, Y., Polychronopoulos, D. & Pavlopoulos, I. Tutorials and guidelines 2 http://www.bioasq.org/sites/default/files/PublicDocuments/BioASQ_D3.7-TutorialsGuidelines2ndVersion_final_0.pdf (2013). ↗
  15. Krithara, A., Nentidis, A., Bougiatiotis, K. & Paliouras, G. BioASQ-QA: A manually curated corpus for biomedical question answering. zenodo https://doi.org/10.5281/zenodo.7655130 (2023). — [doi:10.5281/zenodo.7655130](https://doi.org/10.5281/zenodo.7655130) ↗
  16. Nentidis, A. et al. Overview of BioASQ 2020: The eighth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In 11th International Conference of the CLEF Association, vol. 12260 of Lecture Notes in Computer Science, 194–214, https://doi.org/10.1007/978-3-030-58219-7_1… — [doi:10.1007/978-3-030-58219-7_16](https://doi.org/10.1007/978-3-030-58219-7_16) ↗
  17. Nentidis, A. et al. Overview of BioASQ 2021: The Ninth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In 12th International Conference of the CLEF Association, vol. 12880 of Lecture Notes in Computer Science, 239–263, https://doi.org/10.1007/978-3-030-85251-1_18… — [doi:10.1007/978-3-030-85251-1_18](https://doi.org/10.1007/978-3-030-85251-1_18) ↗
  18. Nentidis, A. et al. Overview of BioASQ 2022: The Tenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In 13th International Conference of the CLEF Association, vol. 13390 of Lecture Notes in Computer Science, 337–361, https://doi.org/10.1007/978-3-031-13643-6_22… — [doi:10.1007/978-3-031-13643-6_22](https://doi.org/10.1007/978-3-031-13643-6_22) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/krithara-2023-bioasq-qa-manually-curated-corpus/