BioASQ Participants Area

Tasks Overview

A short description of the tasks that will be organised during the first yaer of the challenge follows:

  1. Task 1A: Large-scale online biomedical semantic indexing. Large-scale semantic indexing will be evaluated on the whole of MEDLINE. In particular, participants will be asked to classify incoming documents before the human curators do. BioASQ will distribute new unclassified MEDLINE documents every week and participants will have a limited responce time to estimate the MeSH terms.
    2. Task1a-Description
  2. Task 1B: Introductory biomedical semantic QA. Benchmarks containing development and evaluation questions, as well as golden standard (reference) answers, will be developed. The golden answers will be produced by a team of biomedical experts from research teams around Europe. Established methodologies from QA, summarisation, and classification will be followed to produce the benchmarks and evaluate the participating systems The task will run in two phases:
    1. Phase A: BioASQ will transmit simultaneously questions from the benchmark while participants will have to respond with concepts, snippets and triples in limited time.
    2. Phase B: BioASQ will transmit qustions and concepts, snippets ans triples. Participants will respond with facts, summaries, etc. The evaluation will be based on golden answers while a small percentage will be evaluated manually from the biomedical experts.