BioASQ Participants Area

Test Results for DisTEMIST TASK

The evaluation measures indicating the performance of the systems that submitted results are presented below.

+ Disease recognition

Team Name System name MiP MiR MiF1
iREL results 0.4984 0.3576 0.4164
NLP-CIC-WFU System_mBERT_ 0.6095 0.4938 0.5456
PU++ run1_mbertD5 0.4540 0.4619 0.4579
PU++ run2_mbertM5 0.6010 0.4488 0.5139
Terminología distemist-subtrack1 0.5622 0.3772 0.4515
Better Innovations Lab / Norwegian Centre for E-health Research run1-ner 0.7724 0.6925 0.7303
Better Innovations Lab / Norwegian Centre for E-health Research run2-ner-limited 0.7926 0.6574 0.7187
PICUSLab NER_results 0.7915 0.7629 0.7770
HPI-DHC 1-roberta-clinical-es-linear-lr 0.7302 0.7363 0.7332
HPI-DHC 2-roberta-clinical-es-constant-lr 0.7302 0.7259 0.7280
HPI-DHC 3-roberta-clinical-es-linear-lr-post-process 0.7434 0.7483 0.7458
HPI-DHC 4-roberta-clinical-es-constant-lr-post-process 0.7417 0.7371 0.7394
SINAI run1-clinical_model 0.7519 0.7221 0.7367
SINAI run2-biomedical_model 0.7520 0.7259 0.7387
Unicage STEM_XXL_LEX_3spc 0.2055* 0.3464* 0.258*
Unicage XL_LEX_3spc 0.2486* 0.3303* 0.2836*
Unicage XL_LEX_spc 0.205* 0.338* 0.2552*
Unicage XXL_LEX_3spc 0.2478* 0.3306* 0.2833*
Unicage XXL_LEX_spc 0.2045* 0.338* 0.2548*
BSC DiseaseTagIt-VT 0.1568 0.4057 0.2262
BSC DiseaseTagIt-Base 0.7146 0.6736 0.6935

* post-workshop



+ Disease linking

Team Name System name MiP MiR MiF1
PU++ run1-scieloBERT 0.2267 0.1494 0.1801
PU++ run2-scieloBERT 0.2754 0.1494 0.1937
Terminología distemist-subtrack2 0.4795 0.2292 0.3102
Better Innovations Lab / Norwegian Centre for E-health Research run1-snomed 0.5478 0.4577 0.4987
Better Innovations Lab / Norwegian Centre for E-health Research run2-snomed-limited 0.5497 0.4549 0.4978
PICUSLab EL_results 0.2814 0.2748 0.2780
HPI-DHC 1-tf_idf_ngrams_distemist 0.3576 0.3646 0.3611
HPI-DHC 2-sap_umls_large_distemist 0.3641 0.3738 0.3689
HPI-DHC 3-ensemble 0.4678 0.3890 0.4248
HPI-DHC 4-ensemble-reranking 0.5427 0.4513 0.4928
HPI-DHC 5-ensemble-reranking-postprocess 0.6207 0.5196 0.5657
SINAI run1-clinical_model.prepro 0.4163 0.4081 0.4122
SINAI run2-biomedical_model.prepro 0.4134 0.4069 0.4101
BSC DiseaseTagIt-VT 0.1003 0.1621 0.1240
BSC DiseaseTagIt-Base 0.3041 0.2336 0.2642