BioASQ Participants Area
Test Results for MedProcNER Task
The evaluation measures indicating the performance of the systems that submitted results are presented below.
+ NER
Team Name |
Run name |
P |
R |
F1 |
BIT.UA |
run4-everything |
0.8095 |
0.7878 |
0.7985 |
BIT.UA |
run0-lc-dense-5-wVal |
0.8015 |
0.7878 |
0.7946 |
BIT.UA |
run1-lc-dense-5-full |
0.7954 |
0.7894 |
0.7924 |
BIT.UA |
run3-PlanTL-dense-bilstm-all-wVal |
0.7978 |
0.787 |
0.7923 |
BIT.UA |
run2-lc-bilstm-all-wVal |
0.7941 |
0.7823 |
0.7881 |
Vicomtech |
run1-xlm_roberta_large_dpa_e105 |
0.8054 |
0.7535 |
0.7786 |
Vicomtech |
run2-roberta_bio_es_dpa_e119 |
0.7679 |
0.7629 |
0.7653 |
SINAI |
run1-fine-tuned-roberta |
0.7631 |
0.7505 |
0.7568 |
Vicomtech |
run3-longformer_base_4096_bne_es |
0.7478 |
0.7588 |
0.7533 |
SINAI |
run4-fulltext-LSTM |
0.7538 |
0.7353 |
0.7444 |
SINAI |
run2-lstmcrf-512 |
0.7786 |
0.7043 |
0.7396 |
SINAI |
run5-lstm-BIO |
0.7705 |
0.7049 |
0.7362 |
KFU NLP Team |
predicted_task1 |
0.7192 |
0.7403 |
0.7296 |
SINAI |
run3-fulltext-GRU |
0.7396 |
0.711 |
0.725 |
Fusion |
run4-Spanish-RoBERTa |
0.7165 |
0.7143 |
0.7154 |
Fusion |
run3-XLM-RoBERTA-Clinical |
0.7047 |
0.6916 |
0.6981 |
NLP-CIC-WFU |
Hard4BIO_RoBERTa_postprocessing |
0.7188 |
0.654 |
0.6849 |
NLP-CIC-WFU |
Hard4BIO_RoBERTa |
0.7132 |
0.6507 |
0.6805 |
Fusion |
run1-BioMBERT-NumberTagOnly |
0.6948 |
0.6599 |
0.6769 |
Fusion |
run2-BioMBERT-FullPrep |
0.6894 |
0.6599 |
0.6743 |
Fusion |
run5-Adapted-ALBERT |
0.6928 |
0.6264 |
0.658 |
NLP-CIC-WFU |
Lazy4BIO_RoBERTa_postprocessing |
0.6301 |
0.6002 |
0.6148 |
Onto-NLP |
run1-bsс-bio-ehr-pharmaconer-voting-filtered |
0.7425 |
0.4374 |
0.5505 |
Onto-NLP |
run1-bsc-bio-ehr-es-pharmaconer-voting |
0.7397 |
0.4374 |
0.5497 |
Samy Ateia |
run2-gpt-4 |
0.6355 |
0.3874 |
0.4814 |
saheelmayekar |
predicted_data |
0.3975 |
0.535 |
0.4561 |
Onto-NLP |
run1-pharmaconer_filtered_with_exact_match |
0.3296 |
0.6104 |
0.428 |
Samy Ateia |
run1-gpt3.5-turbo |
0.523 |
0.2106 |
0.3002 |
+ Entity Linking
Team Name |
Run name |
P |
R |
F1 |
Vicomtech |
run1-xlm_roberta_large_dpa_e105_sapbert |
0.5902 |
0.5525 |
0.5707 |
Vicomtech |
run2-roberta_bio_es_dpa_e119_sapbert |
0.5665 |
0.5627 |
0.5646 |
Vicomtech |
run3-roberta_bio_es_dpa_e119_sapbert_condition |
0.5662 |
0.5625 |
0.5643 |
Vicomtech |
run5-longformer_base_4096_bne_es_sapbert |
0.5498 |
0.558 |
0.5539 |
Fusion |
run4-Spanish-RoBERTa_predictions |
0.5377 |
0.5362 |
0.5369 |
Fusion |
run1-BioMBERT-NumberTagOnly_XLMRSapBERT.tsv |
0.5432 |
0.516 |
0.5293 |
Fusion |
run3-XLM-RoBERTA-XLMRSapBERT |
0.5332 |
0.5235 |
0.5283 |
SINAI |
run1-fine-tuned-roberta |
0.531 |
0.5224 |
0.5267 |
Vicomtech |
run4-roberta_bio_es_dpa_e119_sapbert_cross_encoder |
0.5248 |
0.5213 |
0.523 |
Fusion |
run2-BioMBERT-FullPrep_XLMRSapBERT |
0.5332 |
0.5105 |
0.5216 |
Fusion |
run5-Adapted-ALBERT_predictions |
0.5461 |
0.4939 |
0.5187 |
SINAI |
run2-lstmcrf-512 |
0.5455 |
0.4936 |
0.5183 |
SINAI |
run5-lstm-BIO |
0.5352 |
0.4898 |
0.5115 |
SINAI |
run4-fulltext-LSTM |
0.5173 |
0.5047 |
0.5109 |
SINAI |
run3-fulltext-GRU |
0.5079 |
0.4884 |
0.498 |
KFU NLP Team |
predicted_task2 |
0.3917 |
0.4033 |
0.3974 |
Onto-NLP |
run1-pharmaconer-top1 |
0.2742 |
0.508 |
0.3562 |
Onto-NLP |
run1-pharmaconer-voter |
0.2723 |
0.5044 |
0.3536 |
Onto-NLP |
run1-cantemist-top1 |
0.2642 |
0.4895 |
0.3432 |
Onto-NLP |
run1-ehr-top1 |
0.263 |
0.4873 |
0.3416 |
BIT.UA |
run4-everything |
0.3211 |
0.3126 |
0.3168 |
BIT.UA |
run3-PlanTL-dense-bilstm-all-wVal |
0.3188 |
0.3145 |
0.3166 |
BIT.UA |
run0-lc-dense-5-wVal |
0.318 |
0.3126 |
0.3153 |
BIT.UA |
run1-lc-dense-5-full |
0.3143 |
0.3121 |
0.3132 |
BIT.UA |
run2-lc-bilstm-all-wVal |
0.3133 |
0.3087 |
0.311 |
Samy Ateia |
run2-gpt-4 |
0.4304 |
0.1282 |
0.1976 |
Samy Ateia |
run1-gpt-3.5-turbo |
0.4051 |
0.0749 |
0.1264 |
+ Document Indexing
Team Name |
Run name |
P |
R |
F1 |
Vicomtech |
run5_roberta_bio_es_dpa_e119_sapbert_condition |
0.619 |
0.6295 |
0.6242 |
Vicomtech |
run4_xlm_roberta_large_dpa_e105_sapbert |
0.6371 |
0.6109 |
0.6239 |
Vicomtech |
run1_roberta_bio_es_dpa_e119_sapbert |
0.6182 |
0.6295 |
0.6238 |
Vicomtech |
run3_longformer_base_4096_bne_es_sapbert |
0.6039 |
0.6288 |
0.6161 |
Vicomtech |
run2_roberta_bio_es_dpa_e119_sapbert_cross_encoder |
0.5885 |
0.5917 |
0.5901 |
KFU NLP Team |
predicted_task3 |
0.4805 |
0.5054 |
0.4927 |
BIT.UA |
run3-PlanTL-dense-bilstm-all-wVal |
0.3544 |
0.3654 |
0.3598 |
BIT.UA |
run4-everything |
0.3551 |
0.3619 |
0.3585 |
BIT.UA |
run0-lc-dense-5-wVal |
0.3517 |
0.3619 |
0.3567 |
BIT.UA |
run1-lc-dense-5-full |
0.3475 |
0.3612 |
0.3542 |
BIT.UA |
run2-lc-bilstm-all-wVal |
0.3484 |
0.3593 |
0.3537 |
Samy Ateia |
run2-gpt-4 |
0.5266 |
0.1811 |
0.2695 |
Samy Ateia |
run1-gpt3.5-turbo |
0.506 |
0.1083 |
0.1785 |