BioASQ Participants Area
Test Results for Task 7a
The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.
Annotated articles:16780/17711
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
BioASQ_Baseline |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
BioASQ_Baseline |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
+ Dry Run 2
Annotated articles:5408/5577
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
NLM System 4 |
0.6561 |
0.6900 |
0.6371 |
0.6449 |
0.6237 |
0.5146 |
0.5090 |
0.6838 |
0.6306 |
0.4920 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
NLM System 4 |
0.5252 |
0.8082 |
0.7248 |
0.7475 |
0.5668 |
0.5121 |
+ Test batch 1, week 1
Annotated articles:7194/7358
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6268 |
0.6730 |
0.6050 |
0.6170 |
0.6284 |
0.5552 |
0.5320 |
0.6610 |
0.5960 |
0.4640 |
Default MTI |
0.6397 |
0.6419 |
0.6586 |
0.6322 |
0.5937 |
0.5953 |
0.5524 |
0.6328 |
0.6468 |
0.4797 |
DeepMeSH1 |
0.6628 |
0.7286 |
0.6226 |
0.6564 |
0.6886 |
0.5137 |
0.5139 |
0.7319 |
0.6056 |
0.5053 |
DeepMeSH2 |
0.6382 |
0.7160 |
0.5883 |
0.6305 |
0.6705 |
0.4526 |
0.4599 |
0.7213 |
0.5724 |
0.4767 |
DeepMeSH3 |
0.6607 |
0.7260 |
0.6209 |
0.6544 |
0.6870 |
0.5108 |
0.5116 |
0.7293 |
0.6039 |
0.5034 |
MeSHProbeNet-P |
0.6849 |
0.7141 |
0.6714 |
0.6753 |
0.6728 |
0.5670 |
0.5556 |
0.7090 |
0.6624 |
0.5264 |
NLM System 4 |
0.6515 |
0.6840 |
0.6343 |
0.6404 |
0.6125 |
0.5045 |
0.4967 |
0.6768 |
0.6280 |
0.4873 |
MeSHProbeNet-P0 |
0.6848 |
0.7024 |
0.6817 |
0.6753 |
0.6604 |
0.5781 |
0.5609 |
0.6970 |
0.6730 |
0.5262 |
MeSHProbeNet-P1 |
0.6839 |
0.6903 |
0.6917 |
0.6746 |
0.6486 |
0.5893 |
0.5657 |
0.6846 |
0.6832 |
0.5252 |
MeSHProbeNet-P2 |
0.6849 |
0.7140 |
0.6714 |
0.6753 |
0.6728 |
0.5670 |
0.5556 |
0.7089 |
0.6624 |
0.5264 |
MeSHProbeNet-P3 |
0.6843 |
0.7128 |
0.6711 |
0.6746 |
0.6700 |
0.5651 |
0.5529 |
0.7077 |
0.6624 |
0.5256 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |