BioASQ Participants Area
Task 2b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in seperate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.2595 | 0.1919 | 0.1678 | 0.1040 | 0.0017 |
| Wishart-S1 | 0.1447 | 0.1082 | 0.0858 | 0.0543 | 0.0002 |
| SNUMedinfo1 | 0.0658 | 0.6114 | 0.1131 | 0.2794 | 0.0773 |
| SNUMedinfo3 | 0.0661 | 0.6155 | 0.1136 | 0.2762 | 0.0744 |
| SNUMedinfo4 | 0.0663 | 0.6154 | 0.1139 | 0.2675 | 0.0694 |
| SNUMedinfo5 | 0.0667 | 0.6161 | 0.1146 | 0.2596 | 0.0662 |
| SNUMedinfo2 | 0.0655 | 0.6030 | 0.1125 | 0.2727 | 0.0686 |
| All-Figdoc | 0.0260 | 0.0528 | 0.0319 | 0.0082 | 0.0001 |
| UMass-irSDM | 0.0275 | 0.0558 | 0.0339 | 0.0271 | 0.0002 |
| Doc-Figdoc-UMLS | 0.0275 | 0.0558 | 0.0339 | 0.0081 | 0.0002 |
| All-Figdoc-UMLS | 0.0275 | 0.0558 | 0.0339 | 0.0087 | 0.0002 |
| Wishart-S2 | 0.1223 | 0.1248 | 0.0837 | 0.0601 | 0.0002 |
| main system | 0.0447 | 0.2453 | 0.0713 | 0.1108 | 0.0018 |
| Top 50 Baseline | 0.2495 | 0.3671 | 0.2275 | 0.1735 | 0.0062 |
| Top 100 Baseline | 0.2477 | 0.3994 | 0.2269 | 0.1766 | 0.0076 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.0603 | 0.0389 | 0.0358 | 0.0296 | 0.0002 |
| Wishart-S1 | 0.0745 | 0.0583 | 0.0453 | 0.0314 | 0.0001 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| All-Figdoc | 0.0000 | 0.0000 | 0.0000 | 0.0001 | 0.0000 |
| UMass-irSDM | 0.0000 | 0.0000 | 0.0000 | 0.0010 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0001 | 0.0000 |
| All-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0002 | 0.0000 |
| Wishart-S2 | 0.0631 | 0.0645 | 0.0435 | 0.0364 | 0.0001 |
| main system | 0.0046 | 0.1910 | 0.0088 | 0.0095 | 0.0003 |
| Top 50 Baseline | 0.0800 | 0.0867 | 0.0629 | 0.0601 | 0.0009 |
| Top 100 Baseline | 0.0798 | 0.0977 | 0.0631 | 0.0608 | 0.0011 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.1368 | 0.0941 | 0.0986 | 0.0365 | 0.0003 |
| Wishart-S1 | 0.5231 | 0.5412 | 0.4683 | 0.6857 | 0.2577 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Wishart-S2 | 0.5231 | 0.5412 | 0.4683 | 0.6857 | 0.2577 |
| main system | 0.4247 | 0.5441 | 0.4074 | 0.3872 | 0.1199 |
| Top 50 Baseline | 0.0974 | 0.8271 | 0.1629 | 0.5346 | 0.3743 |
| Top 100 Baseline | 0.0584 | 0.8819 | 0.1038 | 0.5403 | 0.4151 |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | - | - | - | - | - |
| Wishart-S1 | 0.0688 | 0.0023 | 0.0044 | 0.0017 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Wishart-S2 | 0.0688 | 0.0023 | 0.0044 | 0.0017 | 0.0000 |
| main system | - | - | - | - | - |
| Top 50 Baseline | 0.1208 | 0.1669 | 0.1038 | 0.0557 | 0.0007 |
| Top 100 Baseline | 0.0950 | 0.2496 | 0.1133 | 0.0658 | 0.0015 |
Test batch 2
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| SNUMedinfo1 | 0.0918 | 0.5880 | 0.1487 | 0.3013 | 0.1337 |
| SNUMedinfo2 | 0.0918 | 0.5891 | 0.1489 | 0.2995 | 0.1312 |
| SNUMedinfo3 | 0.0911 | 0.5859 | 0.1478 | 0.3016 | 0.1339 |
| SNUMedinfo4 | 0.0916 | 0.5894 | 0.1487 | 0.3015 | 0.1321 |
| All-Abstract-UMLS | 0.0215 | 0.0223 | 0.0210 | 0.0087 | 0.0000 |
| All-Figdoc | 0.0190 | 0.0225 | 0.0200 | 0.0063 | 0.0000 |
| All-Figdoc-UMLS | 0.0205 | 0.0228 | 0.0207 | 0.0068 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0160 | 0.0197 | 0.0167 | 0.0051 | 0.0000 |
| UMass-irSDM | 0.0485 | 0.0683 | 0.0511 | 0.0393 | 0.0006 |
| SNUMedinfo5 | 0.0914 | 0.5875 | 0.1483 | 0.3007 | 0.1323 |
| main system | 0.0593 | 0.2340 | 0.0885 | 0.0797 | 0.0019 |
| Wishart-S1 | 0.1004 | 0.0840 | 0.0652 | 0.0411 | 0.0001 |
| Wishart-S2 | 0.1443 | 0.0982 | 0.0804 | 0.0483 | 0.0002 |
| Wishart-S3 | 0.1237 | 0.1037 | 0.0796 | 0.0465 | 0.0002 |
| fdu_biokdd | 0.3144 | 0.4948 | 0.2902 | 0.2508 | 0.0564 |
| HPI-S1 | 0.0300 | 0.1292 | 0.0446 | 0.0709 | 0.0008 |
| Top 50 Baseline | 0.2775 | 0.2416 | 0.1808 | 0.1134 | 0.0034 |
| Top 100 Baseline | 0.2756 | 0.2708 | 0.1811 | 0.1167 | 0.0043 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| All-Abstract-UMLS | 0.0000 | 0.0000 | 0.0000 | -0.0001 | 0.0000 |
| All-Figdoc | 0.0000 | 0.0000 | 0.0000 | -0.0035 | 0.0000 |
| All-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0005 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0003 | 0.0000 |
| UMass-irSDM | 0.0000 | 0.0000 | 0.0000 | 0.0023 | 0.0000 |
| SNUMedinfo5 | - | - | - | - | - |
| main system | 0.0058 | 0.2072 | 0.0110 | 0.0062 | 0.0003 |
| Wishart-S1 | 0.0426 | 0.0495 | 0.0329 | 0.0330 | 0.0001 |
| Wishart-S2 | 0.0702 | 0.0606 | 0.0423 | 0.0379 | 0.0001 |
| Wishart-S3 | 0.0475 | 0.0592 | 0.0379 | 0.0367 | 0.0001 |
| fdu_biokdd | - | - | - | - | - |
| HPI-S1 | 0.0097 | 0.0746 | 0.0163 | 0.0482 | 0.0004 |
| Top 50 Baseline | 0.0752 | 0.0612 | 0.0463 | 0.0313 | 0.0002 |
| Top 100 Baseline | 0.0748 | 0.0769 | 0.0461 | 0.0319 | 0.0003 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| main system | 0.4047 | 0.5489 | 0.4109 | 0.3822 | 0.1151 |
| Wishart-S1 | 0.3956 | 0.5415 | 0.4157 | 0.4960 | 0.1667 |
| Wishart-S2 | 0.3956 | 0.5415 | 0.4157 | 0.4960 | 0.1667 |
| Wishart-S3 | 0.3956 | 0.5415 | 0.4157 | 0.4960 | 0.1667 |
| fdu_biokdd | 0.0184 | 0.0785 | 0.0171 | 0.0551 | 0.0001 |
| HPI-S1 | - | - | - | - | - |
| Top 50 Baseline | 0.0793 | 0.8479 | 0.1405 | 0.5458 | 0.3406 |
| Top 100 Baseline | 0.0441 | 0.8947 | 0.0819 | 0.5488 | 0.3711 |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| main system | - | - | - | - | - |
| Wishart-S1 | 0.0341 | 0.0106 | 0.0153 | 0.0038 | 0.0000 |
| Wishart-S2 | 0.0341 | 0.0106 | 0.0153 | 0.0038 | 0.0000 |
| Wishart-S3 | 0.0341 | 0.0106 | 0.0153 | 0.0038 | 0.0000 |
| fdu_biokdd | - | - | - | - | - |
| HPI-S1 | - | - | - | - | - |
| Top 50 Baseline | 0.0727 | 0.2380 | 0.0590 | 0.0917 | 0.0003 |
| Top 100 Baseline | 0.0523 | 0.2512 | 0.0471 | 0.0926 | 0.0006 |
Test batch 3
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.0246 | 0.1664 | 0.0382 | 0.0900 | 0.0009 |
| BTMG2 | 0.2770 | 0.1715 | 0.1553 | 0.1022 | 0.0018 |
| Biomedical Text Ming | 0.2770 | 0.1715 | 0.1553 | 0.1009 | 0.0018 |
| SNUMedinfo1 | 0.0715 | 0.5795 | 0.1194 | 0.2875 | 0.0537 |
| SNUMedinfo2 | 0.0705 | 0.5739 | 0.1178 | 0.2918 | 0.0536 |
| SNUMedinfo3 | 0.0716 | 0.5788 | 0.1195 | 0.2852 | 0.0528 |
| SNUMedinfo4 | 0.0700 | 0.5696 | 0.1169 | 0.2875 | 0.0527 |
| SNUMedinfo5 | 0.0722 | 0.5833 | 0.1206 | 0.2874 | 0.0540 |
| UMass-irSDM | 0.0395 | 0.0696 | 0.0417 | 0.0456 | 0.0002 |
| Doc-Figdoc-UMLS | 0.0070 | 0.0078 | 0.0065 | 0.0011 | 0.0000 |
| All-Figdoc | 0.0135 | 0.0144 | 0.0127 | 0.0019 | 0.0000 |
| All-Abstract-UMLS | 0.0135 | 0.0257 | 0.0141 | 0.0051 | 0.0000 |
| Wishart-S1 | 0.1002 | 0.1325 | 0.0902 | 0.0550 | 0.0003 |
| Wishart-S2 | 0.1002 | 0.1325 | 0.0902 | 0.0550 | 0.0003 |
| Wishart-S3 | 0.0931 | 0.1389 | 0.0958 | 0.0566 | 0.0003 |
| Wishart-S4 | 0.0931 | 0.1389 | 0.0958 | 0.0566 | 0.0003 |
| fdu_biokdd | 0.2730 | 0.5128 | 0.2766 | 0.2773 | 0.0538 |
| Top 50 Baseline | 0.2506 | 0.3243 | 0.2163 | 0.1707 | 0.0038 |
| Top 100 Baseline | 0.2471 | 0.3513 | 0.2131 | 0.1728 | 0.0044 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.0107 | 0.0857 | 0.0172 | 0.0517 | 0.0002 |
| BTMG2 | 0.0746 | 0.0492 | 0.0443 | 0.0225 | 0.0002 |
| Biomedical Text Ming | 0.0746 | 0.0492 | 0.0443 | 0.0215 | 0.0002 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| UMass-irSDM | 0.0000 | 0.0000 | 0.0000 | 0.0023 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | -0.0001 | 0.0000 |
| All-Figdoc | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| All-Abstract-UMLS | - | - | - | - | - |
| Wishart-S1 | 0.0501 | 0.1264 | 0.0516 | 0.0522 | 0.0002 |
| Wishart-S2 | 0.0501 | 0.1264 | 0.0516 | 0.0522 | 0.0002 |
| Wishart-S3 | 0.0417 | 0.1268 | 0.0522 | 0.0574 | 0.0002 |
| Wishart-S4 | 0.0417 | 0.1268 | 0.0522 | 0.0574 | 0.0002 |
| fdu_biokdd | - | - | - | - | - |
| Top 50 Baseline | 0.0686 | 0.0777 | 0.0596 | 0.0480 | 0.0002 |
| Top 100 Baseline | 0.0668 | 0.0789 | 0.0577 | 0.0486 | 0.0002 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.1244 | 0.1244 | 0.1044 | 0.0530 | 0.0006 |
| BTMG2 | 0.1477 | 0.1288 | 0.1174 | 0.0437 | 0.0005 |
| Biomedical Text Ming | 0.1477 | 0.1288 | 0.1174 | 0.0437 | 0.0005 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| Wishart-S1 | 0.3982 | 0.4718 | 0.3861 | 0.4002 | 0.1053 |
| Wishart-S2 | 0.3982 | 0.4718 | 0.3861 | 0.4002 | 0.1053 |
| Wishart-S3 | 0.3982 | 0.4718 | 0.3861 | 0.4002 | 0.1053 |
| Wishart-S4 | 0.3982 | 0.4718 | 0.3861 | 0.4002 | 0.1053 |
| fdu_biokdd | 0.0240 | 0.1005 | 0.0288 | 0.0540 | 0.0001 |
| Top 50 Baseline | 0.0790 | 0.7430 | 0.1326 | 0.4289 | 0.2101 |
| Top 100 Baseline | 0.0459 | 0.7982 | 0.0815 | 0.4339 | 0.2174 |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | - | - | - | - | - |
| BTMG2 | 0.0556 | 0.0026 | 0.0051 | 0.0027 | 0.0000 |
| Biomedical Text Ming | 0.0556 | 0.0026 | 0.0051 | 0.0027 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| Wishart-S1 | 0.1389 | 0.0039 | 0.0075 | 0.0030 | 0.0001 |
| Wishart-S2 | 0.1389 | 0.0039 | 0.0075 | 0.0030 | 0.0001 |
| Wishart-S3 | 0.1389 | 0.0039 | 0.0075 | 0.0030 | 0.0001 |
| Wishart-S4 | 0.1389 | 0.0039 | 0.0075 | 0.0030 | 0.0001 |
| fdu_biokdd | - | - | - | - | - |
| Top 50 Baseline | 0.1567 | 0.0888 | 0.0866 | 0.0374 | 0.0004 |
| Top 100 Baseline | 0.1239 | 0.1330 | 0.0878 | 0.0451 | 0.0006 |
Test batch 4
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.1929 | 0.1497 | 0.1420 | 0.0881 | 0.0009 |
| BTMG2 | 0.1929 | 0.1497 | 0.1420 | 0.0898 | 0.0009 |
| All-Abstract-UMLS | 0.0275 | 0.0317 | 0.0276 | 0.0068 | 0.0001 |
| All-Figdoc | 0.0260 | 0.0332 | 0.0271 | 0.0085 | 0.0001 |
| All-Figdoc-UMLS | 0.0295 | 0.0359 | 0.0302 | 0.0085 | 0.0001 |
| Doc-Figdoc-UMLS | 0.0245 | 0.0283 | 0.0248 | 0.0075 | 0.0001 |
| UMass-irSDM | 0.0490 | 0.0754 | 0.0512 | 0.0438 | 0.0003 |
| Wishart-S1 | 0.0965 | 0.1283 | 0.0882 | 0.0476 | 0.0003 |
| Wishart-S2 | 0.0933 | 0.1223 | 0.0847 | 0.0473 | 0.0003 |
| Wishart-S3 | 0.0965 | 0.1283 | 0.0882 | 0.0476 | 0.0003 |
| SNUMedinfo1 | 0.0707 | 0.5469 | 0.1157 | 0.2663 | 0.0715 |
| SNUMedinfo2 | 0.0696 | 0.5435 | 0.1142 | 0.2650 | 0.0705 |
| SNUMedinfo3 | 0.0695 | 0.5441 | 0.1143 | 0.2713 | 0.0678 |
| SNUMedinfo4 | 0.0685 | 0.5415 | 0.1129 | 0.2696 | 0.0670 |
| SNUMedinfo5 | 0.0697 | 0.5478 | 0.1146 | 0.2615 | 0.0712 |
| HPI-S2 | 0.0177 | 0.1362 | 0.0294 | 0.0554 | 0.0003 |
| Top 50 Baseline | 0.2293 | 0.3208 | 0.2147 | 0.1532 | 0.0066 |
| Top 100 Baseline | 0.2249 | 0.3596 | 0.2115 | 0.1574 | 0.0072 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.0535 | 0.0444 | 0.0380 | 0.0240 | 0.0002 |
| BTMG2 | 0.0535 | 0.0444 | 0.0380 | 0.0240 | 0.0001 |
| All-Abstract-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0002 | 0.0000 |
| All-Figdoc | 0.0000 | 0.0000 | 0.0000 | 0.0002 | 0.0000 |
| All-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0002 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0003 | 0.0000 |
| UMass-irSDM | 0.0000 | 0.0000 | 0.0000 | 0.0011 | 0.0000 |
| Wishart-S1 | 0.0461 | 0.1099 | 0.0443 | 0.0503 | 0.0002 |
| Wishart-S2 | 0.0467 | 0.1134 | 0.0450 | 0.0485 | 0.0002 |
| Wishart-S3 | 0.0461 | 0.1099 | 0.0443 | 0.0503 | 0.0002 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | 0.0071 | 0.0882 | 0.0125 | 0.0300 | 0.0001 |
| Top 50 Baseline | 0.0923 | 0.1076 | 0.0737 | 0.0539 | 0.0008 |
| Top 100 Baseline | 0.0907 | 0.1213 | 0.0721 | 0.0549 | 0.0010 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.5116 | 0.3529 | 0.3869 | 0.2837 | 0.0511 |
| BTMG2 | 0.5116 | 0.3529 | 0.3869 | 0.2837 | 0.0511 |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Wishart-S1 | 0.4713 | 0.4102 | 0.3956 | 0.3671 | 0.0956 |
| Wishart-S2 | 0.4713 | 0.4102 | 0.3956 | 0.3671 | 0.0956 |
| Wishart-S3 | 0.4713 | 0.4102 | 0.3956 | 0.3671 | 0.0956 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | 0.1420 | 0.1185 | 0.1178 | 0.0554 | 0.0006 |
| Top 50 Baseline | 0.0979 | 0.7763 | 0.1584 | 0.4859 | 0.3615 |
| Top 100 Baseline | 0.0601 | 0.8316 | 0.0996 | 0.4915 | 0.3693 |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | - | - | - | - | - |
| BTMG2 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Wishart-S1 | 0.1014 | 0.0112 | 0.0188 | 0.0112 | 0.0000 |
| Wishart-S2 | 0.1014 | 0.0112 | 0.0188 | 0.0112 | 0.0000 |
| Wishart-S3 | 0.1014 | 0.0112 | 0.0188 | 0.0112 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| Top 50 Baseline | 0.0643 | 0.2589 | 0.0698 | 0.0839 | 0.0010 |
| Top 100 Baseline | 0.0370 | 0.2686 | 0.0468 | 0.0847 | 0.0011 |
Test batch 5
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.2223 | 0.1446 | 0.1396 | 0.0883 | 0.0008 |
| BTMG2 | 0.2223 | 0.1446 | 0.1396 | 0.0889 | 0.0008 |
| SNUMedinfo1 | 0.0580 | 0.5178 | 0.0995 | 0.2647 | 0.0476 |
| SNUMedinfo2 | 0.0577 | 0.5127 | 0.0988 | 0.2612 | 0.0458 |
| SNUMedinfo3 | 0.0581 | 0.5188 | 0.0997 | 0.2637 | 0.0469 |
| SNUMedinfo4 | 0.0588 | 0.5199 | 0.1007 | 0.2661 | 0.0450 |
| SNUMedinfo5 | 0.0605 | 0.5333 | 0.1037 | 0.2657 | 0.0589 |
| All-Abstract-UMLS | 0.0290 | 0.0499 | 0.0346 | 0.0165 | 0.0001 |
| All-Figdoc | 0.0240 | 0.0401 | 0.0279 | 0.0080 | 0.0001 |
| All-Figdoc-UMLS | 0.0235 | 0.0411 | 0.0275 | 0.0086 | 0.0001 |
| Doc-Figdoc-UMLS | 0.0165 | 0.0312 | 0.0202 | 0.0062 | 0.0001 |
| UMass-irSDM | 0.0380 | 0.0639 | 0.0432 | 0.0349 | 0.0004 |
| Wishart-S1 | 0.1195 | 0.1901 | 0.1184 | 0.0620 | 0.0015 |
| Wishart-S2 | 0.1195 | 0.1901 | 0.1184 | 0.0620 | 0.0015 |
| Top 50 Baseline | 0.2823 | 0.2424 | 0.2027 | 0.1531 | 0.0028 |
| Top 100 Baseline | 0.2816 | 0.2683 | 0.2032 | 0.1558 | 0.0032 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.0598 | 0.0375 | 0.0387 | 0.0195 | 0.0001 |
| BTMG2 | 0.0598 | 0.0375 | 0.0387 | 0.0196 | 0.0001 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| All-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| Doc-Figdoc-UMLS | 0.0000 | 0.0000 | 0.0000 | 0.0002 | 0.0000 |
| UMass-irSDM | 0.0000 | 0.0000 | 0.0000 | 0.0006 | 0.0000 |
| Wishart-S1 | 0.0464 | 0.1091 | 0.0453 | 0.0476 | 0.0006 |
| Wishart-S2 | 0.0464 | 0.1091 | 0.0453 | 0.0476 | 0.0006 |
| Top 50 Baseline | 0.1104 | 0.0701 | 0.0665 | 0.0539 | 0.0004 |
| Top 100 Baseline | 0.1102 | 0.0733 | 0.0667 | 0.0544 | 0.0004 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.5029 | 0.3932 | 0.4051 | 0.2994 | 0.0852 |
| BTMG2 | 0.5029 | 0.3932 | 0.4051 | 0.2994 | 0.0852 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Wishart-S1 | 0.4510 | 0.4129 | 0.3833 | 0.3835 | 0.0985 |
| Wishart-S2 | 0.4510 | 0.4129 | 0.3833 | 0.3835 | 0.0985 |
| Top 50 Baseline | 0.0763 | 0.7413 | 0.1339 | 0.4711 | 0.2891 |
| Top 100 Baseline | 0.0429 | 0.8028 | 0.0799 | 0.4757 | 0.3163 |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Biomedical Text Ming | 0.0400 | 0.0010 | 0.0019 | 0.0009 | 0.0000 |
| BTMG2 | 0.0400 | 0.0010 | 0.0019 | 0.0009 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| All-Abstract-UMLS | - | - | - | - | - |
| All-Figdoc | - | - | - | - | - |
| All-Figdoc-UMLS | - | - | - | - | - |
| Doc-Figdoc-UMLS | - | - | - | - | - |
| UMass-irSDM | - | - | - | - | - |
| Wishart-S1 | 0.0533 | 0.0021 | 0.0036 | 0.0021 | 0.0000 |
| Wishart-S2 | 0.0533 | 0.0021 | 0.0036 | 0.0021 | 0.0000 |
| Top 50 Baseline | 0.0952 | 0.2263 | 0.0659 | 0.0913 | 0.0008 |
| Top 100 Baseline | 0.0540 | 0.2472 | 0.0477 | 0.0929 | 0.0009 |