BioASQ Participants Area
Task 3b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in seperate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu_biokdd | - | - | - | - | - |
| auth-qa-1 | 0.1625 | 0.0858 | 0.0981 | 0.0609 | 0.0002 |
| SNUMedinfo1 | 0.2890 | 0.3188 | 0.2468 | 0.2173 | 0.0231 |
| SNUMedinfo2 | 0.2890 | 0.3195 | 0.2471 | 0.2172 | 0.0227 |
| SNUMedinfo3 | 0.2770 | 0.3008 | 0.2335 | 0.2106 | 0.0148 |
| SNUMedinfo4 | 0.2870 | 0.3171 | 0.2454 | 0.2158 | 0.0244 |
| SNUMedinfo5 | 0.2660 | 0.2954 | 0.2261 | 0.1974 | 0.0135 |
| fdu | 0.2630 | 0.3103 | 0.2278 | 0.2014 | 0.0131 |
| fdu2 | 0.2680 | 0.3258 | 0.2362 | 0.2055 | 0.0154 |
| fdu3 | 0.2710 | 0.3291 | 0.2394 | 0.2071 | 0.0165 |
| fdu4 | 0.2730 | 0.3331 | 0.2423 | 0.2066 | 0.0158 |
| ilsp.aueb.1 | 0.1424 | 0.1036 | 0.0932 | 0.0545 | 0.0001 |
| HPI-S2 | 0.1808 | 0.1761 | 0.1418 | 0.0930 | 0.0026 |
| fdu5 | 0.0420 | 0.0349 | 0.0309 | 0.0163 | 0.0000 |
| ustb_prir3 | 0.2910 | 0.3242 | 0.2503 | 0.2104 | 0.0243 |
| ustb_prir1 | 0.2850 | 0.3200 | 0.2455 | 0.2088 | 0.0257 |
| testtext | 0.2890 | 0.3189 | 0.2484 | 0.2094 | 0.0245 |
| ustb_prir4 | 0.2880 | 0.3227 | 0.2484 | 0.2069 | 0.0226 |
| qaiiit system 1 | 0.2380 | 0.1871 | 0.1782 | 0.1405 | 0.0015 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu_biokdd | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | 0.1354 | 0.1262 | 0.1139 | 0.0971 | 0.0018 |
| fdu5 | - | - | - | - | - |
| ustb_prir3 | 0.1207 | 0.1133 | 0.1013 | 0.0797 | 0.0009 |
| ustb_prir1 | 0.1180 | 0.1139 | 0.1003 | 0.0775 | 0.0009 |
| testtext | 0.1230 | 0.1112 | 0.1019 | 0.0752 | 0.0009 |
| ustb_prir4 | 0.1112 | 0.1056 | 0.0927 | 0.0741 | 0.0009 |
| qaiiit system 1 | 0.0918 | 0.0826 | 0.0697 | 0.0789 | 0.0006 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu_biokdd | - | - | - | - | - |
| auth-qa-1 | 0.0190 | 0.0512 | 0.0254 | 0.0053 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | 0.2850 | 0.4260 | 0.2912 | 0.0777 | 0.0046 |
| fdu5 | - | - | - | - | - |
| ustb_prir3 | 0.2194 | 0.2498 | 0.1902 | 0.0529 | 0.0033 |
| ustb_prir1 | 0.2226 | 0.2584 | 0.1944 | 0.0543 | 0.0041 |
| testtext | 0.2173 | 0.2238 | 0.1861 | 0.0531 | 0.0026 |
| ustb_prir4 | 0.2233 | 0.2584 | 0.1949 | 0.0543 | 0.0041 |
| qaiiit system 1 | - | - | - | - | - |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu_biokdd | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| qaiiit system 1 | - | - | - | - | - |
Test batch 2
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth-qa-1 | 0.1598 | 0.1208 | 0.1107 | 0.0703 | 0.0002 |
| HPI-S2 | 0.1592 | 0.1625 | 0.1382 | 0.0915 | 0.0024 |
| SNUMedinfo1 | 0.3380 | 0.3504 | 0.2931 | 0.2620 | 0.0268 |
| SNUMedinfo2 | 0.3360 | 0.3502 | 0.2925 | 0.2594 | 0.0242 |
| SNUMedinfo3 | 0.3370 | 0.3515 | 0.2933 | 0.2560 | 0.0266 |
| SNUMedinfo4 | 0.3350 | 0.3469 | 0.2899 | 0.2614 | 0.0264 |
| SNUMedinfo5 | 0.3240 | 0.3402 | 0.2822 | 0.2485 | 0.0277 |
| qaiiit system 1 | 0.2907 | 0.2449 | 0.2250 | 0.1530 | 0.0039 |
| ustb_prir1 | 0.3200 | 0.3251 | 0.2728 | 0.2452 | 0.0249 |
| ustb_prir2 | 0.3240 | 0.3308 | 0.2770 | 0.2435 | 0.0280 |
| ustb_prir3 | 0.3230 | 0.3293 | 0.2758 | 0.2454 | 0.0268 |
| ustb_prir4 | 0.3230 | 0.3293 | 0.2758 | 0.2453 | 0.0267 |
| testtext | 0.3200 | 0.3281 | 0.2743 | 0.2432 | 0.0266 |
| ilsp.aueb.1 | 0.1156 | 0.1109 | 0.0967 | 0.0714 | 0.0003 |
| ilsp.aueb.2 | 0.1230 | 0.1190 | 0.1040 | 0.0736 | 0.0005 |
| fdu | 0.3460 | 0.3695 | 0.3006 | 0.2831 | 0.0302 |
| fdu2 | 0.3450 | 0.3720 | 0.3013 | 0.2818 | 0.0265 |
| fdu3 | 0.3400 | 0.3665 | 0.2964 | 0.2760 | 0.0259 |
| fdu4 | 0.3470 | 0.3747 | 0.3025 | 0.2715 | 0.0255 |
| fdu5 | 0.3100 | 0.3180 | 0.2608 | 0.2548 | 0.0108 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth-qa-1 | - | - | - | - | - |
| HPI-S2 | 0.1116 | 0.0807 | 0.0819 | 0.0719 | 0.0012 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| qaiiit system 1 | 0.1266 | 0.1112 | 0.1011 | 0.1159 | 0.0011 |
| ustb_prir1 | 0.1071 | 0.0889 | 0.0852 | 0.0740 | 0.0012 |
| ustb_prir2 | 0.1164 | 0.0995 | 0.0947 | 0.0720 | 0.0016 |
| ustb_prir3 | 0.1108 | 0.0915 | 0.0886 | 0.0769 | 0.0013 |
| ustb_prir4 | 0.1258 | 0.1035 | 0.0988 | 0.0776 | 0.0017 |
| testtext | 0.1293 | 0.1052 | 0.1019 | 0.0817 | 0.0020 |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth-qa-1 | 0.0035 | 0.0125 | 0.0054 | 0.0025 | 0.0000 |
| HPI-S2 | 0.3591 | 0.5680 | 0.3994 | 0.1080 | 0.0085 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| qaiiit system 1 | - | - | - | - | - |
| ustb_prir1 | 0.0604 | 0.3135 | 0.0960 | 0.0220 | 0.0003 |
| ustb_prir2 | 0.0624 | 0.3299 | 0.0997 | 0.0223 | 0.0003 |
| ustb_prir3 | 0.0480 | 0.2692 | 0.0785 | 0.0160 | 0.0002 |
| ustb_prir4 | 0.0480 | 0.2692 | 0.0785 | 0.0160 | 0.0002 |
| testtext | 0.0480 | 0.2692 | 0.0785 | 0.0160 | 0.0002 |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth-qa-1 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| qaiiit system 1 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
Test batch 3
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.2110 | 0.1651 | 0.1665 | 0.1383 | 0.0045 |
| auth-qa-1 | 0.1543 | 0.0842 | 0.0948 | 0.0621 | 0.0002 |
| ilsp.aueb.1 | 0.1669 | 0.1384 | 0.1347 | 0.0984 | 0.0011 |
| ilsp.aueb.2 | 0.1389 | 0.1193 | 0.1112 | 0.0730 | 0.0006 |
| oaqa-3b-3 | 0.3240 | 0.3052 | 0.2809 | 0.2424 | 0.0387 |
| fdu2 | 0.3330 | 0.3114 | 0.2917 | 0.2576 | 0.0305 |
| fdu3 | 0.3400 | 0.3401 | 0.3006 | 0.2606 | 0.0425 |
| fdu4 | 0.3430 | 0.3420 | 0.3021 | 0.2636 | 0.0495 |
| fdu5 | 0.3410 | 0.3425 | 0.3016 | 0.2636 | 0.0432 |
| fdu | 0.3420 | 0.3388 | 0.3006 | 0.2622 | 0.0456 |
| ustb_prir1 | 0.3550 | 0.3408 | 0.3133 | 0.2621 | 0.0629 |
| ustb_prir2 | 0.3780 | 0.3784 | 0.3401 | 0.2844 | 0.0794 |
| ustb_prir3 | 0.3890 | 0.3953 | 0.3505 | 0.2904 | 0.1018 |
| ustb_prir4 | 0.3870 | 0.3998 | 0.3508 | 0.2863 | 0.1087 |
| SNUMedinfo1 | 0.3640 | 0.3654 | 0.3286 | 0.2700 | 0.0711 |
| SNUMedinfo2 | 0.3600 | 0.3608 | 0.3247 | 0.2671 | 0.0684 |
| SNUMedinfo3 | 0.3800 | 0.3873 | 0.3430 | 0.2850 | 0.0753 |
| testtext | 0.3780 | 0.3818 | 0.3406 | 0.2814 | 0.0727 |
| qaiiit system 1 | 0.2001 | 0.1325 | 0.1425 | 0.0809 | 0.0006 |
| SNUMedinfo4 | 0.3560 | 0.3526 | 0.3193 | 0.2673 | 0.0785 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.1546 | 0.0934 | 0.1058 | 0.1269 | 0.0015 |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-3 | 0.2309 | 0.1315 | 0.1467 | 0.1969 | 0.0065 |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu | 0.2642 | 0.0862 | 0.1117 | 0.1166 | 0.0018 |
| ustb_prir1 | 0.2115 | 0.1367 | 0.1435 | 0.1720 | 0.0138 |
| ustb_prir2 | 0.2278 | 0.1469 | 0.1558 | 0.1780 | 0.0114 |
| ustb_prir3 | 0.2323 | 0.1532 | 0.1605 | 0.1840 | 0.0131 |
| ustb_prir4 | 0.1870 | 0.1344 | 0.1334 | 0.1185 | 0.0056 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| testtext | 0.1751 | 0.1292 | 0.1266 | 0.1128 | 0.0041 |
| qaiiit system 1 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | 0.1803 | 0.2574 | 0.1889 | 0.0528 | 0.0006 |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-3 | 0.3909 | 0.7341 | 0.4722 | 0.1890 | 0.0991 |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| ustb_prir1 | 0.0351 | 0.1173 | 0.0514 | 0.0096 | 0.0001 |
| ustb_prir2 | 0.0351 | 0.1173 | 0.0514 | 0.0096 | 0.0001 |
| ustb_prir3 | 0.0245 | 0.0867 | 0.0357 | 0.0073 | 0.0000 |
| ustb_prir4 | 0.0246 | 0.0867 | 0.0359 | 0.0073 | 0.0000 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| testtext | 0.0245 | 0.0867 | 0.0357 | 0.0073 | 0.0000 |
| qaiiit system 1 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| HPI-S2 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-3 | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| qaiiit system 1 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
Test batch 4
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | 0.1474 | 0.1155 | 0.1176 | 0.0817 | 0.0007 |
| auth-qa-1 | 0.1229 | 0.0654 | 0.0741 | 0.0483 | 0.0002 |
| ilsp.aueb.2 | 0.1335 | 0.1038 | 0.1053 | 0.0712 | 0.0006 |
| oaqa-3b-4 | 0.3258 | 0.3118 | 0.2811 | 0.2466 | 0.0289 |
| qaiiit system 1 | 0.2817 | 0.2203 | 0.2204 | 0.1602 | 0.0029 |
| ustb_prir1 | 0.3464 | 0.3498 | 0.3081 | 0.2612 | 0.0412 |
| ustb_prir2 | 0.3485 | 0.3514 | 0.3057 | 0.2824 | 0.0546 |
| ustb_prir3 | 0.3567 | 0.3606 | 0.3142 | 0.2891 | 0.0444 |
| ustb_prir4 | 0.3464 | 0.3498 | 0.3081 | 0.2666 | 0.0424 |
| testtext | 0.3608 | 0.3618 | 0.3164 | 0.2851 | 0.0571 |
| SNUMedinfo1 | 0.3443 | 0.3429 | 0.3038 | 0.2624 | 0.0360 |
| SNUMedinfo2 | 0.3371 | 0.3360 | 0.2972 | 0.2594 | 0.0306 |
| SNUMedinfo3 | 0.3577 | 0.3569 | 0.3159 | 0.2796 | 0.0341 |
| SNUMedinfo4 | 0.3443 | 0.3432 | 0.3040 | 0.2620 | 0.0359 |
| SNUMedinfo5 | 0.3526 | 0.3471 | 0.3084 | 0.2757 | 0.0422 |
| HPI-S2 | 0.2483 | 0.2003 | 0.1876 | 0.1601 | 0.0075 |
| fdu | 0.3402 | 0.3320 | 0.2975 | 0.2671 | 0.0354 |
| fdu3 | 0.3443 | 0.3350 | 0.3004 | 0.2695 | 0.0413 |
| fdu2 | 0.3433 | 0.3344 | 0.2996 | 0.2706 | 0.0382 |
| fdu5 | 0.2969 | 0.2934 | 0.2586 | 0.2308 | 0.0202 |
| fdu4 | 0.3423 | 0.3337 | 0.2987 | 0.2669 | 0.0412 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-4 | 0.2622 | 0.1429 | 0.1629 | 0.2092 | 0.0186 |
| qaiiit system 1 | 0.1514 | 0.0888 | 0.1038 | 0.1415 | 0.0013 |
| ustb_prir1 | 0.2483 | 0.1556 | 0.1676 | 0.1933 | 0.0150 |
| ustb_prir2 | 0.2504 | 0.1605 | 0.1689 | 0.1944 | 0.0175 |
| ustb_prir3 | 0.2537 | 0.1634 | 0.1722 | 0.2003 | 0.0190 |
| ustb_prir4 | 0.2618 | 0.1623 | 0.1732 | 0.2005 | 0.0212 |
| testtext | 0.2638 | 0.1673 | 0.1776 | 0.2070 | 0.0217 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | 0.2069 | 0.1334 | 0.1432 | 0.1627 | 0.0053 |
| fdu | 0.3016 | 0.1590 | 0.1827 | 0.2363 | 0.0158 |
| fdu3 | 0.3403 | 0.1657 | 0.1935 | 0.2474 | 0.0168 |
| fdu2 | 0.3462 | 0.1657 | 0.1935 | 0.2480 | 0.0169 |
| fdu5 | - | - | - | - | - |
| fdu4 | 0.3362 | 0.1680 | 0.1952 | 0.2462 | 0.0170 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-4 | 0.3636 | 0.6081 | 0.4133 | 0.1884 | 0.0475 |
| qaiiit system 1 | - | - | - | - | - |
| ustb_prir1 | 0.0533 | 0.1429 | 0.0714 | 0.0150 | 0.0001 |
| ustb_prir2 | 0.0533 | 0.1429 | 0.0714 | 0.0150 | 0.0001 |
| ustb_prir3 | 0.0402 | 0.1108 | 0.0538 | 0.0109 | 0.0001 |
| ustb_prir4 | 0.0533 | 0.1429 | 0.0714 | 0.0150 | 0.0001 |
| testtext | 0.0402 | 0.1108 | 0.0538 | 0.0109 | 0.0001 |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | 0.2127 | 0.2073 | 0.1881 | 0.0668 | 0.0006 |
| fdu | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| oaqa-3b-4 | 0.0167 | 0.0208 | 0.0185 | 0.0017 | 0.0000 |
| qaiiit system 1 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
Test batch 5
Documents
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | 0.1696 | 0.1473 | 0.1375 | 0.1062 | 0.0011 |
| HPI-S2 | 0.2486 | 0.2639 | 0.2073 | 0.1370 | 0.0168 |
| ilsp.aueb.2 | 0.1696 | 0.1473 | 0.1375 | 0.1062 | 0.0011 |
| ilsp.aueb.3 | 0.1291 | 0.0937 | 0.0855 | 0.0583 | 0.0005 |
| SNUMedinfo1 | 0.4057 | 0.4899 | 0.3807 | 0.3257 | 0.1261 |
| SNUMedinfo2 | 0.4097 | 0.4946 | 0.3844 | 0.3284 | 0.1259 |
| SNUMedinfo3 | 0.4047 | 0.4902 | 0.3805 | 0.3214 | 0.1235 |
| SNUMedinfo4 | 0.4037 | 0.4879 | 0.3786 | 0.3264 | 0.1247 |
| SNUMedinfo5 | 0.4167 | 0.5067 | 0.3930 | 0.3333 | 0.1220 |
| oaqa-3b-4 | 0.3850 | 0.4416 | 0.3490 | 0.3115 | 0.1331 |
| ustb_prir1 | 0.4147 | 0.4836 | 0.3852 | 0.3235 | 0.1179 |
| ustb_prir2 | 0.4217 | 0.5066 | 0.3953 | 0.3345 | 0.1314 |
| ustb_prir3 | 0.4217 | 0.5089 | 0.3962 | 0.3362 | 0.1647 |
| ustb_prir4 | 0.4227 | 0.5071 | 0.3959 | 0.3301 | 0.1621 |
| fdu | 0.4350 | 0.5096 | 0.4035 | 0.3593 | 0.1424 |
| fdu2 | 0.4310 | 0.5080 | 0.4015 | 0.3557 | 0.1087 |
| fdu5 | 0.4340 | 0.5101 | 0.4030 | 0.3577 | 0.1390 |
| fdu4 | 0.4380 | 0.4885 | 0.4013 | 0.3583 | 0.1088 |
| fdu3 | 0.4340 | 0.4843 | 0.3962 | 0.3600 | 0.1072 |
Snippets
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | 0.1800 | 0.1582 | 0.1522 | 0.1341 | 0.0060 |
| ilsp.aueb.2 | - | - | - | - | - |
| ilsp.aueb.3 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| oaqa-3b-4 | 0.2882 | 0.2152 | 0.2191 | 0.2191 | 0.0359 |
| ustb_prir1 | 0.2950 | 0.2433 | 0.2403 | 0.2196 | 0.0487 |
| ustb_prir2 | 0.3118 | 0.2556 | 0.2545 | 0.2410 | 0.0610 |
| ustb_prir3 | 0.3134 | 0.2573 | 0.2559 | 0.2383 | 0.0611 |
| ustb_prir4 | 0.3129 | 0.2532 | 0.2530 | 0.2386 | 0.0612 |
| fdu | 0.3364 | 0.2260 | 0.2452 | 0.2417 | 0.0302 |
| fdu2 | 0.3285 | 0.2245 | 0.2415 | 0.2424 | 0.0278 |
| fdu5 | - | - | - | - | - |
| fdu4 | 0.3333 | 0.2197 | 0.2395 | 0.2378 | 0.0254 |
| fdu3 | 0.3321 | 0.2183 | 0.2393 | 0.2423 | 0.0213 |
Concepts
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | 0.2230 | 0.2884 | 0.2206 | 0.0545 | 0.0016 |
| ilsp.aueb.2 | - | - | - | - | - |
| ilsp.aueb.3 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| oaqa-3b-4 | 0.3929 | 0.8250 | 0.4957 | 0.1986 | 0.1021 |
| ustb_prir1 | 0.0303 | 0.0875 | 0.0436 | 0.0068 | 0.0001 |
| ustb_prir2 | 0.0303 | 0.0875 | 0.0436 | 0.0068 | 0.0001 |
| ustb_prir3 | 0.0207 | 0.0375 | 0.0251 | 0.0040 | 0.0000 |
| ustb_prir4 | 0.0207 | 0.0375 | 0.0251 | 0.0040 | 0.0000 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
RDF Triples
| System Name | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| ilsp.aueb.1 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| ilsp.aueb.2 | - | - | - | - | - |
| ilsp.aueb.3 | - | - | - | - | - |
| SNUMedinfo1 | - | - | - | - | - |
| SNUMedinfo2 | - | - | - | - | - |
| SNUMedinfo3 | - | - | - | - | - |
| SNUMedinfo4 | - | - | - | - | - |
| SNUMedinfo5 | - | - | - | - | - |
| oaqa-3b-4 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |