BioASQ Participants Area
Task 5b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in seperate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
KNU-SG Team_Korea |
0.2390 |
0.1911 |
0.1598 |
0.1992 |
0.0049 |
HPI-S1 |
0.2554 |
0.2082 |
0.1683 |
0.1577 |
0.0040 |
UNCC System 1 |
0.3478 |
0.2698 |
0.2544 |
0.1840 |
0.0042 |
Olelo |
0.3207 |
0.2206 |
0.1998 |
0.1869 |
0.0049 |
c-idf-qe-1 |
0.2350 |
0.2009 |
0.1584 |
0.1716 |
0.0071 |
c-e-200-dup |
0.2350 |
0.2009 |
0.1584 |
0.1716 |
0.0071 |
c-e-50 |
0.2350 |
0.2009 |
0.1584 |
0.1716 |
0.0071 |
c-f-200 |
0.2350 |
0.2009 |
0.1584 |
0.1716 |
0.0071 |
testtext |
0.4207 |
0.4540 |
0.3356 |
0.3250 |
0.0507 |
ustb_prir1 |
0.4327 |
0.4665 |
0.3475 |
0.3388 |
0.0565 |
ustb_prir2 |
0.4240 |
0.4720 |
0.3375 |
0.3494 |
0.0624 |
ustb_prir3 |
0.4180 |
0.4736 |
0.3356 |
0.3467 |
0.0657 |
ustb_prir4 |
0.3920 |
0.4513 |
0.3133 |
0.3132 |
0.0615 |
fdu |
0.4296 |
0.4567 |
0.3496 |
0.3261 |
0.0586 |
fdu2 |
0.4025 |
0.4459 |
0.3306 |
0.3119 |
0.0477 |
fdu3 |
0.3830 |
0.4221 |
0.3012 |
0.3247 |
0.0346 |
fdu4 |
0.3740 |
0.4221 |
0.2991 |
0.3138 |
0.0317 |
fdu5 |
0.3133 |
0.2487 |
0.2243 |
0.2488 |
0.0090 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
KNU-SG Team_Korea |
0.1670 |
0.0860 |
0.0984 |
0.1393 |
0.0016 |
HPI-S1 |
0.1940 |
0.1453 |
0.1441 |
0.1401 |
0.0017 |
UNCC System 1 |
- |
- |
- |
- |
- |
Olelo |
0.2034 |
0.1564 |
0.1484 |
0.1583 |
0.0020 |
c-idf-qe-1 |
0.1655 |
0.0870 |
0.0921 |
0.1355 |
0.0012 |
c-e-200-dup |
0.1945 |
0.1186 |
0.1188 |
0.0881 |
0.0008 |
c-e-50 |
0.1913 |
0.1010 |
0.1096 |
0.1566 |
0.0023 |
c-f-200 |
0.1621 |
0.0851 |
0.0927 |
0.1346 |
0.0010 |
testtext |
0.2282 |
0.1363 |
0.1375 |
0.1585 |
0.0110 |
ustb_prir1 |
0.2242 |
0.1310 |
0.1334 |
0.1703 |
0.0109 |
ustb_prir2 |
0.2324 |
0.1420 |
0.1434 |
0.1747 |
0.0115 |
ustb_prir3 |
0.2278 |
0.1379 |
0.1383 |
0.1620 |
0.0118 |
ustb_prir4 |
0.2297 |
0.1349 |
0.1354 |
0.1692 |
0.0143 |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
HPI-S1 |
0.4871 |
0.2033 |
0.2750 |
0.0914 |
0.0021 |
UNCC System 1 |
0.3136 |
0.5862 |
0.3548 |
0.1543 |
0.0570 |
Olelo |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-e-200-dup |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
testtext |
0.2824 |
0.5968 |
0.3176 |
0.1418 |
0.0341 |
ustb_prir1 |
0.1930 |
0.5375 |
0.2545 |
0.1206 |
0.0218 |
ustb_prir2 |
0.2800 |
0.6626 |
0.3395 |
0.1695 |
0.0632 |
ustb_prir3 |
0.2086 |
0.6119 |
0.2884 |
0.1477 |
0.0321 |
ustb_prir4 |
0.2800 |
0.6626 |
0.3395 |
0.1695 |
0.0632 |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
HPI-S1 |
- |
- |
- |
- |
- |
UNCC System 1 |
- |
- |
- |
- |
- |
Olelo |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-e-200-dup |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
testtext |
- |
- |
- |
- |
- |
ustb_prir1 |
- |
- |
- |
- |
- |
ustb_prir2 |
- |
- |
- |
- |
- |
ustb_prir3 |
- |
- |
- |
- |
- |
ustb_prir4 |
- |
- |
- |
- |
- |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth |
- |
- |
- |
- |
- |
HPI-S1 |
0.3135 |
0.2195 |
0.1932 |
0.1823 |
0.0052 |
Olelo |
0.3250 |
0.2188 |
0.1995 |
0.2026 |
0.0052 |
c-e-50 |
0.3040 |
0.2370 |
0.2035 |
0.2485 |
0.0052 |
c-50 |
0.3040 |
0.2370 |
0.2035 |
0.2485 |
0.0052 |
c-idf-qe-1 |
0.3040 |
0.2370 |
0.2035 |
0.2485 |
0.0052 |
c-f-200 |
0.3040 |
0.2370 |
0.2035 |
0.2485 |
0.0052 |
KNU-SG Team_Korea |
0.3050 |
0.2627 |
0.2086 |
0.2622 |
0.0076 |
testtext |
0.4370 |
0.4930 |
0.3343 |
0.3843 |
0.0821 |
ustb_prir1 |
0.4320 |
0.4899 |
0.3307 |
0.3717 |
0.0852 |
ustb_prir2 |
0.4290 |
0.4844 |
0.3276 |
0.3731 |
0.0746 |
ustb_prir3 |
0.4430 |
0.4952 |
0.3380 |
0.3880 |
0.0821 |
ustb_prir4 |
0.4450 |
0.4949 |
0.3429 |
0.3777 |
0.0886 |
fdu |
0.4090 |
0.4661 |
0.3171 |
0.3374 |
0.0458 |
fdu2 |
0.4120 |
0.4659 |
0.3187 |
0.3435 |
0.0524 |
fdu3 |
0.4000 |
0.4497 |
0.3038 |
0.3445 |
0.0350 |
fdu4 |
0.4120 |
0.4630 |
0.3131 |
0.3501 |
0.0469 |
Basic QA pipline |
- |
- |
- |
- |
- |
QAauth |
- |
- |
- |
- |
- |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth |
- |
- |
- |
- |
- |
HPI-S1 |
0.2304 |
0.1678 |
0.1754 |
0.1841 |
0.0032 |
Olelo |
0.2014 |
0.1378 |
0.1370 |
0.1595 |
0.0019 |
c-e-50 |
0.2166 |
0.0974 |
0.1156 |
0.1993 |
0.0022 |
c-50 |
0.2010 |
0.1009 |
0.1106 |
0.1955 |
0.0014 |
c-idf-qe-1 |
0.2115 |
0.0923 |
0.1120 |
0.1861 |
0.0016 |
c-f-200 |
0.1914 |
0.0902 |
0.1058 |
0.1860 |
0.0011 |
KNU-SG Team_Korea |
0.2069 |
0.1145 |
0.1261 |
0.1734 |
0.0024 |
testtext |
0.2952 |
0.2189 |
0.2026 |
0.2523 |
0.0089 |
ustb_prir1 |
0.2983 |
0.2358 |
0.2146 |
0.2598 |
0.0129 |
ustb_prir2 |
0.2901 |
0.2228 |
0.2025 |
0.2564 |
0.0121 |
ustb_prir3 |
0.3012 |
0.2316 |
0.2130 |
0.2572 |
0.0115 |
ustb_prir4 |
0.2896 |
0.2128 |
0.1988 |
0.2556 |
0.0103 |
fdu |
0.3037 |
0.1560 |
0.1612 |
0.1664 |
0.0028 |
fdu2 |
0.3311 |
0.1660 |
0.1775 |
0.1711 |
0.0049 |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
Basic QA pipline |
- |
- |
- |
- |
- |
QAauth |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth |
0.2031 |
0.4733 |
0.2488 |
0.1536 |
0.0234 |
HPI-S1 |
0.5389 |
0.2030 |
0.2722 |
0.1146 |
0.0030 |
Olelo |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
c-50 |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
testtext |
0.1974 |
0.4731 |
0.2516 |
0.1359 |
0.0126 |
ustb_prir1 |
0.2297 |
0.5723 |
0.2958 |
0.1642 |
0.0423 |
ustb_prir2 |
0.1985 |
0.5324 |
0.2604 |
0.1472 |
0.0261 |
ustb_prir3 |
0.1574 |
0.4684 |
0.2155 |
0.0980 |
0.0070 |
ustb_prir4 |
0.1941 |
0.4956 |
0.2512 |
0.1414 |
0.0177 |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
Basic QA pipline |
- |
- |
- |
- |
- |
QAauth |
0.2031 |
0.4733 |
0.2488 |
0.1536 |
0.0234 |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth |
- |
- |
- |
- |
- |
HPI-S1 |
- |
- |
- |
- |
- |
Olelo |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
c-50 |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
testtext |
- |
- |
- |
- |
- |
ustb_prir1 |
- |
- |
- |
- |
- |
ustb_prir2 |
- |
- |
- |
- |
- |
ustb_prir3 |
- |
- |
- |
- |
- |
ustb_prir4 |
- |
- |
- |
- |
- |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
Basic QA pipline |
- |
- |
- |
- |
- |
QAauth |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olelo |
0.3246 |
0.2403 |
0.2130 |
0.1901 |
0.0052 |
c-e-50 |
0.2430 |
0.2412 |
0.1852 |
0.1941 |
0.0057 |
fdu |
0.3815 |
0.4427 |
0.3176 |
0.3065 |
0.0365 |
fdu2 |
0.3855 |
0.4475 |
0.3194 |
0.3098 |
0.0532 |
HPI-S1 |
0.2476 |
0.2487 |
0.1832 |
0.1625 |
0.0082 |
c-50 |
0.2430 |
0.2412 |
0.1852 |
0.1941 |
0.0057 |
c-idf-qe-1 |
0.2430 |
0.2412 |
0.1852 |
0.1941 |
0.0057 |
c-f-200 |
0.2430 |
0.2412 |
0.1852 |
0.1941 |
0.0057 |
testtext |
0.4070 |
0.4873 |
0.3394 |
0.3500 |
0.0795 |
ustb_prir1 |
0.4170 |
0.4824 |
0.3447 |
0.3492 |
0.0801 |
ustb_prir2 |
0.4087 |
0.4855 |
0.3427 |
0.3374 |
0.0720 |
ustb_prir3 |
0.4160 |
0.4920 |
0.3452 |
0.3566 |
0.0751 |
ustb_prir4 |
0.4067 |
0.4815 |
0.3404 |
0.3415 |
0.0781 |
UNCC System 1 |
0.3301 |
0.2354 |
0.2172 |
0.1329 |
0.0022 |
KNU-SG Team_Korea |
0.2680 |
0.2745 |
0.2083 |
0.2105 |
0.0100 |
Concepts-auth |
- |
- |
- |
- |
- |
auth |
- |
- |
- |
- |
- |
fdu3 |
0.3610 |
0.4197 |
0.2906 |
0.3097 |
0.0310 |
fdu4 |
0.3710 |
0.4253 |
0.2958 |
0.3183 |
0.0380 |
fdu5 |
0.3140 |
0.2899 |
0.2418 |
0.2563 |
0.0106 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olelo |
0.1951 |
0.1515 |
0.1491 |
0.1467 |
0.0020 |
c-e-50 |
0.1668 |
0.1027 |
0.1104 |
0.1403 |
0.0022 |
fdu |
0.3520 |
0.1540 |
0.1804 |
0.1436 |
0.0047 |
fdu2 |
0.3303 |
0.1422 |
0.1639 |
0.1367 |
0.0038 |
HPI-S1 |
0.1681 |
0.1612 |
0.1441 |
0.1296 |
0.0032 |
c-50 |
0.1769 |
0.1145 |
0.1196 |
0.1494 |
0.0020 |
c-idf-qe-1 |
0.1554 |
0.1022 |
0.1068 |
0.1382 |
0.0019 |
c-f-200 |
0.1636 |
0.1082 |
0.1120 |
0.1340 |
0.0017 |
testtext |
0.2797 |
0.2085 |
0.2068 |
0.2465 |
0.0176 |
ustb_prir1 |
0.2897 |
0.2119 |
0.2116 |
0.2452 |
0.0204 |
ustb_prir2 |
0.3029 |
0.2313 |
0.2274 |
0.2640 |
0.0199 |
ustb_prir3 |
0.2980 |
0.2158 |
0.2170 |
0.2605 |
0.0218 |
ustb_prir4 |
0.3175 |
0.2457 |
0.2396 |
0.2727 |
0.0252 |
UNCC System 1 |
- |
- |
- |
- |
- |
KNU-SG Team_Korea |
0.1874 |
0.1355 |
0.1321 |
0.1411 |
0.0025 |
Concepts-auth |
- |
- |
- |
- |
- |
auth |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olelo |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
HPI-S1 |
0.4466 |
0.1697 |
0.2379 |
0.0707 |
0.0015 |
c-50 |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
testtext |
0.1937 |
0.4852 |
0.2611 |
0.1400 |
0.0138 |
ustb_prir1 |
0.2054 |
0.5207 |
0.2782 |
0.1576 |
0.0208 |
ustb_prir2 |
0.1464 |
0.3699 |
0.1981 |
0.1036 |
0.0036 |
ustb_prir3 |
0.1579 |
0.4410 |
0.2208 |
0.1028 |
0.0124 |
ustb_prir4 |
0.2052 |
0.4983 |
0.2733 |
0.1596 |
0.0203 |
UNCC System 1 |
0.2022 |
0.3894 |
0.2456 |
0.1202 |
0.0104 |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
Concepts-auth |
0.2271 |
0.4580 |
0.2716 |
0.1515 |
0.0155 |
auth |
0.1581 |
0.3832 |
0.2143 |
0.1095 |
0.0089 |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
0.1828 |
0.4494 |
0.2469 |
0.1189 |
0.0096 |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olelo |
- |
- |
- |
- |
- |
c-e-50 |
- |
- |
- |
- |
- |
fdu |
- |
- |
- |
- |
- |
fdu2 |
- |
- |
- |
- |
- |
HPI-S1 |
- |
- |
- |
- |
- |
c-50 |
- |
- |
- |
- |
- |
c-idf-qe-1 |
- |
- |
- |
- |
- |
c-f-200 |
- |
- |
- |
- |
- |
testtext |
- |
- |
- |
- |
- |
ustb_prir1 |
- |
- |
- |
- |
- |
ustb_prir2 |
- |
- |
- |
- |
- |
ustb_prir3 |
- |
- |
- |
- |
- |
ustb_prir4 |
- |
- |
- |
- |
- |
UNCC System 1 |
- |
- |
- |
- |
- |
KNU-SG Team_Korea |
- |
- |
- |
- |
- |
Concepts-auth |
- |
- |
- |
- |
- |
auth |
- |
- |
- |
- |
- |
fdu3 |
- |
- |
- |
- |
- |
fdu4 |
- |
- |
- |
- |
- |
fdu5 |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olelo |
0.3216 |
0.2339 |
0.2209 |
0.1901 |
0.0027 |
HPI-S1 |
0.2357 |
0.2182 |
0.1782 |
0.1443 |
0.0027 |
c-e-50 |
0.2540 |
0.2409 |
0.1983 |
0.2106 |
0.0057 |
c-f-200 |
0.2540 |
0.2409 |
0.1983 |
0.2106 |
0.0057 |
c-idf-qe-1 |
0.2540 |
0.2409 |
0.1983 |
0.2106 |
0.0057 |
c-50 |
0.2540 |
0.2409 |
0.1983 |
0.2106 |
0.0057 |
KNU-SG Team_Korea |
0.2640 |
0.2696 |
0.2084 |
0.2160 |
0.0064 |
testtext |
0.4372 |
0.5369 |
0.3825 |
0.3758 |
0.0667 |
ustb_prir1 |
0.4382 |
0.5166 |
0.3785 |
0.3774 |
0.0632 |
ustb_prir2 |
0.4416 |
0.5368 |
0.3886 |
0.3688 |
0.0793 |
ustb_prir3 |
0.4392 |
0.5173 |
0.3801 |