BioASQ Participants Area
Task 4b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in seperate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.1960 | 0.2647 | 0.1875 | 0.1435 | 0.0078 |
| ustb_prir1 | 0.3040 | 0.4795 | 0.3091 | 0.2113 | 0.0388 |
| ustb_prir2 | 0.3050 | 0.5012 | 0.3108 | 0.2269 | 0.0527 |
| ustb_prir3 | 0.3090 | 0.4884 | 0.3145 | 0.2139 | 0.0456 |
| HPI-S2 | 0.0150 | 0.0146 | 0.0137 | 0.0076 | 0.0000 |
| ustb_prir4 | 0.3110 | 0.4956 | 0.3153 | 0.2267 | 0.0477 |
| auth | - | - | - | - | - |
| testtext | 0.3180 | 0.5018 | 0.3225 | 0.2279 | 0.0482 |
| fdu | 0.3040 | 0.4955 | 0.3095 | 0.2155 | 0.0470 |
| fdu2 | 0.2920 | 0.4851 | 0.2981 | 0.2102 | 0.0404 |
| fdu3 | 0.3000 | 0.4904 | 0.3042 | 0.2085 | 0.0401 |
| fdu4 | 0.2880 | 0.4794 | 0.2941 | 0.2039 | 0.0383 |
| fdu5 | 0.2800 | 0.4624 | 0.2849 | 0.1939 | 0.0395 |
| HPI-S1 | 0.2579 | 0.3380 | 0.2357 | 0.1511 | 0.0088 |
| Auth001 | 0.1310 | 0.1712 | 0.1234 | 0.0837 | 0.0011 |
| WS4A | 0.0242 | 0.0187 | 0.0191 | 0.0103 | 0.0000 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.2001 | 0.1875 | 0.1711 | 0.1365 | 0.0049 |
| ustb_prir1 | 0.1239 | 0.1570 | 0.1192 | 0.0687 | 0.0046 |
| ustb_prir2 | 0.1322 | 0.1640 | 0.1227 | 0.0700 | 0.0042 |
| ustb_prir3 | 0.1323 | 0.1602 | 0.1226 | 0.0696 | 0.0037 |
| HPI-S2 | - | - | - | - | - |
| ustb_prir4 | 0.1175 | 0.1513 | 0.1101 | 0.0633 | 0.0027 |
| auth | - | - | - | - | - |
| testtext | 0.1244 | 0.1573 | 0.1152 | 0.0641 | 0.0035 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| HPI-S1 | 0.2084 | 0.2089 | 0.1783 | 0.1601 | 0.0065 |
| Auth001 | - | - | - | - | - |
| WS4A | 0.0098 | 0.0041 | 0.0054 | 0.0039 | 0.0000 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| ustb_prir1 | 0.1588 | 0.4576 | 0.2160 | 0.1094 | 0.0130 |
| ustb_prir2 | 0.1566 | 0.4052 | 0.2011 | 0.1086 | 0.0076 |
| ustb_prir3 | 0.0159 | 0.0418 | 0.0209 | 0.0122 | 0.0000 |
| HPI-S2 | - | - | - | - | - |
| ustb_prir4 | 0.1162 | 0.3548 | 0.1605 | 0.0791 | 0.0032 |
| auth | 0.3307 | 0.5251 | 0.3691 | 0.1433 | 0.0170 |
| testtext | 0.1293 | 0.3762 | 0.1739 | 0.0761 | 0.0029 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| HPI-S1 | 0.4296 | 0.2481 | 0.2818 | 0.0860 | 0.0021 |
| Auth001 | - | - | - | - | - |
| WS4A | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| auth | - | - | - | - | - |
| testtext | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| HPI-S1 | - | - | - | - | - |
| Auth001 | - | - | - | - | - |
| WS4A | 0.1000 | 0.4167 | 0.1540 | 0.0462 | 0.0004 |
Test batch 2
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.3005 | 0.4360 | 0.3070 | 0.2089 | 0.0506 |
| KNU-SG Team_Korea2 | 0.3005 | 0.4353 | 0.3067 | 0.2081 | 0.0461 |
| KNU-SG Team_Korea3 | 0.2270 | 0.2853 | 0.2178 | 0.1555 | 0.0134 |
| KNU-SG Team_Korea4 | 0.2260 | 0.2836 | 0.2165 | 0.1543 | 0.0123 |
| fdu | 0.3320 | 0.5025 | 0.3433 | 0.2503 | 0.0353 |
| fdu2 | 0.3290 | 0.4836 | 0.3396 | 0.2529 | 0.0291 |
| fdu3 | 0.3510 | 0.5126 | 0.3622 | 0.2578 | 0.0503 |
| fdu4 | 0.3590 | 0.5075 | 0.3672 | 0.2602 | 0.0514 |
| fdu5 | 0.3530 | 0.4955 | 0.3606 | 0.2583 | 0.0460 |
| ustb_prir2 | 0.3560 | 0.5561 | 0.3750 | 0.2725 | 0.0606 |
| ustb_prir1 | 0.3580 | 0.5427 | 0.3762 | 0.2654 | 0.0621 |
| ustb_prir3 | 0.3570 | 0.5467 | 0.3767 | 0.2676 | 0.0588 |
| KNU-SG Team_Korea5 | 0.3005 | 0.4360 | 0.3070 | 0.2089 | 0.0506 |
| ustb_prir4 | 0.3540 | 0.5525 | 0.3724 | 0.2694 | 0.0593 |
| testtext | 0.3570 | 0.5484 | 0.3774 | 0.2695 | 0.0558 |
| auth | - | - | - | - | - |
| WS4A | - | - | - | - | - |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.2451 | 0.2082 | 0.1975 | 0.1573 | 0.0189 |
| KNU-SG Team_Korea2 | 0.2496 | 0.2104 | 0.2004 | 0.1590 | 0.0172 |
| KNU-SG Team_Korea3 | 0.1764 | 0.1402 | 0.1348 | 0.1287 | 0.0054 |
| KNU-SG Team_Korea4 | 0.1767 | 0.1391 | 0.1339 | 0.1283 | 0.0049 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | 0.2484 | 0.1793 | 0.1875 | 0.1775 | 0.0138 |
| fdu4 | 0.2542 | 0.1679 | 0.1852 | 0.1870 | 0.0099 |
| fdu5 | 0.2496 | 0.1631 | 0.1779 | 0.1670 | 0.0107 |
| ustb_prir2 | 0.1777 | 0.1922 | 0.1647 | 0.0879 | 0.0035 |
| ustb_prir1 | 0.1771 | 0.1919 | 0.1637 | 0.0884 | 0.0033 |
| ustb_prir3 | 0.1712 | 0.1946 | 0.1613 | 0.0843 | 0.0033 |
| KNU-SG Team_Korea5 | 0.2451 | 0.2082 | 0.1975 | 0.1573 | 0.0189 |
| ustb_prir4 | 0.1714 | 0.1960 | 0.1622 | 0.0830 | 0.0032 |
| testtext | 0.1683 | 0.2038 | 0.1677 | 0.0774 | 0.0043 |
| auth | - | - | - | - | - |
| WS4A | 0.0032 | 0.0021 | 0.0025 | 0.0022 | 0.0000 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| ustb_prir2 | 0.1274 | 0.4968 | 0.1906 | 0.0814 | 0.0035 |
| ustb_prir1 | 0.1541 | 0.5854 | 0.2164 | 0.1124 | 0.0142 |
| ustb_prir3 | 0.1295 | 0.5084 | 0.1858 | 0.0888 | 0.0065 |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| ustb_prir4 | 0.1098 | 0.4587 | 0.1617 | 0.0655 | 0.0041 |
| testtext | 0.0869 | 0.3609 | 0.1262 | 0.0514 | 0.0011 |
| auth | 0.1115 | 0.4417 | 0.1589 | 0.0814 | 0.0037 |
| WS4A | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| auth | - | - | - | - | - |
| WS4A | 0.2875 | 0.6250 | 0.3654 | 0.2298 | 0.0059 |
Test batch 3
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth | - | - | - | - | - |
| HPI-S1 | 0.3155 | 0.3453 | 0.2880 | 0.1531 | 0.0105 |
| HPI-S2 | 0.0060 | 0.0054 | 0.0053 | 0.0029 | 0.0000 |
| KNU-SG Team_Korea | 0.2640 | 0.4149 | 0.2664 | 0.2019 | 0.0340 |
| KNU-SG Team_Korea2 | 0.2640 | 0.4149 | 0.2664 | 0.2019 | 0.0340 |
| KNU-SG Team_Korea3 | 0.2630 | 0.4143 | 0.2657 | 0.2007 | 0.0336 |
| KNU-SG Team_Korea4 | 0.2630 | 0.4143 | 0.2657 | 0.2007 | 0.0336 |
| KNU-SG Team_Korea5 | 0.2747 | 0.4076 | 0.2717 | 0.2020 | 0.0311 |
| WS4A | - | - | - | - | - |
| fdu | 0.3254 | 0.5012 | 0.3344 | 0.2311 | 0.0766 |
| ustb_prir1 | 0.3410 | 0.5114 | 0.3461 | 0.2488 | 0.0683 |
| ustb_prir2 | 0.3230 | 0.4944 | 0.3269 | 0.2380 | 0.0596 |
| ustb_prir3 | 0.3440 | 0.5159 | 0.3493 | 0.2508 | 0.0691 |
| ustb_prir4 | 0.3450 | 0.5193 | 0.3483 | 0.2553 | 0.0706 |
| testtext | 0.3420 | 0.5162 | 0.3456 | 0.2512 | 0.0688 |
| fdu2 | 0.3394 | 0.5073 | 0.3440 | 0.2526 | 0.0704 |
| fdu3 | 0.0860 | 0.1553 | 0.0885 | 0.0490 | 0.0004 |
| fdu4 | 0.3844 | 0.5019 | 0.3716 | 0.3025 | 0.0435 |
| fdu5 | 0.3514 | 0.4656 | 0.3417 | 0.2658 | 0.0377 |
| oaqa-3b-3 | 0.2370 | 0.3638 | 0.2360 | 0.1798 | 0.0127 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth | - | - | - | - | - |
| HPI-S1 | 0.2359 | 0.2145 | 0.1918 | 0.1696 | 0.0058 |
| HPI-S2 | - | - | - | - | - |
| KNU-SG Team_Korea | 0.2048 | 0.1489 | 0.1507 | 0.1693 | 0.0075 |
| KNU-SG Team_Korea2 | 0.2206 | 0.1965 | 0.1765 | 0.1684 | 0.0086 |
| KNU-SG Team_Korea3 | 0.2047 | 0.1488 | 0.1507 | 0.1681 | 0.0075 |
| KNU-SG Team_Korea4 | 0.2192 | 0.1953 | 0.1751 | 0.1669 | 0.0085 |
| KNU-SG Team_Korea5 | 0.2016 | 0.1409 | 0.1442 | 0.1678 | 0.0084 |
| WS4A | - | - | - | - | - |
| fdu | - | - | - | - | - |
| ustb_prir1 | 0.2092 | 0.2379 | 0.1956 | 0.1127 | 0.0046 |
| ustb_prir2 | 0.2020 | 0.2308 | 0.1899 | 0.1002 | 0.0044 |
| ustb_prir3 | 0.2073 | 0.2376 | 0.1946 | 0.1064 | 0.0050 |
| ustb_prir4 | 0.1986 | 0.2309 | 0.1876 | 0.0999 | 0.0047 |
| testtext | 0.2029 | 0.2336 | 0.1905 | 0.1069 | 0.0052 |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | 0.2884 | 0.1659 | 0.1892 | 0.2105 | 0.0105 |
| fdu5 | 0.2825 | 0.1699 | 0.1926 | 0.2214 | 0.0118 |
| oaqa-3b-3 | 0.0326 | 0.0228 | 0.0255 | 0.0197 | 0.0000 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth | 0.1879 | 0.5714 | 0.2513 | 0.1361 | 0.0255 |
| HPI-S1 | 0.3343 | 0.2083 | 0.2277 | 0.0863 | 0.0008 |
| HPI-S2 | - | - | - | - | - |
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| WS4A | - | - | - | - | - |
| fdu | - | - | - | - | - |
| ustb_prir1 | 0.1879 | 0.5594 | 0.2511 | 0.1386 | 0.0214 |
| ustb_prir2 | 0.1533 | 0.4863 | 0.2072 | 0.1173 | 0.0092 |
| ustb_prir3 | 0.1690 | 0.5149 | 0.2234 | 0.1117 | 0.0156 |
| ustb_prir4 | 0.1534 | 0.4445 | 0.2010 | 0.1029 | 0.0057 |
| testtext | 0.1172 | 0.3466 | 0.1518 | 0.0733 | 0.0023 |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | 0.2128 | 0.5988 | 0.2805 | 0.1566 | 0.0268 |
| oaqa-3b-3 | 0.1542 | 0.4834 | 0.2096 | 0.1067 | 0.0135 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| auth | - | - | - | - | - |
| HPI-S1 | - | - | - | - | - |
| HPI-S2 | - | - | - | - | - |
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| WS4A | 0.2810 | 1.0000 | 0.4031 | 0.1441 | 0.0628 |
| fdu | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| oaqa-3b-3 | - | - | - | - | - |
Test batch 4
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu | 0.3441 | 0.6114 | 0.3734 | 0.2521 | 0.0637 |
| auth | - | - | - | - | - |
| KNU-SG Team_Korea | 0.2730 | 0.5397 | 0.2991 | 0.2203 | 0.0617 |
| KNU-SG Team_Korea2 | 0.1860 | 0.3917 | 0.2122 | 0.1227 | 0.0093 |
| KNU-SG Team_Korea3 | 0.2440 | 0.4755 | 0.2662 | 0.1793 | 0.0337 |
| KNU-SG Team_Korea4 | 0.2710 | 0.5364 | 0.2966 | 0.2197 | 0.0606 |
| KNU-SG Team_Korea5 | 0.2730 | 0.5397 | 0.2991 | 0.2203 | 0.0617 |
| ustb_prir1 | 0.3335 | 0.5869 | 0.3613 | 0.2447 | 0.0595 |
| ustb_prir2 | 0.3515 | 0.6110 | 0.3807 | 0.2590 | 0.0758 |
| ustb_prir3 | 0.3340 | 0.6140 | 0.3637 | 0.2623 | 0.0765 |
| ustb_prir4 | 0.3435 | 0.6084 | 0.3721 | 0.2547 | 0.0743 |
| testtext | 0.3290 | 0.6071 | 0.3581 | 0.2564 | 0.0697 |
| fdu2 | 0.3401 | 0.6118 | 0.3696 | 0.2539 | 0.0639 |
| fdu3 | 0.3537 | 0.5451 | 0.3669 | 0.2576 | 0.0410 |
| fdu4 | 0.3437 | 0.5249 | 0.3537 | 0.2585 | 0.0279 |
| oaqa-3b-4 | 0.1800 | 0.3058 | 0.1938 | 0.1197 | 0.0033 |
| HPI-S1 | - | - | - | - | - |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu | - | - | - | - | - |
| auth | - | - | - | - | - |
| KNU-SG Team_Korea | 0.2598 | 0.1876 | 0.1941 | 0.2268 | 0.0257 |
| KNU-SG Team_Korea2 | 0.2605 | 0.1875 | 0.1946 | 0.2264 | 0.0257 |
| KNU-SG Team_Korea3 | 0.2612 | 0.1897 | 0.1958 | 0.2271 | 0.0276 |
| KNU-SG Team_Korea4 | 0.2619 | 0.1895 | 0.1962 | 0.2271 | 0.0276 |
| KNU-SG Team_Korea5 | 0.2739 | 0.2332 | 0.2210 | 0.2305 | 0.0263 |
| ustb_prir1 | 0.3148 | 0.2292 | 0.2347 | 0.2298 | 0.0348 |
| ustb_prir2 | 0.3164 | 0.2323 | 0.2366 | 0.2330 | 0.0356 |
| ustb_prir3 | 0.3005 | 0.2233 | 0.2265 | 0.2201 | 0.0337 |
| ustb_prir4 | 0.2655 | 0.2029 | 0.2037 | 0.1830 | 0.0147 |
| testtext | 0.3064 | 0.2607 | 0.2486 | 0.1834 | 0.0337 |
| fdu2 | - | - | - | - | - |
| fdu3 | 0.3020 | 0.1845 | 0.2038 | 0.2365 | 0.0168 |
| fdu4 | 0.2728 | 0.1589 | 0.1774 | 0.2183 | 0.0112 |
| oaqa-3b-4 | 0.2114 | 0.1615 | 0.1641 | 0.1813 | 0.0024 |
| HPI-S1 | - | - | - | - | - |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu | - | - | - | - | - |
| auth | 0.1678 | 0.5777 | 0.2424 | 0.1376 | 0.0260 |
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| ustb_prir1 | 0.1515 | 0.5557 | 0.2190 | 0.1174 | 0.0126 |
| ustb_prir2 | 0.1344 | 0.4927 | 0.1929 | 0.0957 | 0.0061 |
| ustb_prir3 | 0.1230 | 0.4734 | 0.1801 | 0.0741 | 0.0049 |
| ustb_prir4 | 0.1066 | 0.4442 | 0.1592 | 0.0701 | 0.0047 |
| testtext | 0.0951 | 0.3691 | 0.1393 | 0.0592 | 0.0022 |
| fdu2 | - | - | - | - | - |
| fdu3 | 0.1700 | 0.6104 | 0.2429 | 0.1319 | 0.0203 |
| fdu4 | - | - | - | - | - |
| oaqa-3b-4 | 0.1781 | 0.5994 | 0.2492 | 0.1332 | 0.0317 |
| HPI-S1 | 0.4155 | 0.2189 | 0.2616 | 0.0721 | 0.0009 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| fdu | - | - | - | - | - |
| auth | - | - | - | - | - |
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| oaqa-3b-4 | - | - | - | - | - |
| HPI-S1 | - | - | - | - | - |
Test batch 5
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.3546 | 0.3966 | 0.3133 | 0.2731 | 0.0700 |
| KNU-SG Team_Korea2 | 0.3567 | 0.3979 | 0.3147 | 0.2775 | 0.0714 |
| KNU-SG Team_Korea3 | 0.3526 | 0.3957 | 0.3121 | 0.2715 | 0.0689 |
| KNU-SG Team_Korea4 | 0.3557 | 0.3976 | 0.3142 | 0.2764 | 0.0705 |
| KNU-SG Team_Korea5 | 0.3206 | 0.3613 | 0.2814 | 0.2372 | 0.0421 |
| auth | - | - | - | - | - |
| The monkey system! | 0.3289 | 0.3998 | 0.3075 | 0.2490 | 0.0328 |
| Monkey with and | 0.3275 | 0.1770 | 0.2033 | 0.1435 | 0.0019 |
| WS4A | 0.0674 | 0.0296 | 0.0304 | 0.0170 | 0.0000 |
| oaqa-3b-3 | 0.1567 | 0.1911 | 0.1370 | 0.1173 | 0.0022 |
| oaqa-3b-3-e | 0.1577 | 0.1917 | 0.1377 | 0.1176 | 0.0022 |
| fdu | 0.3753 | 0.4480 | 0.3481 | 0.2797 | 0.0921 |
| fdu2 | 0.3660 | 0.4486 | 0.3443 | 0.2672 | 0.0955 |
| HPI-S1 | 0.3489 | 0.2882 | 0.2613 | 0.1764 | 0.0208 |
| fdu3 | 0.4093 | 0.4451 | 0.3634 | 0.3274 | 0.0762 |
| fdu4 | 0.4052 | 0.4530 | 0.3679 | 0.3245 | 0.0787 |
| fdu5 | 0.0990 | 0.1208 | 0.0894 | 0.0487 | 0.0007 |
| ustb_prir1 | 0.3794 | 0.4546 | 0.3488 | 0.2872 | 0.0923 |
| ustb_prir2 | 0.3959 | 0.4652 | 0.3651 | 0.2982 | 0.0935 |
| ustb_prir3 | 0.3918 | 0.4585 | 0.3604 | 0.2939 | 0.0989 |
| ustb_prir4 | 0.4000 | 0.4670 | 0.3676 | 0.3066 | 0.0950 |
| testtext | 0.3928 | 0.4607 | 0.3618 | 0.2993 | 0.0928 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | 0.2801 | 0.1593 | 0.1797 | 0.2187 | 0.0352 |
| KNU-SG Team_Korea2 | 0.2812 | 0.1572 | 0.1783 | 0.2250 | 0.0297 |
| KNU-SG Team_Korea3 | 0.2804 | 0.1593 | 0.1797 | 0.2185 | 0.0347 |
| KNU-SG Team_Korea4 | 0.2771 | 0.1560 | 0.1765 | 0.2222 | 0.0291 |
| KNU-SG Team_Korea5 | 0.2268 | 0.1114 | 0.1312 | 0.1655 | 0.0150 |
| auth | - | - | - | - | - |
| The monkey system! | - | - | - | - | - |
| Monkey with and | - | - | - | - | - |
| WS4A | 0.0601 | 0.0187 | 0.0259 | 0.0144 | 0.0000 |
| oaqa-3b-3 | 0.1433 | 0.0841 | 0.0908 | 0.1096 | 0.0009 |
| oaqa-3b-3-e | 0.1283 | 0.0775 | 0.0837 | 0.1381 | 0.0005 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| HPI-S1 | 0.2701 | 0.1866 | 0.1936 | 0.2049 | 0.0182 |
| fdu3 | 0.3663 | 0.1973 | 0.2360 | 0.2880 | 0.0282 |
| fdu4 | 0.3638 | 0.2064 | 0.2381 | 0.2882 | 0.0351 |
| fdu5 | - | - | - | - | - |
| ustb_prir1 | 0.3399 | 0.1959 | 0.2195 | 0.2386 | 0.0392 |
| ustb_prir2 | 0.3400 | 0.1962 | 0.2192 | 0.2383 | 0.0424 |
| ustb_prir3 | 0.3318 | 0.1948 | 0.2169 | 0.2343 | 0.0375 |
| ustb_prir4 | 0.2870 | 0.1742 | 0.1908 | 0.1920 | 0.0192 |
| testtext | 0.3072 | 0.2361 | 0.2359 | 0.1694 | 0.0275 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| auth | 0.1441 | 0.4122 | 0.1992 | 0.1066 | 0.0045 |
| The monkey system! | - | - | - | - | - |
| Monkey with and | - | - | - | - | - |
| WS4A | - | - | - | - | - |
| oaqa-3b-3 | 0.1000 | 0.2887 | 0.1369 | 0.0915 | 0.0026 |
| oaqa-3b-3-e | 0.1118 | 0.2924 | 0.1483 | 0.1002 | 0.0021 |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| HPI-S1 | 0.3237 | 0.1430 | 0.1754 | 0.0439 | 0.0003 |
| fdu3 | - | - | - | - | - |
| fdu4 | 0.1382 | 0.4343 | 0.1937 | 0.1004 | 0.0050 |
| fdu5 | - | - | - | - | - |
| ustb_prir1 | 0.1353 | 0.4417 | 0.1921 | 0.0980 | 0.0060 |
| ustb_prir2 | 0.1324 | 0.4419 | 0.1900 | 0.1031 | 0.0069 |
| ustb_prir3 | 0.1294 | 0.4743 | 0.1879 | 0.0815 | 0.0069 |
| ustb_prir4 | 0.1206 | 0.3965 | 0.1733 | 0.0707 | 0.0042 |
| testtext | 0.1176 | 0.3956 | 0.1709 | 0.0705 | 0.0038 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| KNU-SG Team_Korea | - | - | - | - | - |
| KNU-SG Team_Korea2 | - | - | - | - | - |
| KNU-SG Team_Korea3 | - | - | - | - | - |
| KNU-SG Team_Korea4 | - | - | - | - | - |
| KNU-SG Team_Korea5 | - | - | - | - | - |
| auth | - | - | - | - | - |
| The monkey system! | - | - | - | - | - |
| Monkey with and | - | - | - | - | - |
| WS4A | 0.1800 | 1.0000 | 0.2984 | 0.0554 | 0.0393 |
| oaqa-3b-3 | - | - | - | - | - |
| oaqa-3b-3-e | - | - | - | - | - |
| fdu | - | - | - | - | - |
| fdu2 | - | - | - | - | - |
| HPI-S1 | - | - | - | - | - |
| fdu3 | - | - | - | - | - |
| fdu4 | - | - | - | - | - |
| fdu5 | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| testtext | - | - | - | - | - |