BioASQ Participants Area
Task 6b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| htersys | 0.0030 | 0.0075 | 0.0043 | 0.0030 | 0.0000 |
| aueb-nlp-1 | 0.2965 | 0.6260 | 0.3372 | 0.2265 | 0.0639 |
| aueb-nlp-2 | 0.2945 | 0.6307 | 0.3364 | 0.2275 | 0.0777 |
| aueb-nlp-3 | 0.3015 | 0.6387 | 0.3438 | 0.2327 | 0.0686 |
| sdm/rerank | 0.2500 | 0.5648 | 0.2886 | 0.1840 | 0.0277 |
| aueb-nlp-4 | 0.2965 | 0.6220 | 0.3365 | 0.2288 | 0.0574 |
| OAQA based system | 0.1385 | 0.3352 | 0.1581 | 0.1064 | 0.0014 |
| MindLab QA System | 0.0031 | 0.0031 | 0.0030 | 0.0012 | 0.0000 |
| MindLab QA System ++ | 0.0030 | 0.0031 | 0.0030 | 0.0016 | 0.0000 |
| testtext | 0.2390 | 0.4206 | 0.2618 | 0.1742 | 0.0099 |
| ustb_prir1 | 0.2390 | 0.4206 | 0.2618 | 0.1742 | 0.0099 |
| ustb_prir2 | 0.2310 | 0.4041 | 0.2519 | 0.1709 | 0.0075 |
| ustb_prir3 | 0.2380 | 0.4193 | 0.2605 | 0.1738 | 0.0115 |
| ustb_prir4 | 0.2380 | 0.4193 | 0.2605 | 0.1738 | 0.0115 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| htersys | - | - | - | - | - |
| aueb-nlp-1 | 0.2156 | 0.2496 | 0.1938 | 0.1675 | 0.0108 |
| aueb-nlp-2 | 0.2168 | 0.2599 | 0.1980 | 0.1642 | 0.0123 |
| aueb-nlp-3 | 0.2224 | 0.2593 | 0.2004 | 0.1684 | 0.0118 |
| sdm/rerank | 0.0940 | 0.0870 | 0.0722 | 0.0572 | 0.0004 |
| aueb-nlp-4 | 0.2127 | 0.2501 | 0.1924 | 0.1659 | 0.0111 |
| OAQA based system | 0.1318 | 0.1837 | 0.1322 | 0.1089 | 0.0005 |
| MindLab QA System | 0.0014 | 0.0013 | 0.0013 | 0.0004 | 0.0000 |
| MindLab QA System ++ | 0.0014 | 0.0013 | 0.0013 | 0.0004 | 0.0000 |
| testtext | 0.1664 | 0.1723 | 0.1409 | 0.1151 | 0.0022 |
| ustb_prir1 | 0.1664 | 0.1723 | 0.1409 | 0.1151 | 0.0022 |
| ustb_prir2 | 0.1572 | 0.1616 | 0.1319 | 0.1205 | 0.0019 |
| ustb_prir3 | 0.1691 | 0.1680 | 0.1393 | 0.1193 | 0.0018 |
| ustb_prir4 | 0.1634 | 0.1619 | 0.1364 | 0.1209 | 0.0015 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| htersys | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| sdm/rerank | 0.5313 | 0.4515 | 0.4360 | 0.1315 | 0.0186 |
| aueb-nlp-4 | - | - | - | - | - |
| OAQA based system | 0.0174 | 0.0428 | 0.0245 | 0.0110 | 0.0000 |
| MindLab QA System | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| testtext | 0.1674 | 0.5154 | 0.2250 | 0.1246 | 0.0087 |
| ustb_prir1 | 0.2019 | 0.5896 | 0.2688 | 0.1623 | 0.0117 |
| ustb_prir2 | 0.1609 | 0.5015 | 0.2160 | 0.1117 | 0.0094 |
| ustb_prir3 | 0.0957 | 0.3159 | 0.1405 | 0.0522 | 0.0005 |
| ustb_prir4 | 0.1870 | 0.5090 | 0.2466 | 0.1360 | 0.0147 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| htersys | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| sdm/rerank | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| OAQA based system | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
Test batch 2
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| sys1 | 0.2717 | 0.5568 | 0.3017 | 0.1897 | 0.0339 |
| sys2 | 0.2687 | 0.5480 | 0.2974 | 0.1882 | 0.0312 |
| sys3 | 0.2820 | 0.5945 | 0.3117 | 0.1940 | 0.0408 |
| aueb-nlp-1 | 0.3060 | 0.6294 | 0.3332 | 0.2319 | 0.0560 |
| aueb-nlp-2 | 0.3210 | 0.6420 | 0.3475 | 0.2470 | 0.0701 |
| aueb-nlp-3 | 0.3160 | 0.6365 | 0.3423 | 0.2416 | 0.0646 |
| aueb-nlp-4 | 0.3220 | 0.6431 | 0.3479 | 0.2500 | 0.0660 |
| aueb-nlp-5 | 0.5091 | 0.5530 | 0.4493 | 0.2070 | 0.0508 |
| sys4 | 0.2490 | 0.5663 | 0.2818 | 0.1885 | 0.0364 |
| sys5 | 0.2650 | 0.5938 | 0.2956 | 0.1960 | 0.0414 |
| OAQA based system | 0.1563 | 0.3281 | 0.1647 | 0.1103 | 0.0029 |
| sdm/rerank | 0.2670 | 0.5717 | 0.2964 | 0.1891 | 0.0374 |
| MindLab QA System | 0.2310 | 0.4996 | 0.2503 | 0.1665 | 0.0259 |
| testtext | 0.3201 | 0.6355 | 0.3464 | 0.2467 | 0.0634 |
| Deep ML methods for | 0.1660 | 0.2493 | 0.1642 | 0.1220 | 0.0019 |
| ustb_prir1 | 0.3201 | 0.6355 | 0.3464 | 0.2467 | 0.0634 |
| ustb_prir2 | 0.3221 | 0.6618 | 0.3519 | 0.2458 | 0.0795 |
| MindLab QA System ++ | 0.2310 | 0.4996 | 0.2503 | 0.1665 | 0.0259 |
| ustb_prir3 | 0.3121 | 0.6379 | 0.3396 | 0.2512 | 0.0639 |
| ustb_prir4 | 0.3121 | 0.6379 | 0.3396 | 0.2512 | 0.0639 |
| MindLab QA Reloaded | 0.2611 | 0.4996 | 0.2696 | 0.1425 | 0.0186 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| aueb-nlp-1 | 0.2716 | 0.2055 | 0.1749 | 0.2226 | 0.0202 |
| aueb-nlp-2 | 0.2768 | 0.2133 | 0.1826 | 0.2256 | 0.0236 |
| aueb-nlp-3 | 0.2746 | 0.2041 | 0.1749 | 0.2272 | 0.0210 |
| aueb-nlp-4 | 0.2873 | 0.2146 | 0.1850 | 0.2337 | 0.0231 |
| aueb-nlp-5 | 0.3852 | 0.2976 | 0.2653 | 0.3187 | 0.0352 |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| OAQA based system | 0.2016 | 0.1423 | 0.1387 | 0.1704 | 0.0013 |
| sdm/rerank | 0.1973 | 0.1111 | 0.1040 | 0.1571 | 0.0024 |
| MindLab QA System | 0.2888 | 0.2298 | 0.1986 | 0.2695 | 0.0071 |
| testtext | 0.2191 | 0.3904 | 0.2060 | 0.1463 | 0.0083 |
| Deep ML methods for | 0.1684 | 0.1054 | 0.0934 | 0.1488 | 0.0008 |
| ustb_prir1 | 0.2191 | 0.3904 | 0.2060 | 0.1463 | 0.0083 |
| ustb_prir2 | 0.2213 | 0.6154 | 0.2581 | 0.1727 | 0.0217 |
| MindLab QA System ++ | 0.2878 | 0.2307 | 0.1985 | 0.2736 | 0.0065 |
| ustb_prir3 | 0.2123 | 0.3819 | 0.2041 | 0.1421 | 0.0082 |
| ustb_prir4 | 0.2179 | 0.6188 | 0.2566 | 0.1731 | 0.0205 |
| MindLab QA Reloaded | 0.2888 | 0.2298 | 0.1986 | 0.2695 | 0.0071 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| OAQA based system | 0.0108 | 0.0288 | 0.0155 | 0.0108 | 0.0000 |
| sdm/rerank | 0.5566 | 0.4357 | 0.4445 | 0.0852 | 0.0132 |
| MindLab QA System | - | - | - | - | - |
| testtext | 0.1816 | 0.5692 | 0.2471 | 0.1317 | 0.0170 |
| Deep ML methods for | - | - | - | - | - |
| ustb_prir1 | 0.1816 | 0.5984 | 0.2493 | 0.1426 | 0.0221 |
| ustb_prir2 | 0.1526 | 0.5365 | 0.2163 | 0.1065 | 0.0228 |
| MindLab QA System ++ | - | - | - | - | - |
| ustb_prir3 | 0.1368 | 0.4796 | 0.1861 | 0.0960 | 0.0098 |
| ustb_prir4 | 0.1737 | 0.5587 | 0.2344 | 0.1366 | 0.0172 |
| MindLab QA Reloaded | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| OAQA based system | - | - | - | - | - |
| sdm/rerank | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| testtext | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
Test batch 3
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | 0.3437 | 0.5940 | 0.3432 | 0.2406 | 0.0824 |
| aueb-nlp-2 | 0.3557 | 0.5931 | 0.3511 | 0.2549 | 0.0796 |
| aueb-nlp-3 | 0.3477 | 0.5975 | 0.3462 | 0.2477 | 0.0844 |
| aueb-nlp-4 | 0.3577 | 0.6035 | 0.3545 | 0.2556 | 0.0811 |
| aueb-nlp-5 | 0.5926 | 0.5169 | 0.5000 | 0.2189 | 0.0563 |
| sys1 | 0.3160 | 0.5530 | 0.3207 | 0.2018 | 0.0555 |
| sys3 | 0.3187 | 0.5612 | 0.3229 | 0.2068 | 0.0578 |
| sys4 | 0.2660 | 0.5523 | 0.2742 | 0.2013 | 0.0558 |
| sys5 | 0.2800 | 0.5628 | 0.2862 | 0.2080 | 0.0579 |
| sdm/rerank | 0.3280 | 0.5951 | 0.3353 | 0.2193 | 0.0920 |
| MindLab QA System ++ | 0.2450 | 0.5076 | 0.2503 | 0.1818 | 0.0391 |
| MindLab QA Reloaded | 0.2750 | 0.5485 | 0.2794 | 0.2091 | 0.0573 |
| testtext | 0.3637 | 0.6217 | 0.3655 | 0.2597 | 0.0991 |
| ustb_prir1 | 0.3250 | 0.6140 | 0.3314 | 0.2500 | 0.0923 |
| ustb_prir2 | 0.3340 | 0.6340 | 0.3396 | 0.2622 | 0.1057 |
| ustb_prir3 | 0.3657 | 0.6244 | 0.3680 | 0.2610 | 0.0996 |
| ustb_prir4 | 0.3160 | 0.6203 | 0.3251 | 0.2506 | 0.0965 |
| MindLab QA System | 0.2750 | 0.5485 | 0.2794 | 0.2091 | 0.0573 |
| Deep ML methods for | 0.1960 | 0.3469 | 0.1915 | 0.1165 | 0.0123 |
| MindLab Red Lions++ | 0.1840 | 0.3093 | 0.1772 | 0.1065 | 0.0080 |
| sys2 | 0.2660 | 0.5523 | 0.2742 | 0.2011 | 0.0557 |
| OAQA based system | 0.1530 | 0.2964 | 0.1470 | 0.1147 | 0.0027 |
| YODAQA based system | 0.1981 | 0.1304 | 0.1425 | 0.0494 | 0.0002 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | 0.2588 | 0.2612 | 0.2304 | 0.2278 | 0.0415 |
| aueb-nlp-2 | 0.2532 | 0.2486 | 0.2211 | 0.2197 | 0.0319 |
| aueb-nlp-3 | 0.2515 | 0.2525 | 0.2214 | 0.2221 | 0.0337 |
| aueb-nlp-4 | 0.2600 | 0.2607 | 0.2298 | 0.2306 | 0.0359 |
| aueb-nlp-5 | 0.3807 | 0.3655 | 0.3452 | 0.3320 | 0.0536 |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| sdm/rerank | 0.1865 | 0.1552 | 0.1510 | 0.1533 | 0.0063 |
| MindLab QA System ++ | 0.1935 | 0.2414 | 0.1976 | 0.1787 | 0.0057 |
| MindLab QA Reloaded | 0.2374 | 0.2832 | 0.2377 | 0.2217 | 0.0134 |
| testtext | 0.2951 | 0.2600 | 0.2452 | 0.2021 | 0.0246 |
| ustb_prir1 | 0.2894 | 0.2557 | 0.2413 | 0.1977 | 0.0227 |
| ustb_prir2 | 0.2727 | 0.2331 | 0.2236 | 0.1953 | 0.0274 |
| ustb_prir3 | 0.2951 | 0.2600 | 0.2452 | 0.2021 | 0.0246 |
| ustb_prir4 | 0.2738 | 0.2346 | 0.2253 | 0.1967 | 0.0277 |
| MindLab QA System | 0.2491 | 0.3094 | 0.2504 | 0.2113 | 0.0218 |
| Deep ML methods for | 0.2320 | 0.2814 | 0.2297 | 0.1889 | 0.0197 |
| MindLab Red Lions++ | 0.2191 | 0.2592 | 0.2164 | 0.1991 | 0.0127 |
| sys2 | - | - | - | - | - |
| OAQA based system | 0.1892 | 0.2137 | 0.1807 | 0.1730 | 0.0025 |
| YODAQA based system | - | - | - | - | - |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| sdm/rerank | 0.5340 | 0.4936 | 0.4605 | 0.0904 | 0.0111 |
| MindLab QA System ++ | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| testtext | 0.1689 | 0.4794 | 0.2081 | 0.0966 | 0.0036 |
| ustb_prir1 | 0.1746 | 0.5991 | 0.2421 | 0.1157 | 0.0121 |
| ustb_prir2 | 0.1576 | 0.5597 | 0.2276 | 0.0931 | 0.0094 |
| ustb_prir3 | 0.1000 | 0.4588 | 0.1559 | 0.0655 | 0.0032 |
| ustb_prir4 | 0.1661 | 0.6057 | 0.2405 | 0.1234 | 0.0126 |
| MindLab QA System | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| OAQA based system | 0.0068 | 0.0240 | 0.0105 | 0.0042 | 0.0000 |
| YODAQA based system | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| sdm/rerank | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| OAQA based system | - | - | - | - | - |
| YODAQA based system | 0.7500 | 1.0000 | 0.8333 | 0.6972 | 0.6281 |
Test batch 4
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | 0.2436 | 0.5968 | 0.2749 | 0.1772 | 0.0225 |
| aueb-nlp-2 | 0.2456 | 0.5886 | 0.2760 | 0.1822 | 0.0207 |
| aueb-nlp-3 | 0.2466 | 0.5930 | 0.2779 | 0.1804 | 0.0218 |
| aueb-nlp-4 | 0.2446 | 0.5877 | 0.2751 | 0.1843 | 0.0208 |
| aueb-nlp-5 | 0.4245 | 0.5180 | 0.4052 | 0.1615 | 0.0150 |
| sys1 | 0.2110 | 0.5846 | 0.2489 | 0.1501 | 0.0181 |
| sys3 | 0.2220 | 0.5810 | 0.2567 | 0.1529 | 0.0189 |
| sys4 | 0.2110 | 0.5842 | 0.2466 | 0.1532 | 0.0188 |
| MindLab Red Lions++ | 0.1880 | 0.4643 | 0.2136 | 0.1418 | 0.0066 |
| MindLab QA System ++ | 0.1880 | 0.4643 | 0.2136 | 0.1418 | 0.0066 |
| sdm/rerank | 0.2049 | 0.5788 | 0.2427 | 0.1407 | 0.0200 |
| YODAQA based system | 0.1603 | 0.1915 | 0.1469 | 0.0523 | 0.0003 |
| sys2 | 0.2210 | 0.5922 | 0.2597 | 0.1480 | 0.0193 |
| auth-qa-1 | 0.1609 | 0.2187 | 0.1549 | 0.0560 | 0.0003 |
| sys5 | 0.2220 | 0.5840 | 0.2598 | 0.1491 | 0.0180 |
| testtext | 0.2449 | 0.6327 | 0.2815 | 0.1650 | 0.0281 |
| ustb_prir1 | 0.2330 | 0.6244 | 0.2701 | 0.1665 | 0.0261 |
| ustb_prir2 | 0.2450 | 0.6378 | 0.2827 | 0.1649 | 0.0275 |
| ustb_prir3 | 0.2459 | 0.6328 | 0.2821 | 0.1654 | 0.0281 |
| ustb_prir4 | 0.2290 | 0.6015 | 0.2640 | 0.1638 | 0.0205 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | 0.1798 | 0.2377 | 0.1656 | 0.1444 | 0.0035 |
| aueb-nlp-2 | 0.1822 | 0.2566 | 0.1742 | 0.1555 | 0.0038 |
| aueb-nlp-3 | 0.1836 | 0.2513 | 0.1736 | 0.1480 | 0.0035 |
| aueb-nlp-4 | 0.1761 | 0.2367 | 0.1646 | 0.1479 | 0.0033 |
| aueb-nlp-5 | 0.2403 | 0.3229 | 0.2368 | 0.2138 | 0.0061 |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| MindLab Red Lions++ | 0.1316 | 0.1877 | 0.1343 | 0.1151 | 0.0013 |
| MindLab QA System ++ | 0.1480 | 0.2342 | 0.1579 | 0.1413 | 0.0015 |
| sdm/rerank | 0.0910 | 0.0919 | 0.0772 | 0.0532 | 0.0004 |
| YODAQA based system | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| testtext | 0.1748 | 0.3027 | 0.1738 | 0.1213 | 0.0070 |
| ustb_prir1 | 0.1752 | 0.3028 | 0.1753 | 0.1216 | 0.0077 |
| ustb_prir2 | 0.1694 | 0.2969 | 0.1696 | 0.1204 | 0.0059 |
| ustb_prir3 | 0.1749 | 0.3027 | 0.1739 | 0.1213 | 0.0070 |
| ustb_prir4 | 0.1674 | 0.2953 | 0.1669 | 0.1213 | 0.0061 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| sdm/rerank | 0.5651 | 0.4807 | 0.4910 | 0.0951 | 0.0107 |
| YODAQA based system | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| testtext | 0.1973 | 0.6313 | 0.2684 | 0.1250 | 0.0112 |
| ustb_prir1 | 0.2108 | 0.7599 | 0.2999 | 0.1419 | 0.0302 |
| ustb_prir2 | 0.1500 | 0.6622 | 0.2292 | 0.0876 | 0.0202 |
| ustb_prir3 | 0.1324 | 0.6608 | 0.2074 | 0.0884 | 0.0128 |
| ustb_prir4 | 0.1973 | 0.7378 | 0.2861 | 0.1342 | 0.0305 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| sdm/rerank | - | - | - | - | - |
| YODAQA based system | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| auth-qa-1 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |
Test batch 5
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olelo system at HPI | 0.1028 | 0.1104 | 0.0858 | 0.0464 | 0.0002 |
| aueb-nlp-1 | 0.2195 | 0.4197 | 0.2400 | 0.1403 | 0.0095 |
| aueb-nlp-2 | 0.2215 | 0.4356 | 0.2435 | 0.1416 | 0.0128 |
| aueb-nlp-3 | 0.2175 | 0.4200 | 0.2373 | 0.1422 | 0.0100 |
| aueb-nlp-4 | 0.2265 | 0.4424 | 0.2487 | 0.1464 | 0.0122 |
| aueb-nlp-5 | 0.2702 | 0.3591 | 0.2552 | 0.1250 | 0.0077 |
| sdm/rerank | 0.1975 | 0.4002 | 0.2230 | 0.1328 | 0.0074 |
| MindLab QA Reloaded | 0.1750 | 0.3345 | 0.1944 | 0.1100 | 0.0051 |
| MindLab QA System | 0.1750 | 0.3345 | 0.1944 | 0.1100 | 0.0051 |
| sys1 | 0.1830 | 0.3701 | 0.2043 | 0.1176 | 0.0042 |
| sys2 | 0.1825 | 0.3585 | 0.2034 | 0.1174 | 0.0040 |
| sys3 | 0.1805 | 0.3547 | 0.2009 | 0.1170 | 0.0039 |
| MindLab Red Lions++ | 0.1750 | 0.3345 | 0.1944 | 0.1100 | 0.0051 |
| sys4 | 0.1820 | 0.3539 | 0.2036 | 0.1142 | 0.0039 |
| sys5 | 0.1800 | 0.3520 | 0.2001 | 0.1169 | 0.0035 |
| MindLab QA System ++ | 0.1750 | 0.3345 | 0.1944 | 0.1100 | 0.0051 |
| Deep ML methods for | 0.1750 | 0.3345 | 0.1944 | 0.1100 | 0.0051 |
| testtext | 0.2095 | 0.4070 | 0.2330 | 0.1248 | 0.0089 |
| ustb_prir1 | 0.2090 | 0.3994 | 0.2312 | 0.1268 | 0.0072 |
| ustb_prir2 | 0.2125 | 0.4073 | 0.2375 | 0.1278 | 0.0093 |
| ustb_prir3 | 0.2065 | 0.4029 | 0.2303 | 0.1230 | 0.0083 |
| ustb_prir4 | 0.2035 | 0.3913 | 0.2266 | 0.1224 | 0.0071 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olelo system at HPI | 0.0788 | 0.0595 | 0.0556 | 0.0351 | 0.0001 |
| aueb-nlp-1 | 0.1362 | 0.1755 | 0.1224 | 0.0927 | 0.0024 |
| aueb-nlp-2 | 0.1364 | 0.1781 | 0.1238 | 0.0967 | 0.0028 |
| aueb-nlp-3 | 0.1303 | 0.1706 | 0.1175 | 0.0929 | 0.0021 |
| aueb-nlp-4 | 0.1409 | 0.1849 | 0.1285 | 0.1007 | 0.0024 |
| aueb-nlp-5 | 0.1360 | 0.1669 | 0.1248 | 0.1147 | 0.0020 |
| sdm/rerank | 0.0701 | 0.0561 | 0.0529 | 0.0441 | 0.0002 |
| MindLab QA Reloaded | 0.0908 | 0.1411 | 0.0911 | 0.1000 | 0.0005 |
| MindLab QA System | 0.0923 | 0.1444 | 0.0926 | 0.1006 | 0.0005 |
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| MindLab Red Lions++ | 0.0876 | 0.1446 | 0.0910 | 0.0874 | 0.0005 |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| MindLab QA System ++ | 0.0872 | 0.1515 | 0.0919 | 0.0797 | 0.0006 |
| Deep ML methods for | 0.0813 | 0.1479 | 0.0858 | 0.0730 | 0.0005 |
| testtext | 0.1256 | 0.1485 | 0.1112 | 0.0861 | 0.0008 |
| ustb_prir1 | 0.1264 | 0.1424 | 0.1100 | 0.0865 | 0.0010 |
| ustb_prir2 | 0.1190 | 0.1473 | 0.1091 | 0.0798 | 0.0009 |
| ustb_prir3 | 0.1237 | 0.1495 | 0.1108 | 0.0845 | 0.0009 |
| ustb_prir4 | 0.1185 | 0.1498 | 0.1063 | 0.0814 | 0.0008 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olelo system at HPI | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sdm/rerank | 0.5875 | 0.3895 | 0.4051 | 0.0935 | 0.0150 |
| MindLab QA Reloaded | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| testtext | 0.1931 | 0.4965 | 0.2527 | 0.1467 | 0.0128 |
| ustb_prir1 | 0.2172 | 0.5942 | 0.2908 | 0.1796 | 0.0318 |
| ustb_prir2 | 0.1793 | 0.5483 | 0.2505 | 0.1311 | 0.0249 |
| ustb_prir3 | 0.2034 | 0.5661 | 0.2709 | 0.1417 | 0.0201 |
| ustb_prir4 | 0.2241 | 0.6591 | 0.3030 | 0.1734 | 0.0461 |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olelo system at HPI | - | - | - | - | - |
| aueb-nlp-1 | - | - | - | - | - |
| aueb-nlp-2 | - | - | - | - | - |
| aueb-nlp-3 | - | - | - | - | - |
| aueb-nlp-4 | - | - | - | - | - |
| aueb-nlp-5 | - | - | - | - | - |
| sdm/rerank | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| sys1 | - | - | - | - | - |
| sys2 | - | - | - | - | - |
| sys3 | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| sys4 | - | - | - | - | - |
| sys5 | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| testtext | - | - | - | - | - |
| ustb_prir1 | - | - | - | - | - |
| ustb_prir2 | - | - | - | - | - |
| ustb_prir3 | - | - | - | - | - |
| ustb_prir4 | - | - | - | - | - |