BioASQ Participants Area
Task 7b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
0.2100 |
0.2357 |
0.1845 |
0.0705 |
0.0003 |
lh_sys1 |
0.1910 |
0.5281 |
0.2237 |
0.1401 |
0.0078 |
Deep ML methods for |
0.1930 |
0.5339 |
0.2230 |
0.1569 |
0.0089 |
Ir_sys1 |
0.1770 |
0.5096 |
0.2095 |
0.1231 |
0.0067 |
Ir_sys2 |
0.1910 |
0.5443 |
0.2248 |
0.1388 |
0.0082 |
Ir_sys3 |
0.1520 |
0.3906 |
0.1717 |
0.0899 |
0.0021 |
Ir_sys4 |
0.1560 |
0.3949 |
0.1752 |
0.0926 |
0.0021 |
lh_sys2 |
0.1720 |
0.4534 |
0.1971 |
0.1300 |
0.0039 |
lh_sys3 |
0.1770 |
0.4843 |
0.2053 |
0.1224 |
0.0048 |
lh_sys4 |
0.1910 |
0.5281 |
0.2237 |
0.1425 |
0.0075 |
lh_sys5 |
0.1950 |
0.5420 |
0.2288 |
0.1434 |
0.0084 |
lalala |
0.1914 |
0.2415 |
0.1894 |
0.0777 |
0.0003 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.1529 |
0.2933 |
0.1773 |
0.1411 |
0.0029 |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
from milab |
0.1010 |
0.2157 |
0.1060 |
0.0828 |
0.0005 |
auth-qa-1 |
0.1913 |
0.2402 |
0.1760 |
0.0626 |
0.0005 |
aueb-nlp-1 |
0.2440 |
0.6292 |
0.2770 |
0.2009 |
0.0219 |
aueb-nlp-2 |
0.2480 |
0.6159 |
0.2791 |
0.2056 |
0.0203 |
aueb-nlp-3 |
0.2470 |
0.6335 |
0.2805 |
0.2007 |
0.0221 |
aueb-nlp-4 |
0.2570 |
0.6342 |
0.2888 |
0.2181 |
0.0209 |
aueb-nlp-5 |
0.4368 |
0.5861 |
0.4307 |
0.2032 |
0.0145 |
Deep ML methods for |
0.1990 |
0.5010 |
0.2227 |
0.1572 |
0.0066 |
MindLab QA System ++ |
0.0280 |
0.0884 |
0.0343 |
0.0177 |
0.0001 |
MindLab Red Lions++ |
0.0280 |
0.0884 |
0.0343 |
0.0180 |
0.0001 |
MindLab QA Reloaded |
0.0280 |
0.0884 |
0.0343 |
0.0179 |
0.0001 |
MindLab QA System |
0.0280 |
0.0884 |
0.0343 |
0.0177 |
0.0001 |
Ir_sys1 |
0.1940 |
0.4932 |
0.2247 |
0.1425 |
0.0068 |
Ir_sys2 |
0.2260 |
0.5981 |
0.2610 |
0.1521 |
0.0105 |
Ir_sys3 |
0.2260 |
0.5981 |
0.2610 |
0.1415 |
0.0093 |
Ir_sys4 |
0.2389 |
0.5862 |
0.2679 |
0.1677 |
0.0124 |
lalala |
0.2260 |
0.5981 |
0.2610 |
0.1796 |
0.0132 |
lh_sys1 |
0.2260 |
0.5981 |
0.2610 |
0.1744 |
0.0127 |
lh_sys2 |
0.2150 |
0.5740 |
0.2483 |
0.1637 |
0.0113 |
lh_sys3 |
0.2260 |
0.5981 |
0.2610 |
0.1717 |
0.0123 |
lh_sys4 |
0.2290 |
0.5927 |
0.2613 |
0.1784 |
0.0118 |
lh_sys5 |
0.2140 |
0.5490 |
0.2426 |
0.1617 |
0.0102 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
from milab |
- |
- |
- |
- |
- |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
0.2536 |
0.3785 |
0.2535 |
0.2609 |
0.0100 |
aueb-nlp-2 |
0.2448 |
0.3763 |
0.2454 |
0.2449 |
0.0110 |
aueb-nlp-3 |
0.1892 |
0.2723 |
0.1777 |
0.1663 |
0.0062 |
aueb-nlp-4 |
0.1955 |
0.2652 |
0.1805 |
0.1820 |
0.0066 |
aueb-nlp-5 |
0.2746 |
0.3414 |
0.2623 |
0.2650 |
0.0099 |
Deep ML methods for |
0.1743 |
0.2354 |
0.1752 |
0.1750 |
0.0017 |
MindLab QA System ++ |
0.0468 |
0.0529 |
0.0452 |
0.0390 |
0.0000 |
MindLab Red Lions++ |
0.0398 |
0.0361 |
0.0346 |
0.0364 |
0.0000 |
MindLab QA Reloaded |
0.0405 |
0.0362 |
0.0349 |
0.0345 |
0.0000 |
MindLab QA System |
0.0400 |
0.0486 |
0.0385 |
0.0316 |
0.0000 |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys1 |
0.1269 |
0.1708 |
0.1196 |
0.0894 |
0.0014 |
lh_sys2 |
0.1276 |
0.1712 |
0.1201 |
0.0826 |
0.0012 |
lh_sys3 |
0.1269 |
0.1708 |
0.1196 |
0.0935 |
0.0014 |
lh_sys4 |
0.1269 |
0.1708 |
0.1196 |
0.0768 |
0.0011 |
lh_sys5 |
0.1276 |
0.1712 |
0.1201 |
0.0933 |
0.0013 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
from milab |
- |
- |
- |
- |
- |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
- |
- |
- |
- |
- |
aueb-nlp-2 |
- |
- |
- |
- |
- |
aueb-nlp-3 |
- |
- |
- |
- |
- |
aueb-nlp-4 |
- |
- |
- |
- |
- |
aueb-nlp-5 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
from milab |
- |
- |
- |
- |
- |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
- |
- |
- |
- |
- |
aueb-nlp-2 |
- |
- |
- |
- |
- |
aueb-nlp-3 |
- |
- |
- |
- |
- |
aueb-nlp-4 |
- |
- |
- |
- |
- |
aueb-nlp-5 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
0.2940 |
0.3025 |
0.2522 |
0.1151 |
0.0025 |
aueb-nlp-1 |
0.3420 |
0.6452 |
0.3730 |
0.2761 |
0.0663 |
aueb-nlp-2 |
0.3610 |
0.6842 |
0.3952 |
0.2898 |
0.0841 |
aueb-nlp-3 |
0.3310 |
0.6472 |
0.3658 |
0.2566 |
0.0657 |
aueb-nlp-4 |
0.3460 |
0.6797 |
0.3820 |
0.2839 |
0.0862 |
aueb-nlp-5 |
0.5476 |
0.6465 |
0.5289 |
0.2679 |
0.0740 |
Deep ML methods for |
0.2770 |
0.5658 |
0.3054 |
0.2272 |
0.0359 |
MindLab QA System ++ |
0.2770 |
0.5658 |
0.3054 |
0.2272 |
0.0359 |
MindLab QA System |
0.2870 |
0.5778 |
0.3159 |
0.2392 |
0.0348 |
MindLab Red Lions++ |
0.2870 |
0.5778 |
0.3159 |
0.2392 |
0.0348 |
MindLab QA Reloaded |
0.2840 |
0.5709 |
0.3119 |
0.2323 |
0.0332 |
lh_sys1 |
0.2730 |
0.5634 |
0.3077 |
0.2032 |
0.0293 |
Ir_sys1 |
0.2670 |
0.5507 |
0.3021 |
0.1977 |
0.0313 |
Ir_sys2 |
0.2730 |
0.5634 |
0.3077 |
0.1952 |
0.0282 |
lh_sys3 |
0.2690 |
0.5562 |
0.3030 |
0.2001 |
0.0249 |
Ir_sys3 |
0.2470 |
0.4513 |
0.2701 |
0.1503 |
0.0093 |
lh_sys4 |
0.2760 |
0.5738 |
0.3122 |
0.2142 |
0.0319 |
lalala |
0.2210 |
0.5025 |
0.2570 |
0.1515 |
0.0134 |
lh_sys5 |
0.2730 |
0.5634 |
0.3077 |
0.2094 |
0.0309 |
lh_sys2 |
0.2730 |
0.5634 |
0.3077 |
0.2021 |
0.0298 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
0.3636 |
0.3755 |
0.3219 |
0.3657 |
0.0454 |
aueb-nlp-2 |
0.3933 |
0.4120 |
0.3522 |
0.3864 |
0.0777 |
aueb-nlp-3 |
0.2715 |
0.2661 |
0.2373 |
0.2349 |
0.0266 |
aueb-nlp-4 |
0.2853 |
0.2735 |
0.2454 |
0.2609 |
0.0335 |
aueb-nlp-5 |
0.3678 |
0.3609 |
0.3277 |
0.3481 |
0.0533 |
Deep ML methods for |
0.2049 |
0.2596 |
0.2052 |
0.1955 |
0.0083 |
MindLab QA System ++ |
0.1835 |
0.2141 |
0.1732 |
0.1538 |
0.0080 |
MindLab QA System |
0.1874 |
0.1980 |
0.1675 |
0.1690 |
0.0057 |
MindLab Red Lions++ |
0.2072 |
0.2178 |
0.1804 |
0.1820 |
0.0135 |
MindLab QA Reloaded |
0.2202 |
0.2541 |
0.2046 |
0.2330 |
0.0061 |
lh_sys1 |
0.1360 |
0.1320 |
0.1142 |
0.1234 |
0.0017 |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
0.1360 |
0.1320 |
0.1142 |
0.1234 |
0.0017 |
lh_sys3 |
0.1360 |
0.1320 |
0.1142 |
0.0825 |
0.0012 |
Ir_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lh_sys2 |
0.1360 |
0.1320 |
0.1142 |
0.1234 |
0.0017 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
- |
- |
- |
- |
- |
aueb-nlp-2 |
- |
- |
- |
- |
- |
aueb-nlp-3 |
- |
- |
- |
- |
- |
aueb-nlp-4 |
- |
- |
- |
- |
- |
aueb-nlp-5 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
- |
- |
- |
- |
- |
aueb-nlp-1 |
- |
- |
- |
- |
- |
aueb-nlp-2 |
- |
- |
- |
- |
- |
aueb-nlp-3 |
- |
- |
- |
- |
- |
aueb-nlp-4 |
- |
- |
- |
- |
- |
aueb-nlp-5 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
lh_sys1 |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
lh_sys3 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
lh_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
lh_sys5 |
- |
- |
- |
- |
- |
lh_sys2 |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
auth-qa-1 |
0.1910 |
0.2842 |
0.1908 |
0.0651 |
0.0008 |
aueb-nlp-1 |
0.2541 |
0.6668 |
0.2998 |
0.2102 |
0.0316 |
aueb-nlp-2 |
0.2531 |
0.6523 |
0.2992 |
0.2092 |
0.0279 |
aueb-nlp-3 |
0.2401 |
0.6451 |
0.2857 |
0.1962 |
0.0282 |
aueb-nlp-4 |
0.2481 |
0.6445 |
0.2948 |
0.2080 |
0.0268 |
aueb-nlp-5 |
0.4537 |
0.6416 |
0.4580 |
0.1968 |
0.0291 |
Deep ML methods for |
0.1980 |
0.5399 |
0.2350 |
0.1646 |
0.0101 |
MindLab QA Reloaded |
0.2080 |
0.5664 |
0.2463 |
0.1724 |
0.0121 |
MindLab QA System ++ |
0.2080 |
0.5664 |
0.2463 |
0.1724 |
0.0121 |
MindLab QA System |
0.2080 |
0.5664 |
0.2463 |
0.1724 |
0.0121 |
MindLab Red Lions++ |
0.2080 |
0.5664 |
0.2463 |
0.1724 |
0.0121 |
lh_sys2 |
0.2120 |
0.5922 |
0.2558 |
0.1594 |
0.0142 |
lh_sys1 |
0.2140 |
0.5856 |
0.2564 |
0.1673 |
0.0141 |
lh_sys3 |
0.2040 |
0.5714 |
0.2470 |
0.1569 |
0.0117 |
lh_sys4 |
0.2230 |
0.6121 |
0.2695 |
0.1752 |
0.0186 |
lh_sys5 |
0.2160 |
0.5901 |
0.2591 |
0.1691 |
0.0154 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |