BioASQ Participants Area
Task 8b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.1720 |
0.4397 |
0.2028 |
0.3648 |
0.0114 |
AUEB-System3 |
0.1720 |
0.4397 |
0.2028 |
0.2747 |
0.0098 |
AUEB-System5 |
0.1490 |
0.3914 |
0.1762 |
0.3066 |
0.0068 |
pa |
0.2822 |
0.4307 |
0.2977 |
0.3591 |
0.0109 |
AUEB-System4 |
0.0310 |
0.0821 |
0.0364 |
0.0547 |
0.0001 |
bio-answerfinder |
0.2933 |
0.2841 |
0.2474 |
0.2405 |
0.0024 |
bioinfo-0 |
0.1630 |
0.4557 |
0.1981 |
0.3409 |
0.0091 |
bioinfo-1 |
0.1760 |
0.4684 |
0.2130 |
0.3745 |
0.0142 |
bioinfo-2 |
0.1740 |
0.4997 |
0.2146 |
0.3744 |
0.0173 |
bioinfo-3 |
0.1730 |
0.4972 |
0.2143 |
0.3798 |
0.0172 |
bioinfo-4 |
0.1790 |
0.4946 |
0.2187 |
0.3823 |
0.0163 |
Deep ML methods for |
0.1130 |
0.2092 |
0.1221 |
0.1650 |
0.0007 |
MindLab QA System |
0.1130 |
0.2092 |
0.1221 |
0.1650 |
0.0007 |
MindLab QA Reloaded |
0.1130 |
0.2092 |
0.1221 |
0.1650 |
0.0007 |
MindLab QA System ++ |
0.1130 |
0.2092 |
0.1221 |
0.1650 |
0.0007 |
MindLab Red Lions++ |
0.1130 |
0.2092 |
0.1221 |
0.1650 |
0.0007 |
lalala |
0.1650 |
0.4554 |
0.2018 |
0.3595 |
0.0094 |
Ir_sys1 |
0.1520 |
0.3909 |
0.1814 |
0.3362 |
0.0039 |
Ir_sys2 |
0.1560 |
0.4352 |
0.1902 |
0.3372 |
0.0075 |
Ir_sys3 |
0.1600 |
0.4393 |
0.1934 |
0.3380 |
0.0095 |
Ir_sys4 |
0.1590 |
0.4393 |
0.1935 |
0.3352 |
0.0084 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2142 |
0.2562 |
0.1960 |
0.7703 |
0.0038 |
AUEB-System3 |
0.1183 |
0.1286 |
0.0947 |
0.2323 |
0.0016 |
AUEB-System5 |
0.1395 |
0.1475 |
0.1117 |
0.3273 |
0.0021 |
pa |
0.2030 |
0.2635 |
0.1973 |
0.5441 |
0.0057 |
AUEB-System4 |
0.0242 |
0.0325 |
0.0192 |
0.0320 |
0.0000 |
bio-answerfinder |
0.1773 |
0.1896 |
0.1466 |
0.4704 |
0.0015 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
0.2006 |
0.1911 |
0.1612 |
0.2690 |
0.0012 |
bioinfo-2 |
0.2231 |
0.1826 |
0.1696 |
0.2965 |
0.0013 |
bioinfo-3 |
0.2206 |
0.1880 |
0.1724 |
0.2313 |
0.0011 |
bioinfo-4 |
0.2259 |
0.1910 |
0.1704 |
0.2804 |
0.0017 |
Deep ML methods for |
0.0822 |
0.0774 |
0.0725 |
0.1714 |
0.0002 |
MindLab QA System |
0.0749 |
0.0880 |
0.0729 |
0.1000 |
0.0002 |
MindLab QA Reloaded |
0.0852 |
0.0833 |
0.0764 |
0.1643 |
0.0002 |
MindLab QA System ++ |
0.0800 |
0.0757 |
0.0716 |
0.1792 |
0.0001 |
MindLab Red Lions++ |
0.0915 |
0.0789 |
0.0776 |
0.1530 |
0.0002 |
lalala |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2448 |
0.5175 |
0.2742 |
0.3806 |
0.0522 |
AUEB-System2 |
0.2388 |
0.5115 |
0.2687 |
0.3632 |
0.0455 |
AUEB-System3 |
0.2448 |
0.5175 |
0.2742 |
0.3122 |
0.0416 |
AUEB-System4 |
0.0290 |
0.0837 |
0.0340 |
0.0723 |
0.0001 |
AUEB-System5 |
0.2198 |
0.4618 |
0.2438 |
0.3243 |
0.0307 |
Google-AdHoc-MAGLEV |
0.1970 |
0.4623 |
0.2261 |
0.3192 |
0.0346 |
Google-AdHoc-Neural |
0.1230 |
0.3246 |
0.1470 |
0.1799 |
0.0047 |
Google-AdHoc-BM25 |
0.1984 |
0.4540 |
0.2263 |
0.3054 |
0.0295 |
bio-answerfinder |
0.2584 |
0.3367 |
0.2376 |
0.2497 |
0.0075 |
bioinfo-4 |
0.2310 |
0.5160 |
0.2615 |
0.3686 |
0.0417 |
bioinfo-1 |
0.2240 |
0.5104 |
0.2572 |
0.3707 |
0.0566 |
bioinfo-3 |
0.2240 |
0.5198 |
0.2581 |
0.3719 |
0.0675 |
bioinfo-2 |
0.2180 |
0.5004 |
0.2496 |
0.3643 |
0.0465 |
MindLab QA Reloaded |
0.1420 |
0.2964 |
0.1566 |
0.1989 |
0.0055 |
MindLab Red Lions++ |
0.1420 |
0.2964 |
0.1566 |
0.1989 |
0.0055 |
Deep ML methods for |
0.1420 |
0.2964 |
0.1566 |
0.1989 |
0.0055 |
MindLab QA System |
0.1420 |
0.2964 |
0.1566 |
0.1989 |
0.0055 |
MindLab QA System ++ |
0.1420 |
0.2964 |
0.1566 |
0.1989 |
0.0055 |
pa |
0.2848 |
0.4960 |
0.3013 |
0.3945 |
0.0600 |
Ir_sys1 |
0.1800 |
0.4154 |
0.2050 |
0.2663 |
0.0130 |
Ir_sys2 |
0.1670 |
0.4246 |
0.1973 |
0.3036 |
0.0120 |
bioinfo-0 |
0.2140 |
0.4754 |
0.2428 |
0.3411 |
0.0378 |
Ir_sys4 |
0.1670 |
0.4246 |
0.1973 |
0.3120 |
0.0129 |
lalala |
0.1890 |
0.4338 |
0.2158 |
0.3091 |
0.0163 |
Google-AdHoc-M-Full |
0.1520 |
0.3648 |
0.1762 |
0.2182 |
0.0087 |
Ir_sys3 |
0.1838 |
0.3944 |
0.2043 |
0.2913 |
0.0091 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2257 |
0.2832 |
0.2191 |
0.7464 |
0.0089 |
AUEB-System2 |
0.1930 |
0.2435 |
0.1909 |
0.7058 |
0.0044 |
AUEB-System3 |
0.1415 |
0.1348 |
0.1076 |
0.1845 |
0.0022 |
AUEB-System4 |
0.0170 |
0.0103 |
0.0114 |
0.0306 |
0.0000 |
AUEB-System5 |
0.1380 |
0.1291 |
0.1050 |
0.2550 |
0.0022 |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
bio-answerfinder |
0.1594 |
0.1912 |
0.1410 |
0.1899 |
0.0036 |
bioinfo-4 |
0.1834 |
0.2181 |
0.1712 |
0.3051 |
0.0052 |
bioinfo-1 |
0.1839 |
0.2250 |
0.1723 |
0.2941 |
0.0063 |
bioinfo-3 |
0.1767 |
0.2128 |
0.1646 |
0.2889 |
0.0051 |
bioinfo-2 |
0.1758 |
0.2118 |
0.1651 |
0.2935 |
0.0039 |
MindLab QA Reloaded |
0.0940 |
0.1247 |
0.0905 |
0.2410 |
0.0006 |
MindLab Red Lions++ |
0.1235 |
0.1606 |
0.1153 |
0.2783 |
0.0015 |
Deep ML methods for |
0.0966 |
0.0948 |
0.0862 |
0.2115 |
0.0003 |
MindLab QA System |
0.1185 |
0.1581 |
0.1125 |
0.2819 |
0.0015 |
MindLab QA System ++ |
0.0881 |
0.1118 |
0.0845 |
0.2077 |
0.0004 |
pa |
0.2123 |
0.2943 |
0.2132 |
0.4147 |
0.0222 |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
bioinfo-0 |
0.1722 |
0.1794 |
0.1492 |
0.2193 |
0.0031 |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Google-MAGLEV-RERANK |
0.1920 |
0.5593 |
0.2316 |
0.5045 |
0.0512 |
Google-AdHoc-Neural |
0.1280 |
0.4196 |
0.1604 |
0.2941 |
0.0078 |
Google-AdHoc-MAGLEV |
0.1920 |
0.5593 |
0.2316 |
0.4564 |
0.0434 |
Google-AdHoc-BM25 |
0.2020 |
0.5510 |
0.2429 |
0.4460 |
0.0375 |
AUEB-System1 |
0.2283 |
0.5861 |
0.2654 |
0.5317 |
0.0621 |
AUEB-System2 |
0.2193 |
0.5772 |
0.2575 |
0.5215 |
0.0562 |
AUEB-System3 |
0.2283 |
0.5861 |
0.2654 |
0.3960 |
0.0483 |
AUEB-System4 |
0.2233 |
0.5802 |
0.2604 |
0.5329 |
0.0625 |
AUEB-System5 |
0.1973 |
0.5318 |
0.2328 |
0.4295 |
0.0496 |
bio-answerfinder |
0.2950 |
0.3521 |
0.2740 |
0.3121 |
0.0046 |
Ir_sys1 |
0.1680 |
0.4593 |
0.2027 |
0.3482 |
0.0108 |
Ir_sys2 |
0.1580 |
0.4428 |
0.1900 |
0.3828 |
0.0100 |
Ir_sys3 |
0.1650 |
0.4546 |
0.1986 |
0.3798 |
0.0147 |
bioinfo-1 |
0.2070 |
0.5673 |
0.2496 |
0.5023 |
0.0610 |
bioinfo-2 |
0.2070 |
0.5762 |
0.2505 |
0.5101 |
0.0736 |
bioinfo-3 |
0.2110 |
0.5832 |
0.2542 |
0.5230 |
0.0763 |
bioinfo-4 |
0.2170 |
0.5829 |
0.2592 |
0.5108 |
0.0776 |
bioinfo-0 |
0.2130 |
0.5848 |
0.2563 |
0.5172 |
0.0773 |
MindLab QA System |
0.1830 |
0.5097 |
0.2191 |
0.3938 |
0.0188 |
MindLab QA System ++ |
0.1830 |
0.5097 |
0.2191 |
0.3938 |
0.0188 |
MindLab Red Lions++ |
0.1830 |
0.5097 |
0.2191 |
0.3938 |
0.0188 |
MindLab QA Reloaded |
0.1400 |
0.3876 |
0.1692 |
0.2872 |
0.0064 |
Deep ML methods for |
0.1820 |
0.5094 |
0.2185 |
0.3940 |
0.0188 |
pa |
0.3662 |
0.5671 |
0.3919 |
0.5273 |
0.0606 |
pa-base |
0.3662 |
0.5671 |
0.3919 |
0.5273 |
0.0606 |
Google-AdHoc-M-Full |
0.1920 |
0.5593 |
0.2316 |
0.4564 |
0.0434 |
Ir_sys4 |
0.1680 |
0.4655 |
0.2019 |
0.3912 |
0.0125 |
lalala |
0.1780 |
0.4793 |
0.2124 |
0.4007 |
0.0177 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Google-MAGLEV-RERANK |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
AUEB-System1 |
0.2654 |
0.4026 |
0.2626 |
1.1262 |
0.0349 |
AUEB-System2 |
0.2591 |
0.4097 |
0.2657 |
1.1547 |
0.0349 |
AUEB-System3 |
0.1482 |
0.1811 |
0.1285 |
0.2704 |
0.0065 |
AUEB-System4 |
0.2096 |
0.2854 |
0.1862 |
0.4821 |
0.0239 |
AUEB-System5 |
0.1649 |
0.2057 |
0.1430 |
0.3653 |
0.0056 |
bio-answerfinder |
0.1928 |
0.2174 |
0.1573 |
0.3649 |
0.0027 |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
bioinfo-1 |
0.2240 |
0.3199 |
0.2151 |
0.4528 |
0.0170 |
bioinfo-2 |
0.2278 |
0.3341 |
0.2229 |
0.4704 |
0.0162 |
bioinfo-3 |
0.2205 |
0.3481 |
0.2252 |
0.4864 |
0.0170 |
bioinfo-4 |
0.2265 |
0.3203 |
0.2179 |
0.4474 |
0.0152 |
bioinfo-0 |
- |
- |
- |
- |
- |
MindLab QA System |
0.1284 |
0.1799 |
0.1239 |
0.3561 |
0.0013 |
MindLab QA System ++ |
0.1280 |
0.1861 |
0.1222 |
0.2560 |
0.0022 |
MindLab Red Lions++ |
0.1333 |
0.1855 |
0.1269 |
0.2619 |
0.0022 |
MindLab QA Reloaded |
0.0871 |
0.1408 |
0.0882 |
0.1999 |
0.0006 |
Deep ML methods for |
0.0643 |
0.0582 |
0.0512 |
0.0584 |
0.0002 |
pa |
0.2922 |
0.3887 |
0.2774 |
0.7946 |
0.0372 |
pa-base |
0.2613 |
0.3774 |
0.2583 |
0.8051 |
0.0237 |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Google-MAGLEV-RERANK |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
pa-base |
- |
- |
- |
- |
- |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Google-MAGLEV-RERANK |
- |
- |
- |
- |
- |
Google-AdHoc-Neural |
- |
- |
- |
- |
- |
Google-AdHoc-MAGLEV |
- |
- |
- |
- |
- |
Google-AdHoc-BM25 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Ir_sys1 |
- |
- |
- |
- |
- |
Ir_sys2 |
- |
- |
- |
- |
- |
Ir_sys3 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
pa |
- |
- |
- |
- |
- |
pa-base |
- |
- |
- |
- |
- |
Google-AdHoc-M-Full |
- |
- |
- |
- |
- |
Ir_sys4 |
- |
- |
- |
- |
- |
lalala |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2380 |
0.5941 |
0.2793 |
0.4901 |
0.0686 |
AUEB-System2 |
0.2300 |
0.5886 |
0.2726 |
0.4945 |
0.0692 |
AUEB-System3 |
0.2380 |
0.5941 |
0.2793 |
0.3605 |
0.0523 |
AUEB-System4 |
0.2370 |
0.5941 |
0.2787 |
0.4992 |
0.0700 |
AUEB-System5 |
0.1990 |
0.5157 |
0.2368 |
0.4076 |
0.0271 |
Google-MAGLEV-RERANK |
0.2030 |
0.5644 |
0.2471 |
0.4538 |
0.0699 |
Google-AdHoc-MAGLEV |
0.2030 |
0.5644 |
0.2471 |
0.4289 |
0.0657 |
Google-AdHoc-BM25 |
0.2060 |
0.5674 |
0.2490 |
0.4107 |
0.0554 |
Google-AdHoc-Neural |
0.1410 |
0.4068 |
0.1732 |
0.2823 |
0.0074 |
bioinfo-1 |
0.2180 |
0.5925 |
0.2630 |
0.4707 |
0.0713 |
bioinfo-2 |
0.2280 |
0.6006 |
0.2724 |
0.4810 |
0.0796 |
bioinfo-3 |
0.2180 |
0.5801 |
0.2615 |
0.4792 |
0.0479 |
bioinfo-4 |
0.2260 |
0.5893 |
0.2697 |
0.4677 |
0.0645 |
bio-answerfinder |
0.3956 |
0.4295 |
0.3566 |
0.3693 |
0.0180 |
submit mesh index qa |
0.1940 |
0.5462 |
0.2368 |
0.4021 |
0.0530 |
same as one |
0.1980 |
0.5410 |
0.2409 |
0.4248 |
0.0525 |
Deep ML methods for |
0.1960 |
0.5454 |
0.2376 |
0.4094 |
0.0430 |
MindLab QA System |
0.1936 |
0.5022 |
0.2314 |
0.3740 |
0.0297 |
Ir_sys1 |
0.2000 |
0.5473 |
0.2443 |
0.3898 |
0.0342 |
Ir_sys2 |
0.1880 |
0.5406 |
0.2304 |
0.3910 |
0.0356 |
Ir_sys3 |
0.1840 |
0.5445 |
0.2280 |
0.4047 |
0.0350 |
Ir_sys4 |
0.1980 |
0.5544 |
0.2415 |
0.3959 |
0.0365 |
lalala |
0.2040 |
0.5755 |
0.2480 |
0.4312 |
0.0507 |
bioinfo-0 |
0.2280 |
0.6008 |
0.2729 |
0.4809 |
0.0734 |
pa |
0.3721 |
0.5325 |
0.3874 |
0.4115 |
0.0421 |
Google-AdHoc-M-Full |
0.2030 |
0.5644 |
0.2471 |
0.4289 |
0.0657 |