BioASQ Participants Area
Task 9b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2657 |
0.4722 |
0.2615 |
0.4352 |
0.0452 |
AUEB-System2 |
0.2537 |
0.4646 |
0.2519 |
0.4461 |
0.0423 |
AUEB-System3 |
0.2647 |
0.4713 |
0.2605 |
0.4372 |
0.0453 |
bio-answerfinder |
0.4468 |
0.3782 |
0.3486 |
0.3607 |
0.0185 |
bioinfo-0 |
0.2230 |
0.4188 |
0.2232 |
0.3781 |
0.0172 |
bioinfo-1 |
0.2250 |
0.4331 |
0.2270 |
0.3891 |
0.0196 |
pa-1 |
0.2347 |
0.4368 |
0.2349 |
0.4194 |
0.0264 |
pa-2 |
0.2407 |
0.4417 |
0.2402 |
0.4261 |
0.0289 |
RYGH-1 |
0.2910 |
0.4969 |
0.2828 |
0.4764 |
0.0529 |
RYGH |
0.2860 |
0.4975 |
0.2797 |
0.4561 |
0.0474 |
RYGH-4 |
0.2740 |
0.4944 |
0.2728 |
0.4372 |
0.0455 |
RYGH-5 |
0.2700 |
0.4859 |
0.2683 |
0.4292 |
0.0440 |
bioinfo-2 |
0.2380 |
0.4729 |
0.2429 |
0.4405 |
0.0366 |
RYGH-3 |
0.2490 |
0.4605 |
0.2507 |
0.4017 |
0.0371 |
bioinfo-3 |
0.2380 |
0.4629 |
0.2418 |
0.4403 |
0.0363 |
bioinfo-4 |
0.2500 |
0.4636 |
0.2499 |
0.4426 |
0.0315 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2187 |
0.2197 |
0.1746 |
0.3712 |
0.0199 |
AUEB-System2 |
0.2142 |
0.2000 |
0.1645 |
0.3823 |
0.0185 |
AUEB-System3 |
0.2200 |
0.2140 |
0.1743 |
0.3649 |
0.0199 |
bio-answerfinder |
0.2901 |
0.2636 |
0.2258 |
0.6476 |
0.0111 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
0.2569 |
0.2547 |
0.2227 |
0.9789 |
0.0103 |
pa-2 |
0.2576 |
0.2554 |
0.2234 |
0.9791 |
0.0112 |
RYGH-1 |
0.2897 |
0.2617 |
0.2236 |
0.3979 |
0.0218 |
RYGH |
0.2867 |
0.2584 |
0.2202 |
0.3910 |
0.0218 |
RYGH-4 |
0.2779 |
0.2665 |
0.2183 |
0.4103 |
0.0246 |
RYGH-5 |
0.2846 |
0.2789 |
0.2271 |
0.4150 |
0.0256 |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
0.2716 |
0.2581 |
0.2140 |
0.4045 |
0.0240 |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2610 |
0.5298 |
0.2659 |
0.4497 |
0.0642 |
AUEB-System2 |
0.2600 |
0.5268 |
0.2660 |
0.4514 |
0.0580 |
AUEB-System3 |
0.2610 |
0.5307 |
0.2660 |
0.4528 |
0.0645 |
Olive-DocFetchV1 |
0.1790 |
0.3881 |
0.1872 |
0.3099 |
0.0175 |
Olive-DocFetchV2 |
0.2500 |
0.3317 |
0.2120 |
0.2790 |
0.0096 |
OliveSentSim1 |
0.1110 |
0.2386 |
0.1092 |
0.1052 |
0.0022 |
OliveSentSim2 |
0.1430 |
0.3119 |
0.1466 |
0.2084 |
0.0076 |
OliveAnsScore1 |
0.1360 |
0.2697 |
0.1351 |
0.1387 |
0.0042 |
OliveAnsScore2 |
0.1570 |
0.3331 |
0.1597 |
0.1996 |
0.0086 |
bio-answerfinder |
0.4319 |
0.4531 |
0.3797 |
0.4250 |
0.0280 |
bioinfo-0 |
0.2370 |
0.5618 |
0.2519 |
0.4316 |
0.0795 |
bioinfo-1 |
0.2340 |
0.5412 |
0.2471 |
0.4459 |
0.0721 |
bioinfo-2 |
0.2480 |
0.5238 |
0.2495 |
0.4709 |
0.0620 |
Deep ML methods for |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
MindLab QA System |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
pa-1 |
0.2850 |
0.5304 |
0.2814 |
0.4947 |
0.0641 |
pa-2 |
0.2860 |
0.5338 |
0.2829 |
0.4892 |
0.0635 |
MindLab QA System ++ |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
pa-3 |
0.2740 |
0.5190 |
0.2710 |
0.4750 |
0.0592 |
pa-4 |
0.2780 |
0.5265 |
0.2756 |
0.4824 |
0.0605 |
MindLab QA Reloaded |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
MindLab Red Lions++ |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
RYGH-1 |
0.3020 |
0.6038 |
0.2973 |
0.5263 |
0.1250 |
RYGH-5 |
0.3050 |
0.5994 |
0.3005 |
0.5423 |
0.1194 |
RYGH-4 |
0.3080 |
0.6094 |
0.3031 |
0.5350 |
0.1283 |
bioinfo-3 |
0.2460 |
0.5112 |
0.2450 |
0.4755 |
0.0521 |
bioinfo-4 |
0.2580 |
0.5587 |
0.2649 |
0.4913 |
0.0852 |
RYGH-3 |
0.2990 |
0.5802 |
0.2926 |
0.5138 |
0.0940 |
RYGH |
0.2980 |
0.5903 |
0.2933 |
0.5240 |
0.1141 |
pa-5 |
0.3608 |
0.5295 |
0.3479 |
0.4973 |
0.0479 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2071 |
0.2251 |
0.1658 |
0.3777 |
0.0163 |
AUEB-System2 |
0.1948 |
0.2052 |
0.1553 |
0.3418 |
0.0118 |
AUEB-System3 |
0.2041 |
0.2222 |
0.1688 |
0.3761 |
0.0188 |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
0.3011 |
0.2769 |
0.2318 |
0.6272 |
0.0168 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.1588 |
0.1701 |
0.1303 |
0.1915 |
0.0028 |
MindLab QA System |
0.1573 |
0.1672 |
0.1271 |
0.1883 |
0.0029 |
pa-1 |
0.2292 |
0.2667 |
0.2012 |
0.9002 |
0.0119 |
pa-2 |
0.2279 |
0.2455 |
0.1966 |
0.8729 |
0.0089 |
MindLab QA System ++ |
0.1309 |
0.0935 |
0.0891 |
0.1582 |
0.0006 |
pa-3 |
0.2390 |
0.2759 |
0.2123 |
0.9492 |
0.0160 |
pa-4 |
0.2421 |
0.2781 |
0.2144 |
0.9473 |
0.0159 |
MindLab QA Reloaded |
0.1309 |
0.0935 |
0.0891 |
0.1582 |
0.0006 |
MindLab Red Lions++ |
0.1604 |
0.1737 |
0.1295 |
0.1912 |
0.0033 |
RYGH-1 |
0.3090 |
0.3726 |
0.2521 |
0.5677 |
0.0678 |
RYGH-5 |
0.3092 |
0.3642 |
0.2525 |
0.5403 |
0.0604 |
RYGH-4 |
0.3132 |
0.3766 |
0.2574 |
0.5626 |
0.0685 |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
0.3078 |
0.3481 |
0.2482 |
0.5506 |
0.0462 |
RYGH |
0.3141 |
0.3730 |
0.2533 |
0.5333 |
0.0651 |
pa-5 |
0.2357 |
0.2571 |
0.2062 |
0.8779 |
0.0102 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2347 |
0.5229 |
0.2467 |
0.4477 |
0.0485 |
AUEB-System2 |
0.2306 |
0.5185 |
0.2426 |
0.4401 |
0.0407 |
AUEB-System3 |
0.2368 |
0.5238 |
0.2479 |
0.4506 |
0.0491 |
bio-answerfinder |
0.4169 |
0.4611 |
0.3718 |
0.4163 |
0.0235 |
Olive-DocFetchV1 |
0.2031 |
0.4715 |
0.2156 |
0.3877 |
0.0283 |
Olive-DocFetchV2 |
0.2742 |
0.3781 |
0.2364 |
0.3426 |
0.0142 |
Olive-DocFetchV3 |
0.1849 |
0.4619 |
0.1997 |
0.3938 |
0.0203 |
OliveAnsScore1 |
0.1010 |
0.2063 |
0.1008 |
0.1208 |
0.0012 |
OliveSentSim1 |
0.1062 |
0.1925 |
0.1021 |
0.0834 |
0.0012 |
OliveSentSim2 |
0.1062 |
0.2068 |
0.1051 |
0.1469 |
0.0019 |
OliveAnsScore2 |
0.1186 |
0.2840 |
0.1263 |
0.1789 |
0.0031 |
bioinfo-0 |
0.2278 |
0.4986 |
0.2373 |
0.4332 |
0.0332 |
bioinfo-1 |
0.2309 |
0.5042 |
0.2420 |
0.4392 |
0.0337 |
bioinfo-2 |
0.2577 |
0.5340 |
0.2663 |
0.4717 |
0.0597 |
bioinfo-3 |
0.2588 |
0.5347 |
0.2666 |
0.4900 |
0.0607 |
bioinfo-4 |
0.2649 |
0.5347 |
0.2682 |
0.4912 |
0.0504 |
pa-1 |
0.2505 |
0.5206 |
0.2547 |
0.4935 |
0.0403 |
pa-2 |
0.2464 |
0.5259 |
0.2518 |
0.4891 |
0.0400 |
pa-3 |
0.2505 |
0.5124 |
0.2538 |
0.4741 |
0.0382 |
pa-4 |
0.2423 |
0.5153 |
0.2479 |
0.4673 |
0.0408 |
Deep ML methods for |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA System |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA System ++ |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA Reloaded |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
RYGH-1 |
0.2876 |
0.5746 |
0.2902 |
0.5265 |
0.0733 |
RYGH-3 |
0.2876 |
0.5737 |
0.2895 |
0.5271 |
0.0678 |
RYGH-4 |
0.2948 |
0.5762 |
0.2942 |
0.5273 |
0.0738 |
RYGH-5 |
0.2856 |
0.5635 |
0.2870 |
0.5241 |
0.0615 |
RYGH |
0.2918 |
0.5803 |
0.2943 |
0.5267 |
0.0737 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.1940 |
0.2247 |
0.1668 |
0.3734 |
0.0236 |
AUEB-System2 |
0.1780 |
0.1861 |
0.1461 |
0.3217 |
0.0121 |
AUEB-System3 |
0.1995 |
0.2212 |
0.1689 |
0.3473 |
0.0225 |
bio-answerfinder |
0.2930 |
0.3264 |
0.2614 |
0.6840 |
0.0217 |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
0.2423 |
0.3025 |
0.2391 |
0.9200 |
0.0114 |
pa-2 |
0.2383 |
0.2947 |
0.2352 |
0.9022 |
0.0092 |
pa-3 |
0.2308 |
0.2830 |
0.2226 |
0.8669 |
0.0081 |
pa-4 |
0.2321 |
0.2774 |
0.2217 |
0.8528 |
0.0072 |
Deep ML methods for |
0.1413 |
0.1490 |
0.1237 |
0.1390 |
0.0018 |
MindLab QA System |
0.1024 |
0.0785 |
0.0738 |
0.0675 |
0.0004 |
MindLab QA System ++ |
0.1024 |
0.0785 |
0.0738 |
0.0675 |
0.0004 |
MindLab QA Reloaded |
0.1413 |
0.1490 |
0.1237 |
0.1390 |
0.0018 |
RYGH-1 |
0.3175 |
0.3629 |
0.2772 |
0.5646 |
0.0465 |
RYGH-3 |
0.3141 |
0.3557 |
0.2739 |
0.5498 |
0.0386 |
RYGH-4 |
0.3230 |
0.3566 |
0.2799 |
0.5347 |
0.0390 |
RYGH-5 |
0.3132 |
0.3467 |
0.2738 |
0.5186 |
0.0375 |
RYGH |
0.3174 |
0.3712 |
0.2788 |
0.5606 |
0.0562 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
0.2190 |
0.4657 |
0.2322 |
0.4141 |
0.0257 |
Olive-DocFetchV2 |
0.3020 |
0.4022 |
0.2634 |
0.3681 |
0.0161 |
Olive-DocFetchV3 |
0.1980 |
0.4645 |
0.2153 |
0.4011 |
0.0308 |
AUEB-System1 |
0.2500 |
0.5271 |
0.2708 |
0.4453 |
0.0408 |
AUEB-System2 |
0.2460 |
0.5188 |
0.2661 |
0.4486 |
0.0411 |
AUEB-System3 |
0.2510 |
0.5277 |
0.2716 |
0.4444 |
0.0408 |
bio-answerfinder |
0.4179 |
0.4090 |
0.3534 |
0.3841 |
0.0208 |
bio-answerfinder-2 |
0.3972 |
0.4152 |
0.3549 |
0.3936 |
0.0214 |
pa-1 |
0.2670 |
0.4877 |
0.2720 |
0.4776 |
0.0414 |
pa-2 |
0.2710 |
0.4871 |
0.2743 |
0.4751 |
0.0377 |
pa-3 |
0.2643 |
0.4989 |
0.2729 |
0.4665 |
0.0444 |
pa-4 |
0.2683 |
0.5039 |
0.2768 |
0.4649 |
0.0445 |
bioinfo-0 |
0.2420 |
0.5113 |
0.2540 |
0.4601 |
0.0341 |
bioinfo-1 |
0.2430 |
0.5056 |
0.2550 |
0.4556 |
0.0305 |
OliveAnsScore1 |
0.1630 |
0.3392 |
0.1718 |
0.1932 |
0.0066 |
OliveAnsScore2 |
0.1810 |
0.3922 |
0.1951 |
0.2512 |
0.0110 |
OliveSentSim1 |
0.1140 |
0.2010 |
0.1143 |
0.0799 |
0.0013 |
OliveSentSim2 |
0.1500 |
0.3096 |
0.1548 |
0.2083 |
0.0061 |
RYGH-1 |
0.2650 |
0.5386 |
0.2732 |
0.4966 |
0.0479 |
RYGH-3 |
0.2680 |
0.5458 |
0.2784 |
0.5222 |
0.0599 |
RYGH-4 |
0.2610 |
0.5323 |
0.2682 |
0.5106 |
0.0533 |
RYGH-5 |
0.2690 |
0.5434 |
0.2783 |
0.5154 |
0.0547 |
RYGH |
0.2630 |
0.5347 |
0.2700 |
0.5113 |
0.0528 |
bioinfo-2 |
0.2610 |
0.5325 |
0.2710 |
0.5127 |
0.0463 |
bioinfo-3 |
0.2590 |
0.5439 |
0.2717 |
0.5015 |
0.0458 |
bioinfo-4 |
0.2590 |
0.5439 |
0.2717 |
0.5015 |
0.0458 |
pa-5 |
0.3736 |
0.5220 |
0.3687 |
0.5156 |
0.0439 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
0.1994 |
0.2213 |
0.1648 |
0.3607 |
0.0132 |
AUEB-System2 |
0.1847 |
0.2092 |
0.1477 |
0.3626 |
0.0136 |
AUEB-System3 |
0.2020 |
0.2236 |
0.1644 |
0.3426 |
0.0144 |
bio-answerfinder |
0.2602 |
0.2639 |
0.2185 |
0.7283 |
0.0080 |
bio-answerfinder-2 |
0.2675 |
0.2667 |
0.2227 |
0.7247 |
0.0108 |
pa-1 |
0.2306 |
0.2770 |
0.2018 |
0.8857 |
0.0135 |
pa-2 |
0.2305 |
0.2739 |
0.2009 |
0.8711 |
0.0116 |
pa-3 |
0.2302 |
0.2733 |
0.2050 |
0.8640 |
0.0135 |
pa-4 |
0.2331 |
0.2736 |
0.2056 |
0.8639 |
0.0132 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
0.1985 |
0.2183 |
0.1568 |
0.3310 |
0.0124 |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
RYGH-1 |
0.2908 |
0.3554 |
0.2564 |
0.5890 |
0.0338 |
RYGH-3 |
0.2950 |
0.3505 |
0.2576 |
0.5623 |
0.0380 |
RYGH-4 |
0.2922 |
0.3511 |
0.2547 |
0.5699 |
0.0300 |
RYGH-5 |
0.2982 |
0.3521 |
0.2598 |
0.5673 |
0.0361 |
RYGH |
0.2897 |
0.3493 |
0.2529 |
0.5689 |
0.0324 |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
0.2466 |
0.2713 |
0.2017 |
0.4385 |
0.0199 |
bioinfo-4 |
0.2166 |
0.2450 |
0.1884 |
0.5973 |
0.0100 |
pa-5 |
0.2622 |
0.3136 |
0.2338 |
0.9989 |
0.0170 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
Test batch 5
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
0.2313 |
0.4500 |
0.2328 |
0.3965 |
0.0405 |
Olive-DocFetchV2 |
0.2313 |
0.4500 |
0.2328 |
0.3965 |
0.0405 |
Olive-DocFetchV3 |
0.2008 |
0.4229 |
0.2066 |
0.3710 |
0.0215 |
AUEB-System1 |
0.2803 |
0.4740 |
0.2743 |
0.4318 |
0.0380 |
AUEB-System2 |
0.2763 |
0.4755 |
0.2723 |
0.4340 |
0.0422 |
AUEB-System3 |
0.2723 |
0.4702 |
0.2680 |
0.4147 |
0.0382 |
AUEB-System4 |
0.2633 |
0.4546 |
0.2597 |
0.4171 |
0.0381 |
AUEB-System5 |
0.2733 |
0.4716 |
0.2692 |
0.4240 |
0.0390 |
OliveAnsScore1 |
0.1810 |
0.2978 |
0.1677 |
0.1657 |
0.0069 |
OliveAnsScore2 |
0.2060 |
0.3861 |
0.2009 |
0.2408 |
0.0154 |
OliveSentSim1 |
0.1380 |
0.2190 |
0.1270 |
0.0975 |
0.0020 |
OliveSentSim2 |
0.1720 |
0.3111 |
0.1665 |
0.2219 |
0.0077 |
bio-answerfinder |
0.4295 |
0.3890 |
0.3635 |
0.3670 |
0.0157 |
bio-answerfinder-2 |
0.3969 |
0.3750 |
0.3315 |
0.3415 |
0.0151 |
bioinfo-0 |
0.2460 |
0.4617 |
0.2392 |
0.3964 |
0.0429 |
bioinfo-1 |
0.2450 |
0.4610 |
0.2384 |
0.3995 |
0.0480 |
pa-1 |
0.2483 |
0.4559 |
0.2428 |
0.4597 |
0.0490 |
pa-2 |
0.2433 |
0.4521 |
0.2378 |
0.4517 |
0.0368 |
pa-3 |
0.2483 |
0.4559 |
0.2428 |
0.4383 |
0.0484 |
pa-4 |
0.2433 |
0.4521 |
0.2378 |
0.4310 |
0.0359 |
RYGH-1 |
0.3095 |
0.5367 |
0.2968 |
0.5038 |
0.0820 |
RYGH |
0.3175 |
0.5435 |
0.3031 |
0.5083 |
0.0989 |
RYGH-3 |
0.3095 |
0.5352 |
0.2954 |
0.5017 |
0.0758 |
RYGH-4 |
0.3208 |
0.5429 |
0.3052 |
0.5175 |
0.0856 |
RYGH-5 |
0.3075 |
0.5331 |
0.2936 |
0.4946 |
0.0730 |
bioinfo-2 |
0.2770 |
0.5294 |
0.2742 |
0.4701 |
0.0796 |
bioinfo-4 |
0.2610 |
0.4862 |
0.2554 |
0.4480 |
0.0532 |
bioinfo-3 |
0.2690 |
0.5069 |
0.2645 |
0.4529 |
0.0603 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
0.2216 |
0.2236 |
0.1743 |
0.3536 |
0.0136 |
AUEB-System2 |
0.2102 |
0.1838 |
0.1508 |
0.3189 |
0.0075 |
AUEB-System3 |
0.2131 |
0.2160 |
0.1679 |
0.3496 |
0.0107 |
AUEB-System4 |
0.2036 |
0.1711 |
0.1430 |
0.3110 |
0.0053 |
AUEB-System5 |
0.2188 |
0.2206 |
0.1690 |
0.3261 |
0.0142 |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
bio-answerfinder |
0.2929 |
0.2946 |
0.2386 |
0.5821 |
0.0153 |
bio-answerfinder-2 |
0.2816 |
0.2689 |
0.2184 |
0.4775 |
0.0127 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
0.2168 |
0.2325 |
0.1889 |
0.7126 |
0.0040 |
pa-1 |
0.2098 |
0.2743 |
0.1971 |
0.7795 |
0.0118 |
pa-2 |
0.2015 |
0.2705 |
0.1919 |
0.7653 |
0.0091 |
pa-3 |
0.1902 |
0.2477 |
0.1822 |
0.6835 |
0.0086 |
pa-4 |
0.1848 |
0.2432 |
0.1775 |
0.6691 |
0.0060 |
RYGH-1 |
0.3183 |
0.3535 |
0.2565 |
0.5272 |
0.0670 |
RYGH |
0.3124 |
0.3484 |
0.2514 |
0.5074 |
0.0681 |
RYGH-3 |
0.3126 |
0.3530 |
0.2535 |
0.5271 |
0.0625 |
RYGH-4 |
0.3134 |
0.3517 |
0.2532 |
0.5123 |
0.0610 |
RYGH-5 |
0.3094 |
0.3576 |
0.2551 |
0.5336 |
0.0632 |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-4 |
0.2401 |
0.2285 |
0.1910 |
0.5632 |
0.0046 |
bioinfo-3 |
0.2411 |
0.2541 |
0.2077 |
0.7441 |
0.0105 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |