BioASQ Participants Area
Task 9b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2657 |
0.4722 |
0.2615 |
0.4352 |
0.0452 |
AUEB-System2 |
0.2537 |
0.4646 |
0.2519 |
0.4461 |
0.0423 |
AUEB-System3 |
0.2647 |
0.4713 |
0.2605 |
0.4372 |
0.0453 |
bio-answerfinder |
0.4468 |
0.3782 |
0.3486 |
0.3607 |
0.0185 |
bioinfo-0 |
0.2230 |
0.4188 |
0.2232 |
0.3781 |
0.0172 |
bioinfo-1 |
0.2250 |
0.4331 |
0.2270 |
0.3891 |
0.0196 |
pa-1 |
0.2347 |
0.4368 |
0.2349 |
0.4194 |
0.0264 |
pa-2 |
0.2407 |
0.4417 |
0.2402 |
0.4261 |
0.0289 |
RYGH-1 |
0.2910 |
0.4969 |
0.2828 |
0.4764 |
0.0529 |
RYGH |
0.2860 |
0.4975 |
0.2797 |
0.4561 |
0.0474 |
RYGH-4 |
0.2740 |
0.4944 |
0.2728 |
0.4372 |
0.0455 |
RYGH-5 |
0.2700 |
0.4859 |
0.2683 |
0.4292 |
0.0440 |
bioinfo-2 |
0.2380 |
0.4729 |
0.2429 |
0.4405 |
0.0366 |
RYGH-3 |
0.2490 |
0.4605 |
0.2507 |
0.4017 |
0.0371 |
bioinfo-3 |
0.2380 |
0.4629 |
0.2418 |
0.4403 |
0.0363 |
bioinfo-4 |
0.2500 |
0.4636 |
0.2499 |
0.4426 |
0.0315 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2187 |
0.2197 |
0.1746 |
0.3712 |
0.0199 |
AUEB-System2 |
0.2142 |
0.2000 |
0.1645 |
0.3823 |
0.0185 |
AUEB-System3 |
0.2200 |
0.2140 |
0.1743 |
0.3649 |
0.0199 |
bio-answerfinder |
0.2901 |
0.2636 |
0.2258 |
0.6476 |
0.0111 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
0.2569 |
0.2547 |
0.2227 |
0.9789 |
0.0103 |
pa-2 |
0.2576 |
0.2554 |
0.2234 |
0.9791 |
0.0112 |
RYGH-1 |
0.2897 |
0.2617 |
0.2236 |
0.3979 |
0.0218 |
RYGH |
0.2867 |
0.2584 |
0.2202 |
0.3910 |
0.0218 |
RYGH-4 |
0.2779 |
0.2665 |
0.2183 |
0.4103 |
0.0246 |
RYGH-5 |
0.2846 |
0.2789 |
0.2271 |
0.4150 |
0.0256 |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
0.2716 |
0.2581 |
0.2140 |
0.4045 |
0.0240 |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2610 |
0.5298 |
0.2659 |
0.4497 |
0.0642 |
AUEB-System2 |
0.2600 |
0.5268 |
0.2660 |
0.4514 |
0.0580 |
AUEB-System3 |
0.2610 |
0.5307 |
0.2660 |
0.4528 |
0.0645 |
Olive-DocFetchV1 |
0.1790 |
0.3881 |
0.1872 |
0.3099 |
0.0175 |
Olive-DocFetchV2 |
0.2500 |
0.3317 |
0.2120 |
0.2790 |
0.0096 |
OliveSentSim1 |
0.1110 |
0.2386 |
0.1092 |
0.1052 |
0.0022 |
OliveSentSim2 |
0.1430 |
0.3119 |
0.1466 |
0.2084 |
0.0076 |
OliveAnsScore1 |
0.1360 |
0.2697 |
0.1351 |
0.1387 |
0.0042 |
OliveAnsScore2 |
0.1570 |
0.3331 |
0.1597 |
0.1996 |
0.0086 |
bio-answerfinder |
0.4319 |
0.4531 |
0.3797 |
0.4250 |
0.0280 |
bioinfo-0 |
0.2370 |
0.5618 |
0.2519 |
0.4316 |
0.0795 |
bioinfo-1 |
0.2340 |
0.5412 |
0.2471 |
0.4459 |
0.0721 |
bioinfo-2 |
0.2480 |
0.5238 |
0.2495 |
0.4709 |
0.0620 |
Deep ML methods for |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
MindLab QA System |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
pa-1 |
0.2850 |
0.5304 |
0.2814 |
0.4947 |
0.0641 |
pa-2 |
0.2860 |
0.5338 |
0.2829 |
0.4892 |
0.0635 |
MindLab QA System ++ |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
pa-3 |
0.2740 |
0.5190 |
0.2710 |
0.4750 |
0.0592 |
pa-4 |
0.2780 |
0.5265 |
0.2756 |
0.4824 |
0.0605 |
MindLab QA Reloaded |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
MindLab Red Lions++ |
0.2314 |
0.4514 |
0.2312 |
0.3753 |
0.0305 |
RYGH-1 |
0.3020 |
0.6038 |
0.2973 |
0.5263 |
0.1250 |
RYGH-5 |
0.3050 |
0.5994 |
0.3005 |
0.5423 |
0.1194 |
RYGH-4 |
0.3080 |
0.6094 |
0.3031 |
0.5350 |
0.1283 |
bioinfo-3 |
0.2460 |
0.5112 |
0.2450 |
0.4755 |
0.0521 |
bioinfo-4 |
0.2580 |
0.5587 |
0.2649 |
0.4913 |
0.0852 |
RYGH-3 |
0.2990 |
0.5802 |
0.2926 |
0.5138 |
0.0940 |
RYGH |
0.2980 |
0.5903 |
0.2933 |
0.5240 |
0.1141 |
pa-5 |
0.3608 |
0.5295 |
0.3479 |
0.4973 |
0.0479 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2071 |
0.2251 |
0.1658 |
0.3777 |
0.0163 |
AUEB-System2 |
0.1948 |
0.2052 |
0.1553 |
0.3418 |
0.0118 |
AUEB-System3 |
0.2041 |
0.2222 |
0.1688 |
0.3761 |
0.0188 |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
0.3011 |
0.2769 |
0.2318 |
0.6272 |
0.0168 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.1588 |
0.1701 |
0.1303 |
0.1915 |
0.0028 |
MindLab QA System |
0.1573 |
0.1672 |
0.1271 |
0.1883 |
0.0029 |
pa-1 |
0.2292 |
0.2667 |
0.2012 |
0.9002 |
0.0119 |
pa-2 |
0.2279 |
0.2455 |
0.1966 |
0.8729 |
0.0089 |
MindLab QA System ++ |
0.1309 |
0.0935 |
0.0891 |
0.1582 |
0.0006 |
pa-3 |
0.2390 |
0.2759 |
0.2123 |
0.9492 |
0.0160 |
pa-4 |
0.2421 |
0.2781 |
0.2144 |
0.9473 |
0.0159 |
MindLab QA Reloaded |
0.1309 |
0.0935 |
0.0891 |
0.1582 |
0.0006 |
MindLab Red Lions++ |
0.1604 |
0.1737 |
0.1295 |
0.1912 |
0.0033 |
RYGH-1 |
0.3090 |
0.3726 |
0.2521 |
0.5677 |
0.0678 |
RYGH-5 |
0.3092 |
0.3642 |
0.2525 |
0.5403 |
0.0604 |
RYGH-4 |
0.3132 |
0.3766 |
0.2574 |
0.5626 |
0.0685 |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
0.3078 |
0.3481 |
0.2482 |
0.5506 |
0.0462 |
RYGH |
0.3141 |
0.3730 |
0.2533 |
0.5333 |
0.0651 |
pa-5 |
0.2357 |
0.2571 |
0.2062 |
0.8779 |
0.0102 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
pa-5 |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.2347 |
0.5229 |
0.2467 |
0.4477 |
0.0485 |
AUEB-System2 |
0.2306 |
0.5185 |
0.2426 |
0.4401 |
0.0407 |
AUEB-System3 |
0.2368 |
0.5238 |
0.2479 |
0.4506 |
0.0491 |
bio-answerfinder |
0.4169 |
0.4611 |
0.3718 |
0.4163 |
0.0235 |
Olive-DocFetchV1 |
0.2031 |
0.4715 |
0.2156 |
0.3877 |
0.0283 |
Olive-DocFetchV2 |
0.2742 |
0.3781 |
0.2364 |
0.3426 |
0.0142 |
Olive-DocFetchV3 |
0.1849 |
0.4619 |
0.1997 |
0.3938 |
0.0203 |
OliveAnsScore1 |
0.1010 |
0.2063 |
0.1008 |
0.1208 |
0.0012 |
OliveSentSim1 |
0.1062 |
0.1925 |
0.1021 |
0.0834 |
0.0012 |
OliveSentSim2 |
0.1062 |
0.2068 |
0.1051 |
0.1469 |
0.0019 |
OliveAnsScore2 |
0.1186 |
0.2840 |
0.1263 |
0.1789 |
0.0031 |
bioinfo-0 |
0.2278 |
0.4986 |
0.2373 |
0.4332 |
0.0332 |
bioinfo-1 |
0.2309 |
0.5042 |
0.2420 |
0.4392 |
0.0337 |
bioinfo-2 |
0.2577 |
0.5340 |
0.2663 |
0.4717 |
0.0597 |
bioinfo-3 |
0.2588 |
0.5347 |
0.2666 |
0.4900 |
0.0607 |
bioinfo-4 |
0.2649 |
0.5347 |
0.2682 |
0.4912 |
0.0504 |
pa-1 |
0.2505 |
0.5206 |
0.2547 |
0.4935 |
0.0403 |
pa-2 |
0.2464 |
0.5259 |
0.2518 |
0.4891 |
0.0400 |
pa-3 |
0.2505 |
0.5124 |
0.2538 |
0.4741 |
0.0382 |
pa-4 |
0.2423 |
0.5153 |
0.2479 |
0.4673 |
0.0408 |
Deep ML methods for |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA System |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA System ++ |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
MindLab QA Reloaded |
0.2085 |
0.4430 |
0.2153 |
0.3624 |
0.0253 |
RYGH-1 |
0.2876 |
0.5746 |
0.2902 |
0.5265 |
0.0733 |
RYGH-3 |
0.2876 |
0.5737 |
0.2895 |
0.5271 |
0.0678 |
RYGH-4 |
0.2948 |
0.5762 |
0.2942 |
0.5273 |
0.0738 |
RYGH-5 |
0.2856 |
0.5635 |
0.2870 |
0.5241 |
0.0615 |
RYGH |
0.2918 |
0.5803 |
0.2943 |
0.5267 |
0.0737 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.1940 |
0.2247 |
0.1668 |
0.3734 |
0.0236 |
AUEB-System2 |
0.1780 |
0.1861 |
0.1461 |
0.3217 |
0.0121 |
AUEB-System3 |
0.1995 |
0.2212 |
0.1689 |
0.3473 |
0.0225 |
bio-answerfinder |
0.2930 |
0.3264 |
0.2614 |
0.6840 |
0.0217 |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
0.2423 |
0.3025 |
0.2391 |
0.9200 |
0.0114 |
pa-2 |
0.2383 |
0.2947 |
0.2352 |
0.9022 |
0.0092 |
pa-3 |
0.2308 |
0.2830 |
0.2226 |
0.8669 |
0.0081 |
pa-4 |
0.2321 |
0.2774 |
0.2217 |
0.8528 |
0.0072 |
Deep ML methods for |
0.1413 |
0.1490 |
0.1237 |
0.1390 |
0.0018 |
MindLab QA System |
0.1024 |
0.0785 |
0.0738 |
0.0675 |
0.0004 |
MindLab QA System ++ |
0.1024 |
0.0785 |
0.0738 |
0.0675 |
0.0004 |
MindLab QA Reloaded |
0.1413 |
0.1490 |
0.1237 |
0.1390 |
0.0018 |
RYGH-1 |
0.3175 |
0.3629 |
0.2772 |
0.5646 |
0.0465 |
RYGH-3 |
0.3141 |
0.3557 |
0.2739 |
0.5498 |
0.0386 |
RYGH-4 |
0.3230 |
0.3566 |
0.2799 |
0.5347 |
0.0390 |
RYGH-5 |
0.3132 |
0.3467 |
0.2738 |
0.5186 |
0.0375 |
RYGH |
0.3174 |
0.3712 |
0.2788 |
0.5606 |
0.0562 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
Olive-DocFetchV1 |
- |
- |
- |
- |
- |
Olive-DocFetchV2 |
- |
- |
- |
- |
- |
Olive-DocFetchV3 |
- |
- |
- |
- |
- |
OliveAnsScore1 |
- |
- |
- |
- |
- |
OliveSentSim1 |
- |
- |
- |
- |
- |
OliveSentSim2 |
- |
- |
- |
- |
- |
OliveAnsScore2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
pa-1 |
- |
- |
- |
- |
- |
pa-2 |
- |
- |
- |
- |
- |
pa-3 |
- |
- |
- |
- |
- |
pa-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
MindLab QA System ++ |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Olive-DocFetchV1 |
0.2190 |
0.4657 |
0.2322 |
0.4141 |
0.0257 |
Olive-DocFetchV2 |
0.3020 |
0.4022 |
0.2634 |
0.3681 |
0.0161 |
Olive-DocFetchV3 |
0.1980 |
0.4645 |
0.2153 |
0.4011 |
0.0308 |
AUEB-System1 |
0.2500 |
0.5271 |
0.2708 |
0.4453 |
0.0408 |
AUEB-System2 |
0.2460 |
0.5188 |
0.2661 |
0.4486 |
0.0411 |
AUEB-System3 |
0.2510 |
0.5277 |
0.2716 |
0.4444 |
0.0408 |
bio-answerfinder |
0.4179 |
0.4090 |
0.3534 |
0.3841 |
0.0208 |
bio-answerfinder-2 |
0.3972 |
0.4152 |
0.3549 |
0.3936 |
0.0214 |
pa-1 |
0.2670 |
0.4877 |
0.2720 |
0.4776 |
0.0414 |
pa-2 |
0.2710 |
0.4871 |
0.2743 |
0.4751 |
0.0377 |
pa-3 |
0.2643 |
0.4989 |
0.2729 |
0.4665 |
0.0444 |
pa-4 |
0.2683 |
0.5039 |
0.2768 |
0.4649 |
0.0445 |
bioinfo-0 |
0.2420 |
0.5113 |
0.2540 |
0.4601 |
0.0341 |
bioinfo-1 |
0.2430 |
0.5056 |
0.2550 |
0.4556 |
0.0305 |
OliveAnsScore1 |
0.1630 |
0.3392 |
0.1718 |
0.1932 |
0.0066 |
OliveAnsScore2 |
0.1810 |
0.3922 |
0.1951 |
0.2512 |
0.0110 |
OliveSentSim1 |
0.1140 |
0.2010 |
0.1143 |
0.0799 |
0.0013 |
OliveSentSim2 |
0.1500 |
0.3096 |
0.1548 |
0.2083 |
0.0061 |
RYGH-1 |
0.2650 |
0.5386 |
0.2732 |
0.4966 |
0.0479 |
RYGH-3 |
0.2680 |
0.5458 |
0.2784 |
0.5222 |
0.0599 |
RYGH-4 |
0.2610 |
0.5323 |
0.2682 |
0.5106 |
0.0533 |
RYGH-5 |
0.2690 |
0.5434 |
0.2783 |
0.5154 |
0.0547 |
RYGH |
0.2630 |
0.5347 |
0.2700 |
0.5113 |
0.0528 |
bioinfo-2 |
0.2610 |
0.5325 |
0.2710 |
0.5127 |
0.0463 |
bioinfo-3 |
0.2590 |
0.5439 |
0.2717 |
0.5015 |
0.0458 |
bioinfo-4 |
0.2590 |
0.5439 |
0.2717 |
0.5015 |
0.0458 |
pa-5 |
0.3736 |
0.5220 |
0.3687 |
0.5156 |
0.0439 |