BioASQ Participants Area
Task 9b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.2657 | 0.4722 | 0.2615 | 0.4352 | 0.0452 |
| AUEB-System2 | 0.2537 | 0.4646 | 0.2519 | 0.4461 | 0.0423 |
| AUEB-System3 | 0.2647 | 0.4713 | 0.2605 | 0.4372 | 0.0453 |
| bio-answerfinder | 0.4468 | 0.3782 | 0.3486 | 0.3607 | 0.0185 |
| bioinfo-0 | 0.2230 | 0.4188 | 0.2232 | 0.3781 | 0.0172 |
| bioinfo-1 | 0.2250 | 0.4331 | 0.2270 | 0.3891 | 0.0196 |
| pa-1 | 0.2347 | 0.4368 | 0.2349 | 0.4194 | 0.0264 |
| pa-2 | 0.2407 | 0.4417 | 0.2402 | 0.4261 | 0.0289 |
| RYGH-1 | 0.2910 | 0.4969 | 0.2828 | 0.4764 | 0.0529 |
| RYGH | 0.2860 | 0.4975 | 0.2797 | 0.4561 | 0.0474 |
| RYGH-4 | 0.2740 | 0.4944 | 0.2728 | 0.4372 | 0.0455 |
| RYGH-5 | 0.2700 | 0.4859 | 0.2683 | 0.4292 | 0.0440 |
| bioinfo-2 | 0.2380 | 0.4729 | 0.2429 | 0.4405 | 0.0366 |
| RYGH-3 | 0.2490 | 0.4605 | 0.2507 | 0.4017 | 0.0371 |
| bioinfo-3 | 0.2380 | 0.4629 | 0.2418 | 0.4403 | 0.0363 |
| bioinfo-4 | 0.2500 | 0.4636 | 0.2499 | 0.4426 | 0.0315 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.2187 | 0.2197 | 0.1746 | 0.3712 | 0.0199 |
| AUEB-System2 | 0.2142 | 0.2000 | 0.1645 | 0.3823 | 0.0185 |
| AUEB-System3 | 0.2200 | 0.2140 | 0.1743 | 0.3649 | 0.0199 |
| bio-answerfinder | 0.2901 | 0.2636 | 0.2258 | 0.6476 | 0.0111 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| pa-1 | 0.2569 | 0.2547 | 0.2227 | 0.9789 | 0.0103 |
| pa-2 | 0.2576 | 0.2554 | 0.2234 | 0.9791 | 0.0112 |
| RYGH-1 | 0.2897 | 0.2617 | 0.2236 | 0.3979 | 0.0218 |
| RYGH | 0.2867 | 0.2584 | 0.2202 | 0.3910 | 0.0218 |
| RYGH-4 | 0.2779 | 0.2665 | 0.2183 | 0.4103 | 0.0246 |
| RYGH-5 | 0.2846 | 0.2789 | 0.2271 | 0.4150 | 0.0256 |
| bioinfo-2 | - | - | - | - | - |
| RYGH-3 | 0.2716 | 0.2581 | 0.2140 | 0.4045 | 0.0240 |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
Test batch 2
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.2610 | 0.5298 | 0.2659 | 0.4497 | 0.0642 |
| AUEB-System2 | 0.2600 | 0.5268 | 0.2660 | 0.4514 | 0.0580 |
| AUEB-System3 | 0.2610 | 0.5307 | 0.2660 | 0.4528 | 0.0645 |
| Olive-DocFetchV1 | 0.1790 | 0.3881 | 0.1872 | 0.3099 | 0.0175 |
| Olive-DocFetchV2 | 0.2500 | 0.3317 | 0.2120 | 0.2790 | 0.0096 |
| OliveSentSim1 | 0.1110 | 0.2386 | 0.1092 | 0.1052 | 0.0022 |
| OliveSentSim2 | 0.1430 | 0.3119 | 0.1466 | 0.2084 | 0.0076 |
| OliveAnsScore1 | 0.1360 | 0.2697 | 0.1351 | 0.1387 | 0.0042 |
| OliveAnsScore2 | 0.1570 | 0.3331 | 0.1597 | 0.1996 | 0.0086 |
| bio-answerfinder | 0.4319 | 0.4531 | 0.3797 | 0.4250 | 0.0280 |
| bioinfo-0 | 0.2370 | 0.5618 | 0.2519 | 0.4316 | 0.0795 |
| bioinfo-1 | 0.2340 | 0.5412 | 0.2471 | 0.4459 | 0.0721 |
| bioinfo-2 | 0.2480 | 0.5238 | 0.2495 | 0.4709 | 0.0620 |
| Deep ML methods for | 0.2314 | 0.4514 | 0.2312 | 0.3753 | 0.0305 |
| MindLab QA System | 0.2314 | 0.4514 | 0.2312 | 0.3753 | 0.0305 |
| pa-1 | 0.2850 | 0.5304 | 0.2814 | 0.4947 | 0.0641 |
| pa-2 | 0.2860 | 0.5338 | 0.2829 | 0.4892 | 0.0635 |
| MindLab QA System ++ | 0.2314 | 0.4514 | 0.2312 | 0.3753 | 0.0305 |
| pa-3 | 0.2740 | 0.5190 | 0.2710 | 0.4750 | 0.0592 |
| pa-4 | 0.2780 | 0.5265 | 0.2756 | 0.4824 | 0.0605 |
| MindLab QA Reloaded | 0.2314 | 0.4514 | 0.2312 | 0.3753 | 0.0305 |
| MindLab Red Lions++ | 0.2314 | 0.4514 | 0.2312 | 0.3753 | 0.0305 |
| RYGH-1 | 0.3020 | 0.6038 | 0.2973 | 0.5263 | 0.1250 |
| RYGH-5 | 0.3050 | 0.5994 | 0.3005 | 0.5423 | 0.1194 |
| RYGH-4 | 0.3080 | 0.6094 | 0.3031 | 0.5350 | 0.1283 |
| bioinfo-3 | 0.2460 | 0.5112 | 0.2450 | 0.4755 | 0.0521 |
| bioinfo-4 | 0.2580 | 0.5587 | 0.2649 | 0.4913 | 0.0852 |
| RYGH-3 | 0.2990 | 0.5802 | 0.2926 | 0.5138 | 0.0940 |
| RYGH | 0.2980 | 0.5903 | 0.2933 | 0.5240 | 0.1141 |
| pa-5 | 0.3608 | 0.5295 | 0.3479 | 0.4973 | 0.0479 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.2071 | 0.2251 | 0.1658 | 0.3777 | 0.0163 |
| AUEB-System2 | 0.1948 | 0.2052 | 0.1553 | 0.3418 | 0.0118 |
| AUEB-System3 | 0.2041 | 0.2222 | 0.1688 | 0.3761 | 0.0188 |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bio-answerfinder | 0.3011 | 0.2769 | 0.2318 | 0.6272 | 0.0168 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| Deep ML methods for | 0.1588 | 0.1701 | 0.1303 | 0.1915 | 0.0028 |
| MindLab QA System | 0.1573 | 0.1672 | 0.1271 | 0.1883 | 0.0029 |
| pa-1 | 0.2292 | 0.2667 | 0.2012 | 0.9002 | 0.0119 |
| pa-2 | 0.2279 | 0.2455 | 0.1966 | 0.8729 | 0.0089 |
| MindLab QA System ++ | 0.1309 | 0.0935 | 0.0891 | 0.1582 | 0.0006 |
| pa-3 | 0.2390 | 0.2759 | 0.2123 | 0.9492 | 0.0160 |
| pa-4 | 0.2421 | 0.2781 | 0.2144 | 0.9473 | 0.0159 |
| MindLab QA Reloaded | 0.1309 | 0.0935 | 0.0891 | 0.1582 | 0.0006 |
| MindLab Red Lions++ | 0.1604 | 0.1737 | 0.1295 | 0.1912 | 0.0033 |
| RYGH-1 | 0.3090 | 0.3726 | 0.2521 | 0.5677 | 0.0678 |
| RYGH-5 | 0.3092 | 0.3642 | 0.2525 | 0.5403 | 0.0604 |
| RYGH-4 | 0.3132 | 0.3766 | 0.2574 | 0.5626 | 0.0685 |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| RYGH-3 | 0.3078 | 0.3481 | 0.2482 | 0.5506 | 0.0462 |
| RYGH | 0.3141 | 0.3730 | 0.2533 | 0.5333 | 0.0651 |
| pa-5 | 0.2357 | 0.2571 | 0.2062 | 0.8779 | 0.0102 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| pa-5 | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| MindLab Red Lions++ | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| pa-5 | - | - | - | - | - |
Test batch 3
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.2347 | 0.5229 | 0.2467 | 0.4477 | 0.0485 |
| AUEB-System2 | 0.2306 | 0.5185 | 0.2426 | 0.4401 | 0.0407 |
| AUEB-System3 | 0.2368 | 0.5238 | 0.2479 | 0.4506 | 0.0491 |
| bio-answerfinder | 0.4169 | 0.4611 | 0.3718 | 0.4163 | 0.0235 |
| Olive-DocFetchV1 | 0.2031 | 0.4715 | 0.2156 | 0.3877 | 0.0283 |
| Olive-DocFetchV2 | 0.2742 | 0.3781 | 0.2364 | 0.3426 | 0.0142 |
| Olive-DocFetchV3 | 0.1849 | 0.4619 | 0.1997 | 0.3938 | 0.0203 |
| OliveAnsScore1 | 0.1010 | 0.2063 | 0.1008 | 0.1208 | 0.0012 |
| OliveSentSim1 | 0.1062 | 0.1925 | 0.1021 | 0.0834 | 0.0012 |
| OliveSentSim2 | 0.1062 | 0.2068 | 0.1051 | 0.1469 | 0.0019 |
| OliveAnsScore2 | 0.1186 | 0.2840 | 0.1263 | 0.1789 | 0.0031 |
| bioinfo-0 | 0.2278 | 0.4986 | 0.2373 | 0.4332 | 0.0332 |
| bioinfo-1 | 0.2309 | 0.5042 | 0.2420 | 0.4392 | 0.0337 |
| bioinfo-2 | 0.2577 | 0.5340 | 0.2663 | 0.4717 | 0.0597 |
| bioinfo-3 | 0.2588 | 0.5347 | 0.2666 | 0.4900 | 0.0607 |
| bioinfo-4 | 0.2649 | 0.5347 | 0.2682 | 0.4912 | 0.0504 |
| pa-1 | 0.2505 | 0.5206 | 0.2547 | 0.4935 | 0.0403 |
| pa-2 | 0.2464 | 0.5259 | 0.2518 | 0.4891 | 0.0400 |
| pa-3 | 0.2505 | 0.5124 | 0.2538 | 0.4741 | 0.0382 |
| pa-4 | 0.2423 | 0.5153 | 0.2479 | 0.4673 | 0.0408 |
| Deep ML methods for | 0.2085 | 0.4430 | 0.2153 | 0.3624 | 0.0253 |
| MindLab QA System | 0.2085 | 0.4430 | 0.2153 | 0.3624 | 0.0253 |
| MindLab QA System ++ | 0.2085 | 0.4430 | 0.2153 | 0.3624 | 0.0253 |
| MindLab QA Reloaded | 0.2085 | 0.4430 | 0.2153 | 0.3624 | 0.0253 |
| RYGH-1 | 0.2876 | 0.5746 | 0.2902 | 0.5265 | 0.0733 |
| RYGH-3 | 0.2876 | 0.5737 | 0.2895 | 0.5271 | 0.0678 |
| RYGH-4 | 0.2948 | 0.5762 | 0.2942 | 0.5273 | 0.0738 |
| RYGH-5 | 0.2856 | 0.5635 | 0.2870 | 0.5241 | 0.0615 |
| RYGH | 0.2918 | 0.5803 | 0.2943 | 0.5267 | 0.0737 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | 0.1940 | 0.2247 | 0.1668 | 0.3734 | 0.0236 |
| AUEB-System2 | 0.1780 | 0.1861 | 0.1461 | 0.3217 | 0.0121 |
| AUEB-System3 | 0.1995 | 0.2212 | 0.1689 | 0.3473 | 0.0225 |
| bio-answerfinder | 0.2930 | 0.3264 | 0.2614 | 0.6840 | 0.0217 |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| pa-1 | 0.2423 | 0.3025 | 0.2391 | 0.9200 | 0.0114 |
| pa-2 | 0.2383 | 0.2947 | 0.2352 | 0.9022 | 0.0092 |
| pa-3 | 0.2308 | 0.2830 | 0.2226 | 0.8669 | 0.0081 |
| pa-4 | 0.2321 | 0.2774 | 0.2217 | 0.8528 | 0.0072 |
| Deep ML methods for | 0.1413 | 0.1490 | 0.1237 | 0.1390 | 0.0018 |
| MindLab QA System | 0.1024 | 0.0785 | 0.0738 | 0.0675 | 0.0004 |
| MindLab QA System ++ | 0.1024 | 0.0785 | 0.0738 | 0.0675 | 0.0004 |
| MindLab QA Reloaded | 0.1413 | 0.1490 | 0.1237 | 0.1390 | 0.0018 |
| RYGH-1 | 0.3175 | 0.3629 | 0.2772 | 0.5646 | 0.0465 |
| RYGH-3 | 0.3141 | 0.3557 | 0.2739 | 0.5498 | 0.0386 |
| RYGH-4 | 0.3230 | 0.3566 | 0.2799 | 0.5347 | 0.0390 |
| RYGH-5 | 0.3132 | 0.3467 | 0.2738 | 0.5186 | 0.0375 |
| RYGH | 0.3174 | 0.3712 | 0.2788 | 0.5606 | 0.0562 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| Deep ML methods for | - | - | - | - | - |
| MindLab QA System | - | - | - | - | - |
| MindLab QA System ++ | - | - | - | - | - |
| MindLab QA Reloaded | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
Test batch 4
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | 0.2190 | 0.4657 | 0.2322 | 0.4141 | 0.0257 |
| Olive-DocFetchV2 | 0.3020 | 0.4022 | 0.2634 | 0.3681 | 0.0161 |
| Olive-DocFetchV3 | 0.1980 | 0.4645 | 0.2153 | 0.4011 | 0.0308 |
| AUEB-System1 | 0.2500 | 0.5271 | 0.2708 | 0.4453 | 0.0408 |
| AUEB-System2 | 0.2460 | 0.5188 | 0.2661 | 0.4486 | 0.0411 |
| AUEB-System3 | 0.2510 | 0.5277 | 0.2716 | 0.4444 | 0.0408 |
| bio-answerfinder | 0.4179 | 0.4090 | 0.3534 | 0.3841 | 0.0208 |
| bio-answerfinder-2 | 0.3972 | 0.4152 | 0.3549 | 0.3936 | 0.0214 |
| pa-1 | 0.2670 | 0.4877 | 0.2720 | 0.4776 | 0.0414 |
| pa-2 | 0.2710 | 0.4871 | 0.2743 | 0.4751 | 0.0377 |
| pa-3 | 0.2643 | 0.4989 | 0.2729 | 0.4665 | 0.0444 |
| pa-4 | 0.2683 | 0.5039 | 0.2768 | 0.4649 | 0.0445 |
| bioinfo-0 | 0.2420 | 0.5113 | 0.2540 | 0.4601 | 0.0341 |
| bioinfo-1 | 0.2430 | 0.5056 | 0.2550 | 0.4556 | 0.0305 |
| OliveAnsScore1 | 0.1630 | 0.3392 | 0.1718 | 0.1932 | 0.0066 |
| OliveAnsScore2 | 0.1810 | 0.3922 | 0.1951 | 0.2512 | 0.0110 |
| OliveSentSim1 | 0.1140 | 0.2010 | 0.1143 | 0.0799 | 0.0013 |
| OliveSentSim2 | 0.1500 | 0.3096 | 0.1548 | 0.2083 | 0.0061 |
| RYGH-1 | 0.2650 | 0.5386 | 0.2732 | 0.4966 | 0.0479 |
| RYGH-3 | 0.2680 | 0.5458 | 0.2784 | 0.5222 | 0.0599 |
| RYGH-4 | 0.2610 | 0.5323 | 0.2682 | 0.5106 | 0.0533 |
| RYGH-5 | 0.2690 | 0.5434 | 0.2783 | 0.5154 | 0.0547 |
| RYGH | 0.2630 | 0.5347 | 0.2700 | 0.5113 | 0.0528 |
| bioinfo-2 | 0.2610 | 0.5325 | 0.2710 | 0.5127 | 0.0463 |
| bioinfo-3 | 0.2590 | 0.5439 | 0.2717 | 0.5015 | 0.0458 |
| bioinfo-4 | 0.2590 | 0.5439 | 0.2717 | 0.5015 | 0.0458 |
| pa-5 | 0.3736 | 0.5220 | 0.3687 | 0.5156 | 0.0439 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | 0.1994 | 0.2213 | 0.1648 | 0.3607 | 0.0132 |
| AUEB-System2 | 0.1847 | 0.2092 | 0.1477 | 0.3626 | 0.0136 |
| AUEB-System3 | 0.2020 | 0.2236 | 0.1644 | 0.3426 | 0.0144 |
| bio-answerfinder | 0.2602 | 0.2639 | 0.2185 | 0.7283 | 0.0080 |
| bio-answerfinder-2 | 0.2675 | 0.2667 | 0.2227 | 0.7247 | 0.0108 |
| pa-1 | 0.2306 | 0.2770 | 0.2018 | 0.8857 | 0.0135 |
| pa-2 | 0.2305 | 0.2739 | 0.2009 | 0.8711 | 0.0116 |
| pa-3 | 0.2302 | 0.2733 | 0.2050 | 0.8640 | 0.0135 |
| pa-4 | 0.2331 | 0.2736 | 0.2056 | 0.8639 | 0.0132 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | 0.1985 | 0.2183 | 0.1568 | 0.3310 | 0.0124 |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| RYGH-1 | 0.2908 | 0.3554 | 0.2564 | 0.5890 | 0.0338 |
| RYGH-3 | 0.2950 | 0.3505 | 0.2576 | 0.5623 | 0.0380 |
| RYGH-4 | 0.2922 | 0.3511 | 0.2547 | 0.5699 | 0.0300 |
| RYGH-5 | 0.2982 | 0.3521 | 0.2598 | 0.5673 | 0.0361 |
| RYGH | 0.2897 | 0.3493 | 0.2529 | 0.5689 | 0.0324 |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | 0.2466 | 0.2713 | 0.2017 | 0.4385 | 0.0199 |
| bioinfo-4 | 0.2166 | 0.2450 | 0.1884 | 0.5973 | 0.0100 |
| pa-5 | 0.2622 | 0.3136 | 0.2338 | 0.9989 | 0.0170 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bio-answerfinder-2 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| pa-5 | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bio-answerfinder-2 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| pa-5 | - | - | - | - | - |
Test batch 5
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | 0.2313 | 0.4500 | 0.2328 | 0.3965 | 0.0405 |
| Olive-DocFetchV2 | 0.2313 | 0.4500 | 0.2328 | 0.3965 | 0.0405 |
| Olive-DocFetchV3 | 0.2008 | 0.4229 | 0.2066 | 0.3710 | 0.0215 |
| AUEB-System1 | 0.2803 | 0.4740 | 0.2743 | 0.4318 | 0.0380 |
| AUEB-System2 | 0.2763 | 0.4755 | 0.2723 | 0.4340 | 0.0422 |
| AUEB-System3 | 0.2723 | 0.4702 | 0.2680 | 0.4147 | 0.0382 |
| AUEB-System4 | 0.2633 | 0.4546 | 0.2597 | 0.4171 | 0.0381 |
| AUEB-System5 | 0.2733 | 0.4716 | 0.2692 | 0.4240 | 0.0390 |
| OliveAnsScore1 | 0.1810 | 0.2978 | 0.1677 | 0.1657 | 0.0069 |
| OliveAnsScore2 | 0.2060 | 0.3861 | 0.2009 | 0.2408 | 0.0154 |
| OliveSentSim1 | 0.1380 | 0.2190 | 0.1270 | 0.0975 | 0.0020 |
| OliveSentSim2 | 0.1720 | 0.3111 | 0.1665 | 0.2219 | 0.0077 |
| bio-answerfinder | 0.4295 | 0.3890 | 0.3635 | 0.3670 | 0.0157 |
| bio-answerfinder-2 | 0.3969 | 0.3750 | 0.3315 | 0.3415 | 0.0151 |
| bioinfo-0 | 0.2460 | 0.4617 | 0.2392 | 0.3964 | 0.0429 |
| bioinfo-1 | 0.2450 | 0.4610 | 0.2384 | 0.3995 | 0.0480 |
| pa-1 | 0.2483 | 0.4559 | 0.2428 | 0.4597 | 0.0490 |
| pa-2 | 0.2433 | 0.4521 | 0.2378 | 0.4517 | 0.0368 |
| pa-3 | 0.2483 | 0.4559 | 0.2428 | 0.4383 | 0.0484 |
| pa-4 | 0.2433 | 0.4521 | 0.2378 | 0.4310 | 0.0359 |
| RYGH-1 | 0.3095 | 0.5367 | 0.2968 | 0.5038 | 0.0820 |
| RYGH | 0.3175 | 0.5435 | 0.3031 | 0.5083 | 0.0989 |
| RYGH-3 | 0.3095 | 0.5352 | 0.2954 | 0.5017 | 0.0758 |
| RYGH-4 | 0.3208 | 0.5429 | 0.3052 | 0.5175 | 0.0856 |
| RYGH-5 | 0.3075 | 0.5331 | 0.2936 | 0.4946 | 0.0730 |
| bioinfo-2 | 0.2770 | 0.5294 | 0.2742 | 0.4701 | 0.0796 |
| bioinfo-4 | 0.2610 | 0.4862 | 0.2554 | 0.4480 | 0.0532 |
| bioinfo-3 | 0.2690 | 0.5069 | 0.2645 | 0.4529 | 0.0603 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | 0.2216 | 0.2236 | 0.1743 | 0.3536 | 0.0136 |
| AUEB-System2 | 0.2102 | 0.1838 | 0.1508 | 0.3189 | 0.0075 |
| AUEB-System3 | 0.2131 | 0.2160 | 0.1679 | 0.3496 | 0.0107 |
| AUEB-System4 | 0.2036 | 0.1711 | 0.1430 | 0.3110 | 0.0053 |
| AUEB-System5 | 0.2188 | 0.2206 | 0.1690 | 0.3261 | 0.0142 |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| bio-answerfinder | 0.2929 | 0.2946 | 0.2386 | 0.5821 | 0.0153 |
| bio-answerfinder-2 | 0.2816 | 0.2689 | 0.2184 | 0.4775 | 0.0127 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | 0.2168 | 0.2325 | 0.1889 | 0.7126 | 0.0040 |
| pa-1 | 0.2098 | 0.2743 | 0.1971 | 0.7795 | 0.0118 |
| pa-2 | 0.2015 | 0.2705 | 0.1919 | 0.7653 | 0.0091 |
| pa-3 | 0.1902 | 0.2477 | 0.1822 | 0.6835 | 0.0086 |
| pa-4 | 0.1848 | 0.2432 | 0.1775 | 0.6691 | 0.0060 |
| RYGH-1 | 0.3183 | 0.3535 | 0.2565 | 0.5272 | 0.0670 |
| RYGH | 0.3124 | 0.3484 | 0.2514 | 0.5074 | 0.0681 |
| RYGH-3 | 0.3126 | 0.3530 | 0.2535 | 0.5271 | 0.0625 |
| RYGH-4 | 0.3134 | 0.3517 | 0.2532 | 0.5123 | 0.0610 |
| RYGH-5 | 0.3094 | 0.3576 | 0.2551 | 0.5336 | 0.0632 |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-4 | 0.2401 | 0.2285 | 0.1910 | 0.5632 | 0.0046 |
| bioinfo-3 | 0.2411 | 0.2541 | 0.2077 | 0.7441 | 0.0105 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| AUEB-System4 | - | - | - | - | - |
| AUEB-System5 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bio-answerfinder-2 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Olive-DocFetchV1 | - | - | - | - | - |
| Olive-DocFetchV2 | - | - | - | - | - |
| Olive-DocFetchV3 | - | - | - | - | - |
| AUEB-System1 | - | - | - | - | - |
| AUEB-System2 | - | - | - | - | - |
| AUEB-System3 | - | - | - | - | - |
| AUEB-System4 | - | - | - | - | - |
| AUEB-System5 | - | - | - | - | - |
| OliveAnsScore1 | - | - | - | - | - |
| OliveAnsScore2 | - | - | - | - | - |
| OliveSentSim1 | - | - | - | - | - |
| OliveSentSim2 | - | - | - | - | - |
| bio-answerfinder | - | - | - | - | - |
| bio-answerfinder-2 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| pa-1 | - | - | - | - | - |
| pa-2 | - | - | - | - | - |
| pa-3 | - | - | - | - | - |
| pa-4 | - | - | - | - | - |
| RYGH-1 | - | - | - | - | - |
| RYGH | - | - | - | - | - |
| RYGH-3 | - | - | - | - | - |
| RYGH-4 | - | - | - | - | - |
| RYGH-5 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |