BioASQ Participants Area
Task 10b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
The basic end-to-end |
0.1775 |
0.5185 |
0.2347 |
0.3849 |
0.0105 |
AUEB-System1 |
0.0822 |
0.3694 |
0.1241 |
0.2798 |
0.0018 |
ELECTROLBERT-0 |
0.0456 |
0.1569 |
0.0622 |
0.1121 |
0.0001 |
bio-answerfinder |
0.2709 |
0.4005 |
0.2986 |
0.3541 |
0.0029 |
gsl_zs_hybrid |
0.1244 |
0.5419 |
0.1827 |
0.4154 |
0.0234 |
bio-answerfinder-2 |
0.1296 |
0.4511 |
0.1806 |
0.3434 |
0.0070 |
gsl_zs_rrf1 |
0.1300 |
0.5592 |
0.1911 |
0.4292 |
0.0223 |
gsl_zs_rrf2 |
0.1278 |
0.5463 |
0.1877 |
0.4280 |
0.0202 |
gsl_zs_nn |
0.1156 |
0.5193 |
0.1705 |
0.4126 |
0.0166 |
gsl_zs_rrf3 |
0.1244 |
0.5113 |
0.1814 |
0.3858 |
0.0148 |
Basic e2e mid speed |
0.1697 |
0.4977 |
0.2240 |
0.3771 |
0.0077 |
RYGH-1 |
0.1330 |
0.5730 |
0.1958 |
0.4536 |
0.0262 |
RYGH |
0.1430 |
0.5819 |
0.2065 |
0.4567 |
0.0277 |
RYGH-3 |
0.1421 |
0.6055 |
0.2080 |
0.4770 |
0.0438 |
RYGH-4 |
0.1407 |
0.6058 |
0.2065 |
0.4805 |
0.0440 |
AUEB-System2 |
0.1022 |
0.4373 |
0.1526 |
0.3329 |
0.0035 |
simple baseline solr |
0.0011 |
0.0009 |
0.0010 |
0.0003 |
0.0000 |
bioinfo-0 |
0.1267 |
0.5521 |
0.1870 |
0.4098 |
0.0239 |
bioinfo-1 |
0.1244 |
0.5490 |
0.1845 |
0.4130 |
0.0215 |
bioinfo-2 |
0.1256 |
0.5512 |
0.1859 |
0.4059 |
0.0234 |
bioinfo-3 |
0.1233 |
0.5417 |
0.1824 |
0.4042 |
0.0202 |
ELECTROLBERT-1 |
0.0422 |
0.1534 |
0.0582 |
0.1082 |
0.0001 |
ELECTROLBERT-2 |
0.0422 |
0.1534 |
0.0582 |
0.1082 |
0.0001 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
The basic end-to-end |
0.0911 |
0.0010 |
0.0019 |
0.1808 |
0.0006 |
AUEB-System1 |
0.0693 |
0.2117 |
0.0968 |
0.2495 |
0.0010 |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
bio-answerfinder |
0.1108 |
0.2750 |
0.1401 |
0.4927 |
0.0014 |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
0.0764 |
0.2276 |
0.1027 |
0.2431 |
0.0020 |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Basic e2e mid speed |
0.0888 |
0.0009 |
0.0018 |
0.1798 |
0.0005 |
RYGH-1 |
0.1183 |
0.3762 |
0.1634 |
0.4021 |
0.0116 |
RYGH |
0.1225 |
0.3883 |
0.1675 |
0.4226 |
0.0175 |
RYGH-3 |
0.1230 |
0.4299 |
0.1733 |
0.4375 |
0.0273 |
RYGH-4 |
0.1241 |
0.4244 |
0.1736 |
0.4299 |
0.0269 |
AUEB-System2 |
0.0790 |
0.2575 |
0.1117 |
0.2877 |
0.0019 |
simple baseline solr |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
The basic end-to-end |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
simple baseline solr |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
The basic end-to-end |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
simple baseline solr |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
simple baseline solr |
0.0456 |
0.2403 |
0.0715 |
0.1409 |
0.0003 |
bio-answerfinder |
0.2015 |
0.3793 |
0.2378 |
0.2989 |
0.0027 |
bio-answerfinder-2 |
0.1266 |
0.3668 |
0.1678 |
0.2751 |
0.0026 |
bio-answerfinder-3 |
0.1523 |
0.2804 |
0.1750 |
0.2104 |
0.0009 |
bio-answerfinder-4 |
0.1223 |
0.3502 |
0.1610 |
0.2661 |
0.0023 |
ELECTROLBERT-0 |
0.0456 |
0.2521 |
0.0716 |
0.1632 |
0.0003 |
AUEB-System1 |
0.0633 |
0.3195 |
0.0980 |
0.2276 |
0.0009 |
AUEB-System2 |
0.0900 |
0.4474 |
0.1389 |
0.3147 |
0.0048 |
The basic end-to-end |
0.1203 |
0.4353 |
0.1669 |
0.2904 |
0.0042 |
Basic e2e mid speed |
0.1192 |
0.4350 |
0.1657 |
0.2892 |
0.0045 |
bioinfo-0 |
0.1022 |
0.4572 |
0.1543 |
0.3186 |
0.0106 |
bioinfo-1 |
0.1022 |
0.4572 |
0.1543 |
0.3223 |
0.0104 |
gsl_zs_rrf1 |
0.1044 |
0.4854 |
0.1585 |
0.3572 |
0.0104 |
gsl_zs_hybrid |
0.1011 |
0.4877 |
0.1547 |
0.3666 |
0.0097 |
gsl_zs_nn |
0.0967 |
0.4808 |
0.1493 |
0.3453 |
0.0095 |
gsl_zs_rrf3 |
0.0967 |
0.4601 |
0.1477 |
0.3629 |
0.0065 |
gsl_zs_rrf2 |
0.0978 |
0.4824 |
0.1503 |
0.3647 |
0.0100 |
RYGH-5 |
0.0933 |
0.4470 |
0.1430 |
0.3427 |
0.0077 |
RYGH-1 |
0.1235 |
0.5559 |
0.1862 |
0.3977 |
0.0166 |
RYGH-3 |
0.1179 |
0.5275 |
0.1779 |
0.3855 |
0.0140 |
RYGH |
0.1135 |
0.5194 |
0.1720 |
0.3795 |
0.0135 |
RYGH-4 |
0.1167 |
0.5266 |
0.1764 |
0.3839 |
0.0117 |
ELECTROLBERT-1 |
0.0400 |
0.2190 |
0.0626 |
0.1496 |
0.0002 |
ELECTROLBERT-2 |
0.0356 |
0.1882 |
0.0556 |
0.1075 |
0.0002 |
bioinfo-2 |
0.1033 |
0.4599 |
0.1558 |
0.3339 |
0.0099 |
bioinfo-3 |
0.1011 |
0.4649 |
0.1532 |
0.3199 |
0.0098 |
bioinfo-4 |
0.0878 |
0.4009 |
0.1331 |
0.2813 |
0.0041 |
Deep ML methods for |
0.0800 |
0.4104 |
0.1242 |
0.2710 |
0.0034 |
MindLab QA System |
0.0800 |
0.4104 |
0.1242 |
0.2710 |
0.0034 |
ELECTROLBERT-3 |
0.0378 |
0.2180 |
0.0606 |
0.1197 |
0.0002 |
MindLab Red Lions++ |
0.0800 |
0.4104 |
0.1242 |
0.2710 |
0.0034 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
simple baseline solr |
- |
- |
- |
- |
- |
bio-answerfinder |
0.0977 |
0.2238 |
0.1165 |
0.3712 |
0.0010 |
bio-answerfinder-2 |
0.0822 |
0.2311 |
0.1062 |
0.2947 |
0.0013 |
bio-answerfinder-3 |
0.0846 |
0.1820 |
0.0987 |
0.3019 |
0.0005 |
bio-answerfinder-4 |
0.0782 |
0.2110 |
0.1001 |
0.2791 |
0.0011 |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
AUEB-System1 |
0.0559 |
0.1352 |
0.0695 |
0.1914 |
0.0003 |
AUEB-System2 |
0.0672 |
0.1846 |
0.0850 |
0.2251 |
0.0009 |
The basic end-to-end |
0.0555 |
0.0004 |
0.0009 |
0.0795 |
0.0001 |
Basic e2e mid speed |
0.0564 |
0.0004 |
0.0009 |
0.0787 |
0.0001 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-5 |
0.1002 |
0.3063 |
0.1404 |
0.6069 |
0.0023 |
RYGH-1 |
0.1147 |
0.3363 |
0.1519 |
0.4364 |
0.0050 |
RYGH-3 |
0.1078 |
0.3161 |
0.1427 |
0.4000 |
0.0043 |
RYGH |
0.1073 |
0.3175 |
0.1433 |
0.4234 |
0.0048 |
RYGH-4 |
0.1098 |
0.3196 |
0.1455 |
0.4028 |
0.0041 |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.0645 |
0.1615 |
0.0820 |
0.3043 |
0.0001 |
MindLab QA System |
0.0645 |
0.1615 |
0.0820 |
0.3043 |
0.0001 |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
0.0645 |
0.1615 |
0.0820 |
0.3043 |
0.0001 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
simple baseline solr |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
simple baseline solr |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
MindLab Red Lions++ |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Basic e2e mid speed |
0.1794 |
0.4891 |
0.2357 |
0.3733 |
0.0075 |
bio-answerfinder |
0.2584 |
0.4470 |
0.2790 |
0.3581 |
0.0041 |
bio-answerfinder-2 |
0.1423 |
0.4572 |
0.1877 |
0.3415 |
0.0052 |
ELECTROLBERT-0 |
0.0600 |
0.3549 |
0.0959 |
0.2376 |
0.0011 |
ELECTROLBERT-1 |
0.0489 |
0.3043 |
0.0794 |
0.2091 |
0.0005 |
AUEB-System1 |
0.0956 |
0.4506 |
0.1394 |
0.3227 |
0.0036 |
AUEB-System2 |
0.1111 |
0.5243 |
0.1628 |
0.4227 |
0.0104 |
bio-answerfinder-3 |
0.2241 |
0.4100 |
0.2439 |
0.3420 |
0.0029 |
bio-answerfinder-4 |
0.1435 |
0.4683 |
0.1905 |
0.3474 |
0.0058 |
Deep ML methods for |
0.0633 |
0.3424 |
0.0993 |
0.2068 |
0.0013 |
gsl_zs_hybrid |
0.1122 |
0.5676 |
0.1729 |
0.4256 |
0.0172 |
gsl_zs_rrf1 |
0.1144 |
0.5763 |
0.1764 |
0.4304 |
0.0175 |
gsl_zs_nn |
0.1033 |
0.5209 |
0.1591 |
0.4050 |
0.0100 |
gsl_zs_rrf3 |
0.1033 |
0.5331 |
0.1600 |
0.4087 |
0.0111 |
gsl_zs_rrf2 |
0.0922 |
0.4947 |
0.1439 |
0.3834 |
0.0077 |
RYGH-1 |
0.1280 |
0.6196 |
0.1965 |
0.4698 |
0.0234 |
RYGH |
0.1267 |
0.6287 |
0.1956 |
0.4903 |
0.0200 |
RYGH-3 |
0.1244 |
0.6123 |
0.1916 |
0.4849 |
0.0217 |
RYGH-4 |
0.1251 |
0.6234 |
0.1931 |
0.4872 |
0.0242 |
RYGH-5 |
0.1256 |
0.6333 |
0.1946 |
0.5063 |
0.0256 |
bioinfo-0 |
0.1278 |
0.6302 |
0.1959 |
0.4352 |
0.0282 |
bioinfo-1 |
0.1278 |
0.6302 |
0.1959 |
0.4290 |
0.0260 |
bioinfo-2 |
0.1278 |
0.6302 |
0.1959 |
0.4290 |
0.0260 |
bioinfo-3 |
0.1200 |
0.5940 |
0.1843 |
0.4174 |
0.0186 |
bioinfo-4 |
0.1289 |
0.6321 |
0.1973 |
0.4279 |
0.0256 |
ELECTROLBERT-2 |
0.0889 |
0.4619 |
0.1377 |
0.3209 |
0.0044 |
ELECTROLBERT-3 |
0.0767 |
0.4090 |
0.1197 |
0.2854 |
0.0019 |
The basic end-to-end |
0.1796 |
0.4891 |
0.2360 |
0.3753 |
0.0076 |
BioNIR Prepro-mid |
0.1840 |
0.4851 |
0.2384 |
0.3639 |
0.0074 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Basic e2e mid speed |
0.1170 |
0.3329 |
0.1559 |
0.4463 |
0.0039 |
bio-answerfinder |
0.1270 |
0.3252 |
0.1617 |
0.5445 |
0.0029 |
bio-answerfinder-2 |
0.0984 |
0.2726 |
0.1300 |
0.3249 |
0.0027 |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
AUEB-System1 |
0.0776 |
0.2382 |
0.1062 |
0.3266 |
0.0019 |
AUEB-System2 |
0.0885 |
0.2949 |
0.1230 |
0.3953 |
0.0048 |
bio-answerfinder-3 |
0.1171 |
0.2952 |
0.1506 |
0.4699 |
0.0023 |
bio-answerfinder-4 |
0.0986 |
0.2833 |
0.1309 |
0.3306 |
0.0030 |
Deep ML methods for |
0.0238 |
0.0970 |
0.0344 |
0.0966 |
0.0001 |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-1 |
0.1227 |
0.4234 |
0.1742 |
0.4935 |
0.0106 |
RYGH |
0.1230 |
0.4420 |
0.1760 |
0.4946 |
0.0115 |
RYGH-3 |
0.1212 |
0.4202 |
0.1721 |
0.4792 |
0.0097 |
RYGH-4 |
0.1229 |
0.4328 |
0.1746 |
0.4858 |
0.0106 |
RYGH-5 |
0.1546 |
0.4721 |
0.2125 |
0.8305 |
0.0143 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
The basic end-to-end |
0.1163 |
0.3329 |
0.1554 |
0.4486 |
0.0039 |
BioNIR Prepro-mid |
0.1144 |
0.3208 |
0.1503 |
0.4309 |
0.0040 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
BioNIR Prepro-mid |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
ELECTROLBERT-0 |
- |
- |
- |
- |
- |
ELECTROLBERT-1 |
- |
- |
- |
- |
- |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
ELECTROLBERT-3 |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
BioNIR Prepro-mid |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.0800 |
0.3885 |
0.1238 |
0.2592 |
0.0017 |
AUEB-System2 |
0.0889 |
0.4070 |
0.1362 |
0.3021 |
0.0026 |
bio-answerfinder |
0.2500 |
0.4180 |
0.2718 |
0.3217 |
0.0064 |
bio-answerfinder-2 |
0.1360 |
0.4335 |
0.1849 |
0.3034 |
0.0060 |
Basic e2e mid speed |
0.1503 |
0.4069 |
0.1976 |
0.2924 |
0.0048 |
bio-answerfinder-3 |
0.2069 |
0.3817 |
0.2317 |
0.3106 |
0.0037 |
bio-answerfinder-4 |
0.1383 |
0.4469 |
0.1886 |
0.3056 |
0.0066 |
LaRSA |
0.1152 |
0.4631 |
0.1703 |
0.3342 |
0.0073 |
The basic end-to-end |
0.1432 |
0.3854 |
0.1881 |
0.2804 |
0.0034 |
ELECTROLBERT-2 |
0.1022 |
0.4889 |
0.1574 |
0.3101 |
0.0075 |
gsl_zs_hybrid |
0.1011 |
0.5015 |
0.1573 |
0.3904 |
0.0084 |
gsl_zs_rrf1 |
0.0989 |
0.4960 |
0.1541 |
0.3829 |
0.0082 |
gsl_zs_rrf2 |
0.1011 |
0.5024 |
0.1574 |
0.3913 |
0.0083 |
gsl_zs_nn |
0.0933 |
0.4700 |
0.1452 |
0.3609 |
0.0098 |
gsl_zs_rrf3 |
0.1000 |
0.4997 |
0.1558 |
0.3778 |
0.0089 |
Deep ML methods for |
0.0556 |
0.2326 |
0.0834 |
0.1245 |
0.0004 |
MindLab QA System |
0.0556 |
0.2326 |
0.0834 |
0.1245 |
0.0004 |
RYGH-1 |
0.1091 |
0.5496 |
0.1704 |
0.4040 |
0.0183 |
RYGH |
0.1080 |
0.5381 |
0.1684 |
0.3925 |
0.0138 |
RYGH-4 |
0.1111 |
0.5424 |
0.1720 |
0.3883 |
0.0166 |
RYGH-5 |
0.1100 |
0.5387 |
0.1703 |
0.3873 |
0.0152 |
bioinfo-0 |
0.0989 |
0.4825 |
0.1528 |
0.3404 |
0.0077 |
bioinfo-1 |
0.1011 |
0.4783 |
0.1552 |
0.3502 |
0.0085 |
bioinfo-2 |
0.1011 |
0.4708 |
0.1549 |
0.3552 |
0.0083 |
bioinfo-3 |
0.1133 |
0.5116 |
0.1728 |
0.3613 |
0.0117 |
bioinfo-4 |
0.1000 |
0.4764 |
0.1539 |
0.3519 |
0.0085 |
RYGH-3 |
0.1091 |
0.5478 |
0.1703 |
0.4058 |
0.0169 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
0.0597 |
0.1597 |
0.0834 |
0.2480 |
0.0008 |
AUEB-System2 |
0.0657 |
0.1648 |
0.0905 |
0.2653 |
0.0007 |
bio-answerfinder |
0.1270 |
0.2790 |
0.1619 |
0.4905 |
0.0047 |
bio-answerfinder-2 |
0.0878 |
0.2301 |
0.1182 |
0.2949 |
0.0031 |
Basic e2e mid speed |
0.0887 |
0.2146 |
0.1184 |
0.3321 |
0.0019 |
bio-answerfinder-3 |
0.1114 |
0.2672 |
0.1463 |
0.4456 |
0.0031 |
bio-answerfinder-4 |
0.0887 |
0.2342 |
0.1197 |
0.2973 |
0.0031 |
LaRSA |
0.0553 |
0.1312 |
0.0720 |
0.1393 |
0.0009 |
The basic end-to-end |
0.0850 |
0.2089 |
0.1140 |
0.3225 |
0.0013 |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.0121 |
0.0365 |
0.0163 |
0.0363 |
0.0000 |
MindLab QA System |
0.0121 |
0.0374 |
0.0163 |
0.0358 |
0.0000 |
RYGH-1 |
0.0845 |
0.2801 |
0.1235 |
0.3620 |
0.0059 |
RYGH |
0.0836 |
0.2747 |
0.1215 |
0.3523 |
0.0049 |
RYGH-4 |
0.1119 |
0.3333 |
0.1577 |
0.6606 |
0.0036 |
RYGH-5 |
0.1126 |
0.3292 |
0.1578 |
0.6596 |
0.0036 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
0.0859 |
0.2862 |
0.1257 |
0.3669 |
0.0067 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
LaRSA |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
AUEB-System1 |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
bio-answerfinder |
- |
- |
- |
- |
- |
bio-answerfinder-2 |
- |
- |
- |
- |
- |
Basic e2e mid speed |
- |
- |
- |
- |
- |
bio-answerfinder-3 |
- |
- |
- |
- |
- |
bio-answerfinder-4 |
- |
- |
- |
- |
- |
LaRSA |
- |
- |
- |
- |
- |
The basic end-to-end |
- |
- |
- |
- |
- |
ELECTROLBERT-2 |
- |
- |
- |
- |
- |
gsl_zs_hybrid |
- |
- |
- |
- |
- |
gsl_zs_rrf1 |
- |
- |
- |
- |
- |
gsl_zs_rrf2 |
- |
- |
- |
- |
- |
gsl_zs_nn |
- |
- |
- |
- |
- |
gsl_zs_rrf3 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA System |
- |
- |
- |
- |
- |
RYGH-1 |
- |
- |
- |
- |
- |
RYGH |
- |
- |
- |
- |
- |
RYGH-4 |
- |
- |
- |
- |
- |
RYGH-5 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
RYGH-3 |
- |
- |
- |
- |
- |
Test batch 5
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
bio-answerfinder |
0.2912 |
0.4205 |
0.3160 |
0.3297 |
0.0044 |
bio-answerfinder-2 |
0.1355 |
0.3603 |
0.1751 |
0.2722 |
0.0034 |
AUEB-System1 |
0.0833 |
0.3543 |
0.1183 |
0.2636 |
0.0015 |
AUEB-System2 |
0.1022 |
0.4078 |
0.1447 |
0.3071 |
0.0037 |
bio-answerfinder-3 |
0.2035 |
0.3134 |
0.2199 |
0.2475 |
0.0011 |
bio-answerfinder-4 |
0.1284 |
0.3429 |
0.1649 |
0.2548 |
0.0022 |
ELECTROLBERT-2 |
0.0833 |
0.4148 |
0.1299 |
0.3029 |
0.0041 |
LaRSA |
0.0878 |
0.4535 |
0.1371 |
0.3078 |
0.0055 |
gsl_zs_hybrid |
0.1033 |
0.4773 |
0.1584 |
0.3687 |
0.0076 |
gsl_zs_rrf1 |
0.1000 |
0.4706 |
0.1540 |
0.3757 |
0.0076 |
gsl_zs_nn |
0.0944 |
0.4461 |
0.1455 |
0.3572 |
0.0058 |
gsl_zs_rrf3 |
0.0989 |
0.4579 |
0.1518 |
0.3588 |
0.0059 |
RYGH |
0.1140 |
0.5096 |
0.1732 |
0.3997 |
0.0087 |
RYGH-3 |
0.1028 |
0.4876 |
0.1576 |
0.3985 |
0.0078 |
RYGH-5 |
0.1000 |
0.4826 |
0.1536 |
0.3846 |
0.0081 |
RYGH-1 |
0.1040 |
0.4904 |
0.1592 |
0.3968 |
0.0078 |
RYGH-4 |
0.1100 |
0.5161 |
0.1682 |
0.4154 |
0.0109 |
ELECTROLBERT-3 |
0.0811 |
0.4122 |
0.1263 |
0.3242 |
0.0036 |
ELECTROLBERT-1 |
0.0800 |
0.4011 |
0.1243 |
0.3230 |
0.0032 |
gsl_zs_rrf2 |
0.1000 |
0.4627 |
0.1534 |
0.3623 |
0.0060 |
bioinfo-0 |
0.1056 |
0.4968 |
0.1619 |
0.3734 |
0.0108 |
bioinfo-1 |
0.1044 |
0.4932 |
0.1605 |
0.3706 |
0.0109 |
bioinfo-2 |
0.1067 |
0.4993 |
0.1634 |
0.3704 |
0.0113 |
bioinfo-3 |
0.1033 |
0.4892 |
0.1584 |
0.3718 |
0.0087 |
bioinfo-4 |
0.1022 |
0.4928 |
0.1581 |
0.3760 |
0.0111 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |