BioASQ Participants Area

Task 10b: Test Results of Phase A

* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .

Test batch 1

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
The basic end-to-end	0.2496	0.5135	0.2787	0.4278	0.0272
AUEB-System1	0.1678	0.4092	0.1899	0.3394	0.0073
Fleming-1	0.0878	0.1590	0.0866	0.1285	0.0003
bio-answerfinder	0.3908	0.4170	0.3553	0.4129	0.0138
gsl_zs_hybrid	0.2222	0.5514	0.2482	0.4633	0.0571
bio-answerfinder-2	0.2613	0.4715	0.2578	0.4123	0.0293
gsl_zs_rrf1	0.2311	0.5658	0.2584	0.4779	0.0590
gsl_zs_rrf2	0.2289	0.5529	0.2550	0.4759	0.0533
gsl_zs_nn	0.2067	0.5269	0.2314	0.4570	0.0465
gsl_zs_rrf3	0.2200	0.5167	0.2449	0.4325	0.0382
Basic e2e mid speed	0.2396	0.4960	0.2668	0.4165	0.0249
RYGH-1	0.2889	0.6122	0.2999	0.5624	0.0992
RYGH	0.2730	0.5980	0.2911	0.5365	0.0980
RYGH-3	0.2765	0.6162	0.2937	0.5538	0.1267
RYGH-4	0.2774	0.6177	0.2943	0.5620	0.1286
AUEB-System2	0.2100	0.4634	0.2280	0.4035	0.0137
simple baseline solr	0.0022	0.0015	0.0018	0.0008	0.0000
bioinfo-0	0.2289	0.5574	0.2532	0.4616	0.0764
bioinfo-1	0.2311	0.5569	0.2539	0.4673	0.0694
bioinfo-2	0.2256	0.5550	0.2504	0.4577	0.0743
bioinfo-3	0.2289	0.5529	0.2508	0.4627	0.0709
Fleming-2	0.0878	0.1596	0.0862	0.1253	0.0003
Fleming-3	0.0878	0.1596	0.0862	0.1253	0.0003

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
The basic end-to-end	0.1289	0.0009	0.0017	0.2069	0.0016
AUEB-System1	0.1534	0.2283	0.1475	0.3116	0.0037
Fleming-1	-	-	-	-	-
bio-answerfinder	0.2144	0.2774	0.1914	0.5818	0.0074
gsl_zs_hybrid	-	-	-	-	-
bio-answerfinder-2	0.2075	0.2390	0.1719	0.3365	0.0096
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
Basic e2e mid speed	0.1266	0.0009	0.0017	0.2057	0.0016
RYGH-1	0.3211	0.4301	0.2886	0.5925	0.0636
RYGH	0.2949	0.4282	0.2762	0.5752	0.0925
RYGH-3	0.3075	0.4639	0.2890	0.6043	0.1236
RYGH-4	0.3068	0.4576	0.2865	0.5947	0.1220
AUEB-System2	0.1773	0.2687	0.1658	0.3569	0.0079
simple baseline solr	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
The basic end-to-end	-	-	-	-	-
AUEB-System1	-	-	-	-	-
Fleming-1	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
Basic e2e mid speed	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH-4	-	-	-	-	-
AUEB-System2	-	-	-	-	-
simple baseline solr	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
The basic end-to-end	-	-	-	-	-
AUEB-System1	-	-	-	-	-
Fleming-1	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
Basic e2e mid speed	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH-4	-	-	-	-	-
AUEB-System2	-	-	-	-	-
simple baseline solr	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-

Test batch 2

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
simple baseline solr	0.0867	0.2436	0.0941	0.1743	0.0005
bio-answerfinder	0.3458	0.4061	0.3094	0.4004	0.0107
bio-answerfinder-2	0.2811	0.4112	0.2573	0.4056	0.0131
bio-answerfinder-3	0.2924	0.3090	0.2450	0.3153	0.0035
bio-answerfinder-4	0.2753	0.3968	0.2498	0.3992	0.0115
Fleming-1	0.0833	0.2474	0.0924	0.1793	0.0007
AUEB-System1	0.1378	0.3451	0.1433	0.2712	0.0058
AUEB-System2	0.2067	0.4715	0.2060	0.4068	0.0259
The basic end-to-end	0.1848	0.4408	0.2032	0.3419	0.0095
Basic e2e mid speed	0.1859	0.4435	0.2046	0.3424	0.0112
bioinfo-0	0.2067	0.4658	0.2128	0.3885	0.0261
bioinfo-1	0.2056	0.4650	0.2120	0.3966	0.0262
gsl_zs_rrf1	0.2244	0.5116	0.2310	0.4554	0.0402
gsl_zs_hybrid	0.2233	0.5176	0.2299	0.4681	0.0415
gsl_zs_nn	0.2056	0.5032	0.2162	0.4302	0.0342
gsl_zs_rrf3	0.2167	0.4866	0.2197	0.4667	0.0310
gsl_zs_rrf2	0.2144	0.5092	0.2228	0.4641	0.0340
RYGH-5	0.2300	0.4824	0.2274	0.4730	0.0316
RYGH-1	0.2758	0.5809	0.2736	0.5269	0.0604
RYGH-3	0.2859	0.5722	0.2805	0.5371	0.0558
RYGH	0.2825	0.5632	0.2743	0.5230	0.0528
RYGH-4	0.2868	0.5686	0.2792	0.5347	0.0501
Fleming-2	0.0733	0.2143	0.0810	0.1721	0.0005
Fleming-3	0.0667	0.1853	0.0725	0.1289	0.0003
bioinfo-2	0.2067	0.4667	0.2128	0.4084	0.0253
bioinfo-3	0.2067	0.4767	0.2135	0.3957	0.0272
bioinfo-4	0.1789	0.4086	0.1849	0.3323	0.0123
Deep ML methods for	0.1722	0.4343	0.1801	0.3477	0.0182
MindLab QA System	0.1722	0.4343	0.1801	0.3477	0.0182
Fleming-4	0.0778	0.2206	0.0838	0.1444	0.0005
MindLab Red Lions++	0.1722	0.4343	0.1801	0.3477	0.0182

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
simple baseline solr	-	-	-	-	-
bio-answerfinder	0.2303	0.2545	0.1818	0.4856	0.0053
bio-answerfinder-2	0.2346	0.2689	0.1835	0.4238	0.0078
bio-answerfinder-3	0.2137	0.2153	0.1622	0.4166	0.0026
bio-answerfinder-4	0.2289	0.2500	0.1759	0.4125	0.0062
Fleming-1	-	-	-	-	-
AUEB-System1	0.1259	0.1528	0.1033	0.2510	0.0017
AUEB-System2	0.1722	0.2088	0.1316	0.3194	0.0057
The basic end-to-end	0.1044	0.0005	0.0011	0.1328	0.0004
Basic e2e mid speed	0.1053	0.0006	0.0011	0.1315	0.0005
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-5	0.2268	0.3407	0.2072	0.7596	0.0232
RYGH-1	0.2866	0.3800	0.2487	0.6042	0.0273
RYGH-3	0.2905	0.3723	0.2487	0.5894	0.0251
RYGH	0.2932	0.3782	0.2525	0.6104	0.0255
RYGH-4	0.2880	0.3707	0.2480	0.5929	0.0235
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Deep ML methods for	0.1194	0.1794	0.1095	0.3635	0.0013
MindLab QA System	0.1194	0.1794	0.1095	0.3635	0.0013
Fleming-4	-	-	-	-	-
MindLab Red Lions++	0.1194	0.1794	0.1095	0.3635	0.0013

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
simple baseline solr	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
bio-answerfinder-3	-	-	-	-	-
bio-answerfinder-4	-	-	-	-	-
Fleming-1	-	-	-	-	-
AUEB-System1	-	-	-	-	-
AUEB-System2	-	-	-	-	-
The basic end-to-end	-	-	-	-	-
Basic e2e mid speed	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-5	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-4	-	-	-	-	-
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System	-	-	-	-	-
Fleming-4	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
simple baseline solr	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
bio-answerfinder-3	-	-	-	-	-
bio-answerfinder-4	-	-	-	-	-
Fleming-1	-	-	-	-	-
AUEB-System1	-	-	-	-	-
AUEB-System2	-	-	-	-	-
The basic end-to-end	-	-	-	-	-
Basic e2e mid speed	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-5	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-4	-	-	-	-	-
Fleming-2	-	-	-	-	-
Fleming-3	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System	-	-	-	-	-
Fleming-4	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-

Test batch 3

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
Basic e2e mid speed	0.3047	0.5131	0.3075	0.4394	0.0308
bio-answerfinder	0.3767	0.4826	0.3463	0.4306	0.0256
bio-answerfinder-2	0.2718	0.4898	0.2705	0.4335	0.0235
Fleming-1	0.0944	0.3592	0.1238	0.2581	0.0030
Fleming-2	0.0767	0.3097	0.1004	0.2192	0.0012
AUEB-System1	0.1556	0.4556	0.1779	0.3475	0.0139
AUEB-System2	0.2000	0.5252	0.2183	0.4690	0.0483
bio-answerfinder-3	0.3378	0.4317	0.3073	0.4053	0.0143
bio-answerfinder-4	0.2742	0.5017	0.2742	0.4437	0.0272
Deep ML methods for	0.1356	0.3704	0.1508	0.2528	0.0062
gsl_zs_hybrid	0.2244	0.5808	0.2493	0.5166	0.0740
gsl_zs_rrf1	0.2233	0.5858	0.2499	0.5178	0.0738
gsl_zs_nn	0.2011	0.5292	0.2249	0.4853	0.0319
gsl_zs_rrf3	0.2056	0.5430	0.2295	0.4912	0.0396
gsl_zs_rrf2	0.1778	0.5066	0.2047	0.4424	0.0266
RYGH-1	0.2626	0.6345	0.2833	0.5787	0.0995
RYGH	0.2625	0.6468	0.2860	0.5935	0.0856
RYGH-3	0.2623	0.6356	0.2840	0.5949	0.0943
RYGH-4	0.2598	0.6458	0.2837	0.5930	0.1026
RYGH-5	0.2567	0.6574	0.2861	0.5994	0.1154
bioinfo-0	0.2189	0.6284	0.2538	0.4973	0.0768
bioinfo-1	0.2200	0.6294	0.2547	0.4926	0.0705
bioinfo-2	0.2200	0.6294	0.2547	0.4926	0.0705
bioinfo-3	0.2178	0.6063	0.2502	0.4803	0.0602
bioinfo-4	0.2189	0.6293	0.2543	0.4895	0.0694
Fleming-3	0.1322	0.4667	0.1733	0.3470	0.0114
Fleming-4	0.1122	0.4140	0.1471	0.3049	0.0046
The basic end-to-end	0.3093	0.5148	0.3102	0.4450	0.0311
BioNIR Prepro-mid	0.3070	0.5070	0.3081	0.4223	0.0297

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
Basic e2e mid speed	0.2228	0.3395	0.2036	0.5274	0.0152
bio-answerfinder	0.2272	0.3480	0.2162	0.6097	0.0168
bio-answerfinder-2	0.2152	0.3059	0.1975	0.4151	0.0122
Fleming-1	-	-	-	-	-
Fleming-2	-	-	-	-	-
AUEB-System1	0.1468	0.2471	0.1398	0.3717	0.0092
AUEB-System2	0.1701	0.2928	0.1590	0.4350	0.0219
bio-answerfinder-3	0.2174	0.3123	0.2050	0.5274	0.0116
bio-answerfinder-4	0.2179	0.3174	0.1996	0.4236	0.0141
Deep ML methods for	0.0495	0.1103	0.0518	0.1129	0.0003
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-1	0.2735	0.4700	0.2725	0.6288	0.0630
RYGH	0.2596	0.4831	0.2649	0.6242	0.0630
RYGH-3	0.2701	0.4661	0.2683	0.6111	0.0570
RYGH-4	0.2657	0.4758	0.2673	0.6148	0.0616
RYGH-5	0.2703	0.4932	0.2839	0.9386	0.0762
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Fleming-3	-	-	-	-	-
Fleming-4	-	-	-	-	-
The basic end-to-end	0.2271	0.3417	0.2065	0.5329	0.0152
BioNIR Prepro-mid	0.2162	0.3263	0.1980	0.4977	0.0139

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
Basic e2e mid speed	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
Fleming-1	-	-	-	-	-
Fleming-2	-	-	-	-	-
AUEB-System1	-	-	-	-	-
AUEB-System2	-	-	-	-	-
bio-answerfinder-3	-	-	-	-	-
bio-answerfinder-4	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH-4	-	-	-	-	-
RYGH-5	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Fleming-3	-	-	-	-	-
Fleming-4	-	-	-	-	-
The basic end-to-end	-	-	-	-	-
BioNIR Prepro-mid	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
Basic e2e mid speed	-	-	-	-	-
bio-answerfinder	-	-	-	-	-
bio-answerfinder-2	-	-	-	-	-
Fleming-1	-	-	-	-	-
Fleming-2	-	-	-	-	-
AUEB-System1	-	-	-	-	-
AUEB-System2	-	-	-	-	-
bio-answerfinder-3	-	-	-	-	-
bio-answerfinder-4	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
gsl_zs_hybrid	-	-	-	-	-
gsl_zs_rrf1	-	-	-	-	-
gsl_zs_nn	-	-	-	-	-
gsl_zs_rrf3	-	-	-	-	-
gsl_zs_rrf2	-	-	-	-	-
RYGH-1	-	-	-	-	-
RYGH	-	-	-	-	-
RYGH-3	-	-	-	-	-
RYGH-4	-	-	-	-	-
RYGH-5	-	-	-	-	-
bioinfo-0	-	-	-	-	-
bioinfo-1	-	-	-	-	-
bioinfo-2	-	-	-	-	-
bioinfo-3	-	-	-	-	-
bioinfo-4	-	-	-	-	-
Fleming-3	-	-	-	-	-
Fleming-4	-	-	-	-	-
The basic end-to-end	-	-	-	-	-
BioNIR Prepro-mid	-	-	-	-	-

Test batch 4

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
AUEB-System1	0.1500	0.4069	0.1717	0.3073	0.0076
AUEB-System2	0.1967	0.4411	0.2106	0.3868	0.0145
bio-answerfinder	0.3917	0.4389	0.3358	0.3837	0.0273
bio-answerfinder-2	0.2879	0.4828	0.2838	0.4014	0.0382
Basic e2e mid speed	0.3017	0.4505	0.2873	0.3706	0.0180
bio-answerfinder-3	0.3196	0.3955	0.2784	0.3561	0.0111
bio-answerfinder-4	0.2905	0.4971	0.2879	0.4067	0.0427
LaRSA	0.2773	0.5174	0.2730	0.4357	0.0483
The basic end-to-end	0.2954	0.4306	0.2788	0.3586	0.0139
Fleming-3	0.2000	0.5000	0.2224	0.3769	0.0297
gsl_zs_hybrid	0.2344	0.5496	0.2539	0.4881	0.0561
gsl_zs_rrf1	0.2378	0.5492	0.2553	0.4836	0.0566
gsl_zs_rrf2	0.2411	0.5547	0.2590	0.4927	0.0573
gsl_zs_nn	0.2044	0.5030	0.2230	0.4489	0.0491
gsl_zs_rrf3	0.2378	0.5534	0.2564	0.4791	0.0613
Deep ML methods for	0.1167	0.2429	0.1223	0.1695	0.0012
MindLab QA System	0.1167	0.2429	0.1223	0.1695	0.0012
RYGH-1	0.2699	0.6056	0.2839	0.5265	0.1053
RYGH	0.2715	0.5942	0.2829	0.5169	0.0833
RYGH-4	0.2722	0.5994	0.2858	0.5211	0.0969
RYGH-5	0.2678	0.5973	0.2823	0.5161	0.0966
bioinfo-0	0.2300	0.5318	0.2480	0.4486	0.0427
bioinfo-1	0.2322	0.5269	0.2505	0.4568	0.0431
bioinfo-2	0.2411	0.5207	0.2552	0.4630	0.0392
bioinfo-3	0.2533	0.5518	0.2701	0.4648	0.0491
bioinfo-4	0.2300	0.5251	0.2483	0.4526	0.0459
RYGH-3	0.2721	0.6061	0.2850	0.5284	0.1056

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP