BioASQ Participants Area

Task 7b: Test Results of Phase A

* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .

Test batch 1

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	0.2100	0.2357	0.1845	0.0705	0.0003
lh_sys1	0.1910	0.5281	0.2237	0.1401	0.0078
Deep ML methods for	0.1930	0.5339	0.2230	0.1569	0.0089
Ir_sys1	0.1770	0.5096	0.2095	0.1231	0.0067
Ir_sys2	0.1910	0.5443	0.2248	0.1388	0.0082
Ir_sys3	0.1520	0.3906	0.1717	0.0899	0.0021
Ir_sys4	0.1560	0.3949	0.1752	0.0926	0.0021
lh_sys2	0.1720	0.4534	0.1971	0.1300	0.0039
lh_sys3	0.1770	0.4843	0.2053	0.1224	0.0048
lh_sys4	0.1910	0.5281	0.2237	0.1425	0.0075
lh_sys5	0.1950	0.5420	0.2288	0.1434	0.0084
lalala	0.1914	0.2415	0.1894	0.0777	0.0003

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
lh_sys1	-	-	-	-	-
Deep ML methods for	0.1529	0.2933	0.1773	0.1411	0.0029
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-
lalala	-	-	-	-	-

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
lh_sys1	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-
lalala	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
lh_sys1	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-
lalala	-	-	-	-	-

Test batch 2

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
from milab	0.1010	0.2157	0.1060	0.0828	0.0005
auth-qa-1	0.1913	0.2402	0.1760	0.0626	0.0005
aueb-nlp-1	0.2440	0.6292	0.2770	0.2009	0.0219
aueb-nlp-2	0.2480	0.6159	0.2791	0.2056	0.0203
aueb-nlp-3	0.2470	0.6335	0.2805	0.2007	0.0221
aueb-nlp-4	0.2570	0.6342	0.2888	0.2181	0.0209
aueb-nlp-5	0.4368	0.5861	0.4307	0.2032	0.0145
Deep ML methods for	0.1990	0.5010	0.2227	0.1572	0.0066
MindLab QA System ++	0.0280	0.0884	0.0343	0.0177	0.0001
MindLab Red Lions++	0.0280	0.0884	0.0343	0.0180	0.0001
MindLab QA Reloaded	0.0280	0.0884	0.0343	0.0179	0.0001
MindLab QA System	0.0280	0.0884	0.0343	0.0177	0.0001
Ir_sys1	0.1940	0.4932	0.2247	0.1425	0.0068
Ir_sys2	0.2260	0.5981	0.2610	0.1521	0.0105
Ir_sys3	0.2260	0.5981	0.2610	0.1415	0.0093
Ir_sys4	0.2389	0.5862	0.2679	0.1677	0.0124
lalala	0.2260	0.5981	0.2610	0.1796	0.0132
lh_sys1	0.2260	0.5981	0.2610	0.1744	0.0127
lh_sys2	0.2150	0.5740	0.2483	0.1637	0.0113
lh_sys3	0.2260	0.5981	0.2610	0.1717	0.0123
lh_sys4	0.2290	0.5927	0.2613	0.1784	0.0118
lh_sys5	0.2140	0.5490	0.2426	0.1617	0.0102

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
from milab	-	-	-	-	-
auth-qa-1	-	-	-	-	-
aueb-nlp-1	0.2536	0.3785	0.2535	0.2609	0.0100
aueb-nlp-2	0.2448	0.3763	0.2454	0.2449	0.0110
aueb-nlp-3	0.1892	0.2723	0.1777	0.1663	0.0062
aueb-nlp-4	0.1955	0.2652	0.1805	0.1820	0.0066
aueb-nlp-5	0.2746	0.3414	0.2623	0.2650	0.0099
Deep ML methods for	0.1743	0.2354	0.1752	0.1750	0.0017
MindLab QA System ++	0.0468	0.0529	0.0452	0.0390	0.0000
MindLab Red Lions++	0.0398	0.0361	0.0346	0.0364	0.0000
MindLab QA Reloaded	0.0405	0.0362	0.0349	0.0345	0.0000
MindLab QA System	0.0400	0.0486	0.0385	0.0316	0.0000
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys1	0.1269	0.1708	0.1196	0.0894	0.0014
lh_sys2	0.1276	0.1712	0.1201	0.0826	0.0012
lh_sys3	0.1269	0.1708	0.1196	0.0935	0.0014
lh_sys4	0.1269	0.1708	0.1196	0.0768	0.0011
lh_sys5	0.1276	0.1712	0.1201	0.0933	0.0013

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
from milab	-	-	-	-	-
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
MindLab QA System	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
from milab	-	-	-	-	-
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
MindLab QA System	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
Ir_sys3	-	-	-	-	-
Ir_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

Test batch 3

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	0.2940	0.3025	0.2522	0.1151	0.0025
aueb-nlp-1	0.3420	0.6452	0.3730	0.2761	0.0663
aueb-nlp-2	0.3610	0.6842	0.3952	0.2898	0.0841
aueb-nlp-3	0.3310	0.6472	0.3658	0.2566	0.0657
aueb-nlp-4	0.3460	0.6797	0.3820	0.2839	0.0862
aueb-nlp-5	0.5476	0.6465	0.5289	0.2679	0.0740
Deep ML methods for	0.2770	0.5658	0.3054	0.2272	0.0359
MindLab QA System ++	0.2770	0.5658	0.3054	0.2272	0.0359
MindLab QA System	0.2870	0.5778	0.3159	0.2392	0.0348
MindLab Red Lions++	0.2870	0.5778	0.3159	0.2392	0.0348
MindLab QA Reloaded	0.2840	0.5709	0.3119	0.2323	0.0332
lh_sys1	0.2730	0.5634	0.3077	0.2032	0.0293
Ir_sys1	0.2670	0.5507	0.3021	0.1977	0.0313
Ir_sys2	0.2730	0.5634	0.3077	0.1952	0.0282
lh_sys3	0.2690	0.5562	0.3030	0.2001	0.0249
Ir_sys3	0.2470	0.4513	0.2701	0.1503	0.0093
lh_sys4	0.2760	0.5738	0.3122	0.2142	0.0319
lalala	0.2210	0.5025	0.2570	0.1515	0.0134
lh_sys5	0.2730	0.5634	0.3077	0.2094	0.0309
lh_sys2	0.2730	0.5634	0.3077	0.2021	0.0298

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	0.3636	0.3755	0.3219	0.3657	0.0454
aueb-nlp-2	0.3933	0.4120	0.3522	0.3864	0.0777
aueb-nlp-3	0.2715	0.2661	0.2373	0.2349	0.0266
aueb-nlp-4	0.2853	0.2735	0.2454	0.2609	0.0335
aueb-nlp-5	0.3678	0.3609	0.3277	0.3481	0.0533
Deep ML methods for	0.2049	0.2596	0.2052	0.1955	0.0083
MindLab QA System ++	0.1835	0.2141	0.1732	0.1538	0.0080
MindLab QA System	0.1874	0.1980	0.1675	0.1690	0.0057
MindLab Red Lions++	0.2072	0.2178	0.1804	0.1820	0.0135
MindLab QA Reloaded	0.2202	0.2541	0.2046	0.2330	0.0061
lh_sys1	0.1360	0.1320	0.1142	0.1234	0.0017
Ir_sys1	-	-	-	-	-
Ir_sys2	0.1360	0.1320	0.1142	0.1234	0.0017
lh_sys3	0.1360	0.1320	0.1142	0.0825	0.0012
Ir_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys5	-	-	-	-	-
lh_sys2	0.1360	0.1320	0.1142	0.1234	0.0017

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
lh_sys1	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
Ir_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys5	-	-	-	-	-
lh_sys2	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
lh_sys1	-	-	-	-	-
Ir_sys1	-	-	-	-	-
Ir_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
Ir_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lalala	-	-	-	-	-
lh_sys5	-	-	-	-	-
lh_sys2	-	-	-	-	-

Test batch 4

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	0.1910	0.2842	0.1908	0.0651	0.0008
aueb-nlp-1	0.2541	0.6668	0.2998	0.2102	0.0316
aueb-nlp-2	0.2531	0.6523	0.2992	0.2092	0.0279
aueb-nlp-3	0.2401	0.6451	0.2857	0.1962	0.0282
aueb-nlp-4	0.2481	0.6445	0.2948	0.2080	0.0268
aueb-nlp-5	0.4537	0.6416	0.4580	0.1968	0.0291
Deep ML methods for	0.1980	0.5399	0.2350	0.1646	0.0101
MindLab QA Reloaded	0.2080	0.5664	0.2463	0.1724	0.0121
MindLab QA System ++	0.2080	0.5664	0.2463	0.1724	0.0121
MindLab QA System	0.2080	0.5664	0.2463	0.1724	0.0121
MindLab Red Lions++	0.2080	0.5664	0.2463	0.1724	0.0121
lh_sys2	0.2120	0.5922	0.2558	0.1594	0.0142
lh_sys1	0.2140	0.5856	0.2564	0.1673	0.0141
lh_sys3	0.2040	0.5714	0.2470	0.1569	0.0117
lh_sys4	0.2230	0.6121	0.2695	0.1752	0.0186
lh_sys5	0.2160	0.5901	0.2591	0.1691	0.0154

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	0.3209	0.4321	0.3018	0.3249	0.0281
aueb-nlp-2	0.3254	0.4308	0.3048	0.3409	0.0344
aueb-nlp-3	0.2563	0.3581	0.2346	0.2213	0.0196
aueb-nlp-4	0.2550	0.3325	0.2318	0.2173	0.0178
aueb-nlp-5	0.3256	0.4403	0.3010	0.2976	0.0379
Deep ML methods for	0.1945	0.2836	0.1961	0.1889	0.0039
MindLab QA Reloaded	0.2276	0.2857	0.2093	0.2214	0.0052
MindLab QA System ++	0.2112	0.2317	0.1819	0.1931	0.0058
MindLab QA System	0.1998	0.2669	0.1865	0.1892	0.0064
MindLab Red Lions++	0.2168	0.2718	0.1982	0.2000	0.0067
lh_sys2	0.0793	0.0708	0.0649	0.0360	0.0002
lh_sys1	0.1202	0.1323	0.1057	0.0748	0.0012
lh_sys3	0.1150	0.1069	0.0966	0.0850	0.0011
lh_sys4	0.1150	0.1069	0.0966	0.0874	0.0011
lh_sys5	0.1150	0.1069	0.0966	0.0850	0.0011

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

Test batch 5

Documents

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	0.0990	0.1748	0.1053	0.0248	0.0001
aueb-nlp-1	0.1510	0.4346	0.1799	0.1049	0.0037
aueb-nlp-2	0.1570	0.4221	0.1802	0.1218	0.0036
aueb-nlp-3	0.1440	0.4406	0.1757	0.0968	0.0042
aueb-nlp-4	0.1580	0.4632	0.1905	0.1080	0.0052
aueb-nlp-5	0.2708	0.4196	0.2712	0.1004	0.0038
MindLab QA System ++	0.1220	0.4007	0.1488	0.0800	0.0026
MindLab Red Lions++	0.1220	0.4007	0.1488	0.0800	0.0026
MindLab QA System	0.1220	0.4007	0.1488	0.0800	0.0026
MindLab QA Reloaded	0.1220	0.4007	0.1488	0.0800	0.0026
Deep ML methods for	0.1240	0.4004	0.1497	0.0823	0.0026
auth-qa-2	0.0170	0.0478	0.0188	0.0064	0.0000
lh_sys1	0.1280	0.4235	0.1570	0.0829	0.0028
lh_sys2	0.1180	0.3849	0.1461	0.0779	0.0020
lh_sys3	0.1210	0.4181	0.1501	0.0808	0.0027
lh_sys4	0.1310	0.4311	0.1612	0.0850	0.0028
lh_sys5	0.1350	0.4432	0.1653	0.0884	0.0033

Snippets

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	0.1168	0.2681	0.1312	0.1146	0.0010
aueb-nlp-2	0.1459	0.3019	0.1575	0.1383	0.0018
aueb-nlp-3	0.1033	0.2361	0.1126	0.0859	0.0010
aueb-nlp-4	0.1118	0.2600	0.1222	0.0948	0.0013
aueb-nlp-5	0.1391	0.2830	0.1520	0.1194	0.0016
MindLab QA System ++	0.0720	0.1923	0.0874	0.0781	0.0004
MindLab Red Lions++	0.0701	0.1970	0.0854	0.0578	0.0005
MindLab QA System	0.0737	0.2129	0.0885	0.0670	0.0005
MindLab QA Reloaded	0.0720	0.1923	0.0874	0.0781	0.0004
Deep ML methods for	0.0756	0.2440	0.0946	0.0537	0.0008
auth-qa-2	-	-	-	-	-
lh_sys1	0.0486	0.0743	0.0474	0.0449	0.0001
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

Concepts

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
auth-qa-2	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

RDF Triples

System	Mean precision	Recall	F-Measure	MAP	GMAP
auth-qa-1	-	-	-	-	-
aueb-nlp-1	-	-	-	-	-
aueb-nlp-2	-	-	-	-	-
aueb-nlp-3	-	-	-	-	-
aueb-nlp-4	-	-	-	-	-
aueb-nlp-5	-	-	-	-	-
MindLab QA System ++	-	-	-	-	-
MindLab Red Lions++	-	-	-	-	-
MindLab QA System	-	-	-	-	-
MindLab QA Reloaded	-	-	-	-	-
Deep ML methods for	-	-	-	-	-
auth-qa-2	-	-	-	-	-
lh_sys1	-	-	-	-	-
lh_sys2	-	-	-	-	-
lh_sys3	-	-	-	-	-
lh_sys4	-	-	-	-	-
lh_sys5	-	-	-	-	-

Copyright © 2023, the BioASQ Challenge (http://bioasq.org)

Platinum sponsors of the BioASQ Challenge between 2012 and 2023

Google was a proud sponsor of the BioASQ Challenge from 2018 to 2022.
BioASQ was supported by the National Library of Medicine of the National Institutes of Health under award number R13LM012214. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
The research leading to these results has received funding from the European Commission's Seventh Framework Programme (FP7/2007-2013, ICT-2011.4.4(d), Intelligent Information Management, Targeted Competition Framework) under grant agreement n° 318652.