BioASQ Participants Area

Test Results for Task 1a

Here you can find the evaluation measures for the systems that have uploaded results for the test sets. The "System Description" of each system is used.
The tables are updated periodically as new annotations from MEDLINE become available. As a result, the values of the measures as well as the ranking may change while new annotations become available. When "-" are placed in a table it means that no results were available when the page was updated.
The baselines of the challenge are the systems: BioASQ_Baseline and MTI First Line Index

Evaluation Overview

In the following table you can see the number of the articles for each test set that has been released. In parentheses, there is the number of the articles that have been annotated by the curators for each test set.

	Batch 1	Batch 2	Batch 3
Week 1	1934(1592)	4869(4068)	7578(6022)
Week 2	830(743)	5551(4788)	10139(3918)
Week 3	790(762)	7144(6081)	8722(2969)
Week 4	2233(722)	4623(3908)	1976(1318)
Week 5	6562(5901)	8233(6651)	1744(1209)
Week 6	4414(4152)	8381(6762)	1357(696)

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2614	0.2582	0.2805	0.2466	0.3831	0.3807	0.3136	0.2459	0.2791	0.1493
MTI First Line Index	0.5405	0.5899	0.5279	0.5330	0.5457	0.4999	0.4700	0.5818	0.5047	0.3792
Baseline Prediction	0.4517	0.4488	0.4856	0.4462	0.4800	0.3536	0.3340	0.4488	0.4546	0.2973
svm	0.4517	0.4488	0.4856	0.4462	0.4800	0.3536	0.3340	0.4488	0.4546	0.2973
MeSH Indexing	0.4262	0.3328	0.6717	0.4197	0.3157	0.5282	0.3990	0.3178	0.6467	0.2761

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3060	0.5400	0.5373	0.5053	0.3366	0.3117
MTI First Line Index	0.4601	0.7451	0.6540	0.6677	0.5139	0.4494
Baseline Prediction	0.4036	0.6457	0.6620	0.6278	0.4087	0.4287
svm	0.4036	0.6457	0.6620	0.6278	0.4087	0.4287
MeSH Indexing	0.4048	0.5166	0.8098	0.6017	0.3428	0.5552

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2382	0.2247	0.2721	0.2237	0.3107	0.3505	0.2909	0.2139	0.2687	0.1338
MTI First Line Index	0.5376	0.5494	0.5520	0.5225	0.4894	0.4711	0.4396	0.5448	0.5305	0.3706
MeSH Indexing	0.4561	0.3638	0.6888	0.4498	0.3304	0.5356	0.4343	0.3490	0.6581	0.3042
Wishart-S1	0.4404	0.4042	0.5224	0.4306	0.3981	0.3976	0.3562	0.4047	0.4830	0.2848
Wishart-S2	0.0348	0.0598	0.0447	0.0381	0.0579	0.0535	0.0508	0.0322	0.0379	0.0210
Wishart-S3	0.4416	0.4054	0.5244	0.4322	0.4125	0.3975	0.3558	0.4054	0.4848	0.2859
Wishart-S4	0.4407	0.4046	0.5225	0.4310	0.4242	0.3890	0.3486	0.4046	0.4839	0.2851
Wishart-S5	0.4407	0.4046	0.5224	0.4310	0.4241	0.3890	0.3485	0.4046	0.4838	0.2851
cole_hce1	0.2192	0.1322	0.6740	0.2143	0.0708	0.5362	0.2724	0.1321	0.6423	0.1227
cole_hce2	0.2329	0.1404	0.6950	0.2263	0.1050	0.5855	0.3045	0.1403	0.6845	0.1313
utai_rebayct	0.1718	0.1035	0.5409	0.1683	0.1735	0.4676	0.3267	0.1035	0.5053	0.0934
Panacea	0.4208	0.6035	0.3237	0.3918	0.6134	0.1715	0.1809	0.6086	0.3216	0.2636
MCTeamMM	0.2117	0.2049	0.2458	0.2105	0.3827	0.3003	0.2785	0.2051	0.2189	0.1231
MCTeamMM10	0.2100	0.2238	0.2247	0.2110	0.4012	0.2727	0.2598	0.2239	0.1976	0.1240
MCTeamSR	0.4561	0.4360	0.5102	0.4447	0.4677	0.3283	0.3152	0.4360	0.4780	0.3013
MCTeamSR8	0.4584	0.5166	0.4454	0.4523	0.5666	0.2669	0.2720	0.5166	0.4119	0.3090
Hippocrates	0.4370	0.4873	0.3925	0.4121	0.6125	0.1721	0.1809	0.4998	0.3882	0.2774
MCTeamMR24	0.3996	0.3304	0.5503	0.3917	0.4067	0.4072	0.3740	0.3286	0.5099	0.2531
Galen	0.4271	0.3497	0.5398	0.4065	0.4565	0.3389	0.3017	0.3582	0.5286	0.2693

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2871	0.4831	0.5314	0.4682	0.3061	0.3063
MTI First Line Index	0.4474	0.6905	0.6717	0.6444	0.4798	0.4607
MeSH Indexing	0.4199	0.5292	0.8110	0.6093	0.3623	0.5570
Wishart-S1	0.3893	0.5692	0.7101	0.6003	0.3683	0.4541
Wishart-S2	0.1619	0.2764	0.2779	0.2291	0.1954	0.1801
Wishart-S3	0.3908	0.5741	0.7054	0.6016	0.3708	0.4527
Wishart-S4	0.3912	0.5794	0.7047	0.6045	0.3723	0.4515
Wishart-S5	0.3912	0.5793	0.7047	0.6044	0.3725	0.4515
cole_hce1	0.2579	0.2473	0.8651	0.3703	0.1709	0.5921
cole_hce2	0.2745	0.2715	0.8863	0.4000	0.1840	0.6067
utai_rebayct	0.2305	0.2249	0.8094	0.3379	0.1543	0.5191
Panacea	0.3462	0.7500	0.4190	0.4928	0.5125	0.2881
MCTeamMM	0.2762	0.4471	0.5048	0.4453	0.2918	0.2878
MCTeamMM10	0.2777	0.4799	0.4668	0.4431	0.3177	0.2693
MCTeamSR	0.4003	0.6156	0.6201	0.5839	0.4166	0.4239
MCTeamSR8	0.3907	0.7056	0.5204	0.5620	0.4769	0.3634
Hippocrates	0.3584	0.6941	0.4529	0.5109	0.4607	0.3227
MCTeamMR24	0.3764	0.5117	0.6988	0.5578	0.3429	0.4662
Galen	0.3817	0.5239	0.6979	0.5701	0.3440	0.4723

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2616	0.2570	0.2550	0.2374	0.3508	0.3453	0.3029	0.2561	0.2673	0.1443
MTI First Line Index	0.5437	0.6026	0.5040	0.5247	0.5482	0.4468	0.4335	0.6033	0.4948	0.3730
multifactorial	0.1232	0.7793	0.0771	0.1381	0.7793	0.0003	0.0003	0.7793	0.0669	0.0771
MeSH Indexing	0.5303	0.5168	0.5842	0.5202	0.4807	0.4491	0.4276	0.5035	0.5600	0.3662
Wishart-S1	0.4588	0.4966	0.4556	0.4545	0.5107	0.3380	0.3270	0.4966	0.4264	0.3052
Wishart-S2	0.4705	0.5083	0.4616	0.4564	0.5021	0.3555	0.3372	0.4908	0.4518	0.3067
Wishart-S3	0.4706	0.5381	0.4393	0.4568	0.5295	0.3330	0.3212	0.5198	0.4300	0.3077
Wishart-S4	0.4665	0.5648	0.4173	0.4516	0.5582	0.3105	0.3054	0.5453	0.4077	0.3038
Wishart-S5	0.4263	0.5326	0.3869	0.4209	0.5019	0.3532	0.3356	0.5040	0.3693	0.2775
cole_hce1	0.2401	0.1490	0.6434	0.2356	0.0853	0.5180	0.3152	0.1487	0.6233	0.1365
cole_hce2	0.2806	0.1738	0.7388	0.2742	0.1278	0.6150	0.3729	0.1737	0.7301	0.1632
utai_rebayct	0.1900	0.1176	0.5167	0.1865	0.1799	0.4701	0.3536	0.1176	0.4947	0.1046
MCTeamMM	0.4598	0.4977	0.4419	0.4492	0.5103	0.2824	0.2831	0.4977	0.4273	0.3059
MCTeamMM10	0.2063	0.2371	0.1963	0.2055	0.4072	0.2466	0.2381	0.2396	0.1812	0.1199
MCTeamSR	0.4406	0.5868	0.3717	0.4371	0.6142	0.2120	0.2217	0.5868	0.3527	0.2946
MCTeamSR8	0.1888	0.2774	0.1571	0.1918	0.4500	0.1963	0.1954	0.2788	0.1427	0.1117
MCTeamMR24	0.4230	0.4181	0.4550	0.4168	0.4662	0.3214	0.3149	0.4157	0.4304	0.2731
Galen	0.4480	0.3869	0.5123	0.4273	0.4557	0.3680	0.3282	0.3983	0.5119	0.2858
MeSH Indexing New	0.5231	0.4861	0.6072	0.5135	0.4525	0.4732	0.4412	0.4729	0.5853	0.3606
MeSH Indexing Pre	0.5339	0.5146	0.5937	0.5238	0.4884	0.4576	0.4323	0.5024	0.5696	0.3691
MeSH Indexing Ref	0.5209	0.4627	0.6202	0.5083	0.4413	0.4822	0.4439	0.4627	0.5959	0.3551
UCD-CMd	0.1185	0.5113	0.0758	0.1235	0.3958	0.0586	0.0591	0.4274	0.0688	0.0702
UCD-CMr	0.2024	0.1531	0.3281	0.1967	0.2670	0.4088	0.3452	0.1469	0.3253	0.1127
UCD-CMp	0.1185	0.5113	0.0758	0.1235	0.3958	0.0586	0.0591	0.4274	0.0688	0.0702
UCD-CMgg	0.2174	0.1802	0.2766	0.2071	0.3573	0.3411	0.3053	0.1799	0.2747	0.1204
MeSH Indexing Add	0.5203	0.4622	0.6194	0.5077	0.4383	0.4818	0.4435	0.4622	0.5952	0.3545

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3027	0.5453	0.5141	0.4972	0.3471	0.2964
MTI First Line Index	0.4556	0.7523	0.6217	0.6476	0.5241	0.4368
multifactorial	0.0928	0.7894	0.0645	0.1172	0.2972	0.0565
MeSH Indexing	0.4610	0.6922	0.6899	0.6578	0.4816	0.4830
Wishart-S1	0.4041	0.6877	0.6217	0.6261	0.4363	0.4025
Wishart-S2	0.3979	0.6999	0.6186	0.6193	0.4350	0.4014
Wishart-S3	0.3960	0.7241	0.5921	0.6129	0.4512	0.3856
Wishart-S4	0.3879	0.7407	0.5620	0.5986	0.4592	0.3678
Wishart-S5	0.3693	0.7074	0.5894	0.6056	0.4371	0.3490
cole_hce1	0.2791	0.2763	0.8448	0.4045	0.1892	0.5767
cole_hce2	0.3084	0.3147	0.8875	0.4506	0.2116	0.6179
utai_rebayct	0.2526	0.2535	0.7982	0.3714	0.1741	0.5102
MCTeamMM	0.4039	0.6945	0.5358	0.5744	0.4700	0.3801
MCTeamMM10	0.2832	0.5247	0.4353	0.4511	0.3446	0.2563
MCTeamSR	0.3729	0.7846	0.4189	0.5166	0.5304	0.3096
MCTeamSR8	0.2719	0.5991	0.3589	0.4242	0.3964	0.2190
MCTeamMR24	0.3972	0.6320	0.5766	0.5734	0.4267	0.4025
Galen	0.3945	0.5082	0.6837	0.5606	0.3697	0.4544
MeSH Indexing New	0.4611	0.6647	0.7187	0.6602	0.4612	0.5024
MeSH Indexing Pre	0.4639	0.6929	0.6965	0.6626	0.4813	0.4885
MeSH Indexing Ref	0.4594	0.6391	0.7356	0.6593	0.4442	0.5121
UCD-CMd	0.1483	0.7012	0.1343	0.2022	0.3743	0.0985
UCD-CMr	0.2632	0.3531	0.6217	0.4275	0.2231	0.3645
UCD-CMp	0.1483	0.7012	0.1343	0.2022	0.3743	0.0985
UCD-CMgg	0.2598	0.4048	0.5414	0.4385	0.2318	0.3265
MeSH Indexing Add	0.4592	0.6392	0.7349	0.6586	0.4436	0.5123

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2767	0.2743	0.2994	0.2645	0.3263	0.3726	0.3229	0.2577	0.2988	0.1627
MTI First Line Index	0.5575	0.6166	0.5372	0.5498	0.5318	0.4831	0.4638	0.6045	0.5172	0.3979
multifactorial	0.1191	0.7047	0.0772	0.1370	0.7047	0.0003	0.0003	0.7047	0.0651	0.0772
MeSH Indexing	0.5230	0.5072	0.5966	0.5224	0.4590	0.4630	0.4352	0.4865	0.5653	0.3689
Wishart-S1	0.4738	0.5446	0.4646	0.4664	0.5079	0.3354	0.3213	0.5136	0.4398	0.3171
Wishart-S2	0.4735	0.5089	0.4893	0.4661	0.4805	0.3624	0.3417	0.4830	0.4644	0.3161
Wishart-S3	0.4715	0.4912	0.5012	0.4643	0.4657	0.3746	0.3490	0.4665	0.4765	0.3141
Wishart-S5	0.4706	0.4903	0.4910	0.4716	0.4984	0.3638	0.3492	0.4909	0.4520	0.3210
cole_hce1	0.3334	0.2419	0.5678	0.3280	0.1668	0.4240	0.3302	0.2419	0.5361	0.2023
cole_hce2	0.3929	0.2851	0.6570	0.3844	0.2416	0.5038	0.3964	0.2851	0.6317	0.2470
utai_rebayct	0.2506	0.1819	0.4331	0.2478	0.2306	0.3410	0.2932	0.1819	0.4029	0.1446
MCTeamMM	0.4439	0.4222	0.4927	0.4374	0.4644	0.3207	0.3100	0.4222	0.4679	0.2962
MCTeamMM10	0.4282	0.3687	0.5339	0.4199	0.3886	0.3586	0.3270	0.3687	0.5107	0.2805
MCTeamSR	0.3945	0.3041	0.5827	0.3855	0.3044	0.4051	0.3420	0.3041	0.5616	0.2507
MCTeamSR8	0.2196	0.2817	0.2009	0.2254	0.4368	0.2342	0.2328	0.2810	0.1802	0.1338
MCTeamMR24	0.4352	0.4502	0.4507	0.4329	0.4980	0.3053	0.3063	0.4483	0.4228	0.2884
MeSH Indexing New	0.5103	0.4723	0.6196	0.5115	0.4290	0.4867	0.4483	0.4516	0.5864	0.3588
MeSH Indexing Pre	0.5230	0.5072	0.5966	0.5224	0.4590	0.4630	0.4352	0.4865	0.5653	0.3689
MeSH Indexing Ref	0.5008	0.4311	0.6314	0.4934	0.4082	0.4939	0.4482	0.4311	0.5972	0.3407
UCD-CMd	0.1427	0.5070	0.0945	0.1492	0.4351	0.0910	0.0908	0.4272	0.0857	0.0860
UCD-CMr	0.2109	0.1543	0.4170	0.2145	0.2408	0.4889	0.3972	0.1426	0.4046	0.1233
UCD-CMp	0.1427	0.5070	0.0945	0.1492	0.4351	0.0910	0.0908	0.4272	0.0857	0.0860
UCD-CMgg	0.2450	0.1944	0.3650	0.2432	0.3367	0.4231	0.3664	0.1878	0.3524	0.1433
MeSH Indexing Add	0.5016	0.4319	0.6324	0.4943	0.4085	0.4937	0.4480	0.4319	0.5982	0.3414

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3094	0.5404	0.5391	0.5090	0.3424	0.3096
MTI First Line Index	0.4609	0.7477	0.6454	0.6670	0.5198	0.4427
multifactorial	0.0923	0.7203	0.0594	0.1082	0.2841	0.0565
MeSH Indexing	0.4539	0.6780	0.7011	0.6628	0.4618	0.4827
Wishart-S1	0.3887	0.7127	0.5911	0.6070	0.4381	0.3854
Wishart-S2	0.3965	0.6854	0.6216	0.6151	0.4277	0.4065
Wishart-S3	0.3975	0.6693	0.6364	0.6168	0.4187	0.4167
Wishart-S5	0.4064	0.6737	0.6380	0.6330	0.4239	0.4173
cole_hce1	0.3351	0.4068	0.7551	0.5115	0.2651	0.4918
cole_hce2	0.3745	0.4515	0.8021	0.5603	0.2988	0.5414
utai_rebayct	0.2884	0.3518	0.6811	0.4471	0.2290	0.4216
MCTeamMM	0.3904	0.6181	0.6046	0.5888	0.4007	0.4060
MCTeamMM10	0.3863	0.5576	0.6603	0.5837	0.3626	0.4417
MCTeamSR	0.3679	0.4833	0.7203	0.5594	0.3117	0.4826
MCTeamSR8	0.2776	0.5587	0.4044	0.4473	0.3667	0.2352
MCTeamMR24	0.3877	0.6628	0.5544	0.5777	0.4335	0.3753
MeSH Indexing New	0.4490	0.6452	0.7282	0.6589	0.4359	0.5005
MeSH Indexing Pre	0.4539	0.6780	0.7011	0.6628	0.4618	0.4827
MeSH Indexing Ref	0.4414	0.6107	0.7506	0.6520	0.4109	0.5117
UCD-CMd	0.1620	0.6877	0.1547	0.2279	0.3809	0.1102
UCD-CMr	0.2618	0.3284	0.6884	0.4280	0.2056	0.4011
UCD-CMp	0.1620	0.6877	0.1547	0.2279	0.3809	0.1102
UCD-CMgg	0.2621	0.3989	0.6159	0.4659	0.2165	0.3610
MeSH Indexing Add	0.4423	0.6104	0.7527	0.6529	0.4111	0.5134

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2377	0.2682	0.2569	0.2330	0.3212	0.3666	0.3171	0.2316	0.2440	0.1386
MTI First Line Index	0.5298	0.5876	0.5073	0.5141	0.5284	0.4999	0.4763	0.5800	0.4876	0.3648
MeSH Indexing	0.5225	0.4990	0.5796	0.5079	0.4526	0.4926	0.4574	0.4869	0.5636	0.3580
Wishart-S1	0.4497	0.4651	0.4769	0.4290	0.4287	0.4208	0.3803	0.4338	0.4669	0.2850
Wishart-S2	0.4415	0.4318	0.5041	0.4237	0.3954	0.4503	0.3973	0.3995	0.4934	0.2795
Wishart-S3	0.4363	0.4111	0.5202	0.4191	0.3789	0.4693	0.4078	0.3815	0.5096	0.2754
Wishart-S5	0.4623	0.4793	0.4727	0.4543	0.4865	0.3873	0.3699	0.4791	0.4467	0.3044
cole_hce1	0.3989	0.3510	0.4871	0.3911	0.2656	0.3796	0.3329	0.3510	0.4620	0.2531
cole_hce2	0.4732	0.4163	0.5629	0.4600	0.3786	0.4206	0.3822	0.4163	0.5480	0.3134
utai_rebayct	0.2799	0.2462	0.3397	0.2733	0.4513	0.2006	0.1871	0.2462	0.3241	0.1647
MCTeamMM	0.4660	0.4395	0.5124	0.4539	0.4528	0.3590	0.3408	0.4395	0.4959	0.3098
MCTeamMM10	0.4595	0.5351	0.4228	0.4526	0.5649	0.2697	0.2730	0.5351	0.4025	0.3080
MCTeamSR	0.2071	0.2070	0.2255	0.2062	0.3613	0.3094	0.2872	0.2081	0.2060	0.1208
MCTeamSR8	0.1934	0.2450	0.1770	0.1967	0.4006	0.2424	0.2347	0.2456	0.1595	0.1157
MCTeamMR24	0.4196	0.3732	0.5095	0.4133	0.4164	0.4017	0.3765	0.3710	0.4829	0.2710
Galen	0.5549	0.5450	0.5796	0.5406	0.5489	0.4717	0.4469	0.5496	0.5604	0.3874
MeSH Indexing New	0.5198	0.4797	0.6057	0.5088	0.4326	0.5133	0.4669	0.4654	0.5886	0.3581
MeSH Indexing Pre	0.5234	0.4977	0.5914	0.5112	0.4574	0.4968	0.4599	0.4832	0.5709	0.3602
MeSH Indexing Ref	0.5107	0.4366	0.6331	0.4967	0.4143	0.5358	0.4805	0.4370	0.6143	0.3460
UCD-CMr	0.1883	0.1390	0.3690	0.1894	0.2351	0.4936	0.3954	0.1288	0.3500	0.1077
UCD-CMp	0.1181	0.4717	0.0773	0.1246	0.4163	0.0808	0.0798	0.4083	0.0690	0.0718
UCD-CMgg	0.2173	0.1747	0.3176	0.2128	0.3317	0.4240	0.3627	0.1697	0.3019	0.1238
MeSH Indexing Add	0.5119	0.4376	0.6345	0.4979	0.4138	0.5376	0.4818	0.4380	0.6157	0.3471

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2981	0.5347	0.5241	0.4917	0.3383	0.3009
MTI First Line Index	0.4577	0.7456	0.6655	0.6721	0.5179	0.4443
MeSH Indexing	0.4610	0.6752	0.7073	0.6593	0.4732	0.4911
Wishart-S1	0.3990	0.6586	0.6530	0.6098	0.4198	0.4284
Wishart-S2	0.3987	0.6255	0.6865	0.6111	0.3981	0.4498
Wishart-S3	0.3973	0.6030	0.7044	0.6075	0.3862	0.4616
Wishart-S5	0.4118	0.6630	0.6501	0.6297	0.4277	0.4236
cole_hce1	0.3874	0.5366	0.6799	0.5751	0.3584	0.4560
cole_hce2	0.4330	0.5921	0.7198	0.6248	0.4079	0.4972
utai_rebayct	0.2852	0.4800	0.5292	0.4804	0.2575	0.3495
MCTeamMM	0.4181	0.6303	0.6408	0.6091	0.4240	0.4430
MCTeamMM10	0.4019	0.7283	0.5195	0.5766	0.4936	0.3657
MCTeamSR	0.2817	0.4527	0.5125	0.4591	0.2962	0.2887
MCTeamSR8	0.2781	0.5197	0.4207	0.4420	0.3486	0.2456
MCTeamMR24	0.3967	0.5507	0.6718	0.5783	0.3774	0.4542
Galen	0.4746	0.6942	0.7138	0.6804	0.4851	0.4962
MeSH Indexing New	0.4634	0.6553	0.7370	0.6641	0.4583	0.5115
MeSH Indexing Pre	0.4652	0.6744	0.7168	0.6639	0.4730	0.4995
MeSH Indexing Ref	0.4600	0.6101	0.7725	0.6575	0.4263	0.5376
UCD-CMr	0.2534	0.3147	0.6803	0.4132	0.1977	0.3973
UCD-CMp	0.1503	0.6420	0.1364	0.2030	0.3542	0.1015
UCD-CMgg	0.2538	0.3803	0.6110	0.4480	0.2104	0.3531
MeSH Indexing Add	0.4611	0.6096	0.7743	0.6578	0.4272	0.5380

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2596	0.2620	0.2871	0.2537	0.3915	0.3909	0.3236	0.2427	0.2790	0.1520
MTI First Line Index	0.5571	0.6046	0.5392	0.5455	0.5586	0.5090	0.4796	0.5974	0.5219	0.3925
MeSH Indexing	0.5343	0.5221	0.5901	0.5279	0.4871	0.4615	0.4321	0.5076	0.5639	0.3739
cole_hce1	0.4108	0.3752	0.4835	0.4048	0.2808	0.3554	0.3084	0.3752	0.4539	0.2639
cole_hce2	0.4276	0.3905	0.5015	0.4208	0.3057	0.3678	0.3233	0.3905	0.4725	0.2781
utai_rebayct	0.3756	0.3430	0.4414	0.3699	0.3669	0.2870	0.2764	0.3430	0.4150	0.2357
MCTeamMM	0.4601	0.4519	0.4916	0.4512	0.4924	0.3289	0.3242	0.4519	0.4686	0.3063
MCTeamMM10	0.4519	0.5528	0.4075	0.4493	0.6038	0.2553	0.2761	0.5528	0.3822	0.3051
MCTeamSR	0.2254	0.2330	0.2385	0.2253	0.4636	0.3025	0.2906	0.2343	0.2172	0.1322
MCTeamSR8	0.2111	0.2807	0.1898	0.2165	0.5185	0.2418	0.2488	0.2814	0.1689	0.1275
Hippocrates	0.5605	0.5898	0.5317	0.5390	0.6334	0.3765	0.3791	0.6042	0.5228	0.3882
MCTeamMR24	0.4252	0.3940	0.4977	0.4211	0.4938	0.3775	0.3679	0.3915	0.4652	0.2764
Galen	0.5668	0.5515	0.5860	0.5506	0.5903	0.4178	0.4091	0.5606	0.5732	0.3964
MeSH Indexing New	0.5275	0.4950	0.6110	0.5211	0.4621	0.4776	0.4361	0.4810	0.5841	0.3671
MeSH Indexing Pre	0.5327	0.5167	0.5985	0.5275	0.4879	0.4669	0.4346	0.4993	0.5710	0.3729
MeSH Indexing Ref	0.5195	0.4599	0.6239	0.5082	0.4462	0.4883	0.4382	0.4608	0.5953	0.3545
UCD-CMr	0.1999	0.1421	0.4057	0.2007	0.3006	0.5091	0.3776	0.1341	0.3929	0.1145
UCD-CMp	0.1256	0.4522	0.0831	0.1296	0.4863	0.1030	0.1023	0.3852	0.0750	0.0747
UCD-CMgg	0.2357	0.1852	0.3474	0.2308	0.4041	0.4435	0.3684	0.1816	0.3355	0.1352
MeSH Indexing Add	0.5197	0.4601	0.6243	0.5085	0.4451	0.4885	0.4383	0.4610	0.5956	0.3546

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2649	0.2686	0.2863	0.2567	0.3825	0.3915	0.3243	0.2497	0.2822	0.1543
MTI First Line Index	0.5549	0.6031	0.5323	0.5437	0.5489	0.5039	0.4743	0.5975	0.5179	0.3892
MeSH Indexing	0.5355	0.5331	0.5805	0.5306	0.4889	0.4623	0.4334	0.5168	0.5556	0.3756
Wishart-S1	0.4668	0.5609	0.4226	0.4512	0.5507	0.3235	0.3195	0.5328	0.4154	0.3037
Wishart-S2	0.4678	0.5298	0.4452	0.4530	0.5184	0.3467	0.3330	0.5016	0.4383	0.3047
Wishart-S3	0.4652	0.4940	0.4691	0.4511	0.4884	0.3720	0.3465	0.4680	0.4624	0.3024
Wishart-S4	0.4626	0.4750	0.4825	0.4490	0.4732	0.3867	0.3539	0.4501	0.4758	0.3002
Wishart-S5	0.4652	0.5194	0.4501	0.4620	0.5314	0.3609	0.3548	0.5177	0.4224	0.3112
cole_hce1	0.4118	0.3831	0.4747	0.4068	0.2826	0.3566	0.3117	0.3831	0.4451	0.2651
cole_hce2	0.4824	0.4488	0.5486	0.4739	0.4134	0.3871	0.3553	0.4488	0.5214	0.3237
utai_rebayct	0.2986	0.2778	0.3366	0.2919	0.5692	0.1584	0.1629	0.2778	0.3227	0.1767
MCTeamMM	0.4653	0.4663	0.4888	0.4580	0.4929	0.3319	0.3250	0.4663	0.4644	0.3111
MCTeamMM10	0.3344	0.2541	0.5015	0.3231	0.2583	0.3892	0.2974	0.2540	0.4894	0.1992
MCTeamSR	0.2253	0.2381	0.2290	0.2236	0.4620	0.3077	0.2939	0.2391	0.2130	0.1308
MCTeamSR8	0.2027	0.1694	0.2551	0.1921	0.3586	0.3326	0.2912	0.1709	0.2491	0.1102
Hippocrates	0.5628	0.5963	0.5282	0.5419	0.6247	0.3865	0.3849	0.6100	0.5225	0.3895
MCTeamMR24	0.3340	0.2416	0.5898	0.3320	0.3682	0.5222	0.4232	0.2369	0.5661	0.2037
Galen	0.5699	0.5590	0.5828	0.5546	0.5871	0.4354	0.4216	0.5674	0.5725	0.3995
MeSH Indexing New	0.5291	0.5054	0.6009	0.5242	0.4627	0.4780	0.4369	0.4898	0.5752	0.3691
MeSH Indexing Pre	0.5364	0.5288	0.5914	0.5318	0.4902	0.4726	0.4400	0.5113	0.5642	0.3761
MeSH Indexing Ref	0.5225	0.4713	0.6148	0.5127	0.4480	0.4876	0.4388	0.4716	0.5856	0.3577
MeSH Indexing Add	0.5230	0.4717	0.6152	0.5132	0.4477	0.4879	0.4387	0.4721	0.5861	0.3581
main system	0.4618	0.6327	0.3766	0.4450	0.6417	0.2361	0.2579	0.6285	0.3650	0.3038

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3098	0.5385	0.5444	0.5109	0.3393	0.3147
MTI First Line Index	0.4660	0.7534	0.6589	0.6759	0.5197	0.4533
MeSH Indexing	0.4638	0.7035	0.6961	0.6705	0.4869	0.4807
Wishart-S1	0.3908	0.7367	0.5686	0.6029	0.4579	0.3732
Wishart-S2	0.3963	0.7123	0.5990	0.6129	0.4445	0.3917
Wishart-S3	0.4001	0.6830	0.6307	0.6195	0.4278	0.4125
Wishart-S4	0.4008	0.6666	0.6471	0.6211	0.4179	0.4237
Wishart-S5	0.4041	0.7026	0.6145	0.6285	0.4428	0.3984
cole_hce1	0.3876	0.5714	0.6397	0.5778	0.3790	0.4294
cole_hce2	0.4289	0.6278	0.6856	0.6296	0.4258	0.4668
utai_rebayct	0.2882	0.5306	0.4923	0.4868	0.2781	0.3264
MCTeamMM	0.4080	0.6541	0.6032	0.6001	0.4372	0.4130
MCTeamMM10	0.3302	0.4491	0.6868	0.5208	0.2798	0.4419
MCTeamSR	0.2840	0.5057	0.4889	0.4728	0.3186	0.2753
MCTeamSR8	0.2669	0.4070	0.5269	0.4352	0.2610	0.3017
Hippocrates	0.4623	0.7430	0.6465	0.6678	0.5120	0.4481
MCTeamMR24	0.3345	0.4134	0.7776	0.5214	0.2618	0.5044
Galen	0.4769	0.7101	0.7077	0.6876	0.4919	0.4912
MeSH Indexing New	0.4637	0.6784	0.7199	0.6703	0.4693	0.4977
MeSH Indexing Pre	0.4661	0.7023	0.7043	0.6732	0.4867	0.4866
MeSH Indexing Ref	0.4585	0.6480	0.7400	0.6657	0.4450	0.5101
MeSH Indexing Add	0.4588	0.6479	0.7397	0.6657	0.4449	0.5110
main system	0.3728	0.7965	0.4405	0.5292	0.5336	0.3127

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MeSH Indexing Add	0.5124	0.4671	0.5941	0.5022	0.4561	0.4736	0.4340	0.4672	0.5673	0.3484
main system	0.4083	0.7105	0.2930	0.3882	0.7419	0.1760	0.2006	0.7153	0.2857	0.2580
BioASQ_Baseline	0.2628	0.2698	0.2840	0.2581	0.3914	0.3798	0.3191	0.2502	0.2768	0.1544
MTI First Line Index	0.5589	0.6110	0.5333	0.5472	0.5645	0.4998	0.4761	0.6056	0.5189	0.3933
MeSH Indexing	0.5321	0.5329	0.5710	0.5265	0.4964	0.4491	0.4269	0.5180	0.5469	0.3717
Wishart-S1	0.4685	0.5229	0.4465	0.4549	0.5202	0.3401	0.3297	0.5041	0.4377	0.3059
Wishart-S2	0.5102	0.6351	0.4449	0.5045	0.6075	0.4056	0.4061	0.6350	0.4264	0.3508
Wishart-S3	0.4174	0.4671	0.3990	0.4119	0.4830	0.2945	0.2951	0.4746	0.3725	0.2722
Wishart-S4	0.4871	0.5989	0.4440	0.4914	0.5803	0.4832	0.4682	0.5929	0.4133	0.3402
Wishart-S5	0.5067	0.6328	0.4415	0.5016	0.6055	0.4026	0.4039	0.6346	0.4217	0.3483
cole_hce1	0.4090	0.3853	0.4638	0.4038	0.2859	0.3430	0.3054	0.3853	0.4357	0.2629
cole_hce2	0.4749	0.4475	0.5300	0.4661	0.4220	0.3734	0.3511	0.4475	0.5060	0.3175
utai_rebayct	0.2973	0.2801	0.3322	0.2915	0.5660	0.1559	0.1592	0.2801	0.3167	0.1764
MCTeamMM	0.2114	0.2933	0.1843	0.2166	0.5167	0.2349	0.2406	0.2936	0.1652	0.1269
MCTeamMM10	0.2114	0.2933	0.1843	0.2166	0.5167	0.2349	0.2406	0.2936	0.1652	0.1269
MCTeamSR	0.2273	0.2434	0.2322	0.2277	0.4590	0.2946	0.2835	0.2436	0.2130	0.1334
MCTeamSR8	0.2050	0.1737	0.2592	0.1963	0.3626	0.3243	0.2860	0.1738	0.2498	0.1128
Hippocrates	0.5654	0.5968	0.5399	0.5490	0.6181	0.4055	0.4043	0.6065	0.5295	0.3951
MCTeamMR24	0.2202	0.1719	0.3272	0.2161	0.3642	0.4118	0.3438	0.1695	0.3142	0.1247
Galen	0.5689	0.5643	0.5786	0.5549	0.5995	0.4202	0.4138	0.5719	0.5661	0.3995
MeSH Indexing New	0.5261	0.5037	0.5938	0.5204	0.4726	0.4706	0.4362	0.4889	0.5694	0.3661
MeSH Indexing Pre	0.5305	0.5237	0.5811	0.5255	0.4921	0.4577	0.4302	0.5075	0.5558	0.3708
MeSH Indexing Ref	0.5201	0.4747	0.6019	0.5103	0.4599	0.4765	0.4374	0.4747	0.5750	0.3560

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MeSH Indexing Add	0.4512	0.6548	0.7261	0.6641	0.4427	0.4961
main system	0.3161	0.8503	0.3347	0.4453	0.5595	0.2385
BioASQ_Baseline	0.3103	0.5514	0.5429	0.5173	0.3411	0.3127
MTI First Line Index	0.4718	0.7637	0.6584	0.6803	0.5290	0.4564
MeSH Indexing	0.4615	0.7098	0.6881	0.6706	0.4876	0.4757
Wishart-S1	0.3970	0.7130	0.5974	0.6159	0.4431	0.3919
Wishart-S2	0.4285	0.7906	0.5959	0.6537	0.5116	0.3919
Wishart-S3	0.3784	0.6736	0.5771	0.5964	0.4222	0.3680
Wishart-S4	0.4261	0.7674	0.6163	0.6587	0.5112	0.3887
Wishart-S5	0.4274	0.7887	0.5934	0.6518	0.5119	0.3898
cole_hce1	0.3837	0.5761	0.6335	0.5787	0.3793	0.4205
cole_hce2	0.4248	0.6338	0.6727	0.6280	0.4262	0.4567
utai_rebayct	0.2883	0.5437	0.4922	0.4937	0.2806	0.3229
MCTeamMM	0.2761	0.5955	0.3965	0.4518	0.3831	0.2305
MCTeamMM10	0.2761	0.5955	0.3965	0.4518	0.3831	0.2305
MCTeamSR	0.2862	0.5193	0.4884	0.4801	0.3241	0.2759
MCTeamSR8	0.2671	0.4200	0.5282	0.4431	0.2623	0.3018
Hippocrates	0.4709	0.7511	0.6574	0.6780	0.5150	0.4603
MCTeamMR24	0.2734	0.3995	0.6279	0.4687	0.2369	0.3550
Galen	0.4788	0.7232	0.7017	0.6909	0.4976	0.4891
MeSH Indexing New	0.4612	0.6834	0.7156	0.6714	0.4681	0.4949
MeSH Indexing Pre	0.4624	0.7040	0.6984	0.6729	0.4839	0.4817
MeSH Indexing Ref	0.4564	0.6575	0.7292	0.6675	0.4476	0.5017

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2652	0.2725	0.2892	0.2598	0.3864	0.3862	0.3216	0.2513	0.2807	0.1560
MTI First Line Index	0.5467	0.5970	0.5253	0.5350	0.5481	0.5084	0.4788	0.5889	0.5102	0.3814
MeSH Indexing	0.5224	0.5024	0.5944	0.5188	0.4807	0.4752	0.4386	0.4852	0.5658	0.3640
Wishart-S1	0.4586	0.5074	0.4489	0.4439	0.5051	0.3544	0.3399	0.4770	0.4415	0.2975
Wishart-S2	0.5021	0.6173	0.4450	0.4981	0.5905	0.4108	0.4075	0.6146	0.4244	0.3463
Wishart-S3	0.4114	0.4462	0.4074	0.4076	0.4687	0.2987	0.2960	0.4510	0.3783	0.2693
Wishart-S4	0.4853	0.5859	0.4490	0.4888	0.5607	0.4923	0.4708	0.5778	0.4183	0.3381
Wishart-S5	0.4991	0.6141	0.4421	0.4952	0.5869	0.4105	0.4078	0.6130	0.4209	0.3436
cole_hce1	0.4001	0.3647	0.4712	0.3948	0.2781	0.3513	0.3076	0.3647	0.4431	0.2555
cole_hce2	0.4657	0.4245	0.5418	0.4572	0.4150	0.3895	0.3593	0.4245	0.5158	0.3093
utai_rebayct	0.2908	0.2650	0.3378	0.2846	0.5497	0.1650	0.1699	0.2650	0.3220	0.1717
MCTeamMM	0.4496	0.4406	0.4825	0.4419	0.4923	0.3329	0.3282	0.4406	0.4589	0.2974
MCTeamMM10	0.3630	0.5518	0.2937	0.3621	0.5961	0.1975	0.2193	0.5454	0.2720	0.2320
MCTeamSR	0.2187	0.1697	0.3284	0.2132	0.3541	0.4122	0.3415	0.1673	0.3157	0.1232
MCTeamSR8	0.2032	0.1695	0.2579	0.1919	0.3555	0.3225	0.2853	0.1716	0.2491	0.1104
Hippocrates	0.5553	0.5796	0.5330	0.5362	0.6037	0.4156	0.4109	0.5894	0.5249	0.3842
MCTeamMR24	0.3281	0.2370	0.5840	0.3261	0.3650	0.5118	0.4181	0.2320	0.5602	0.1996
Galen	0.5582	0.5472	0.5727	0.5430	0.5870	0.4294	0.4189	0.5544	0.5621	0.3887
MeSH Indexing New	0.5159	0.4810	0.6093	0.5121	0.4628	0.4856	0.4408	0.4644	0.5804	0.3578
MeSH Indexing Pre	0.5233	0.5082	0.5876	0.5193	0.4842	0.4687	0.4350	0.4909	0.5602	0.3645
MeSH Indexing Ref	0.5236	0.5083	0.5873	0.5195	0.4862	0.4682	0.4354	0.4915	0.5601	0.3649
MeSH Indexing Add	0.5264	0.5231	0.5747	0.5221	0.4994	0.4596	0.4327	0.5064	0.5480	0.3675

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2651	0.2686	0.2943	0.2602	0.3916	0.3889	0.3210	0.2486	0.2839	0.1561
MTI First Line Index	0.5555	0.6033	0.5410	0.5460	0.5622	0.5100	0.4816	0.5966	0.5197	0.3922
MeSH Indexing	0.5293	0.5174	0.5942	0.5249	0.4942	0.4610	0.4289	0.5007	0.5614	0.3700
Wishart-S1	0.4621	0.5069	0.4550	0.4504	0.5153	0.3392	0.3267	0.4846	0.4415	0.3022
Wishart-S2	0.5051	0.6220	0.4506	0.5016	0.6019	0.4070	0.4068	0.6202	0.4260	0.3494
Wishart-S3	0.4090	0.4516	0.3987	0.4038	0.4736	0.2832	0.2845	0.4603	0.3680	0.2665
Wishart-S4	0.4870	0.5902	0.4529	0.4922	0.5763	0.4952	0.4755	0.5843	0.4175	0.3418
Wishart-S5	0.5019	0.6168	0.4482	0.4984	0.5958	0.4065	0.4063	0.6167	0.4230	0.3465
cole_hce1	0.4080	0.3795	0.4745	0.4030	0.2817	0.3497	0.3065	0.3795	0.4411	0.2623
cole_hce2	0.4736	0.4405	0.5421	0.4650	0.4190	0.3808	0.3538	0.4405	0.5121	0.3163
utai_rebayct	0.2931	0.2726	0.3351	0.2873	0.5670	0.1529	0.1589	0.2726	0.3169	0.1734
MCTeamMM	0.4590	0.4598	0.4854	0.4514	0.4958	0.3240	0.3212	0.4599	0.4582	0.3058
MCTeamMM10	0.2301	0.2131	0.2701	0.2275	0.4365	0.3459	0.3153	0.2128	0.2503	0.1329
MCTeamSR	0.2273	0.2407	0.2363	0.2274	0.4771	0.3015	0.2909	0.2405	0.2155	0.1334
MCTeamSR8	0.2301	0.2131	0.2701	0.2275	0.4365	0.3459	0.3153	0.2128	0.2503	0.1329
Hippocrates	0.5672	0.5922	0.5532	0.5519	0.6099	0.4080	0.4063	0.6003	0.5376	0.3970
MCTeamMR24	0.4590	0.4598	0.4854	0.4514	0.4958	0.3240	0.3212	0.4599	0.4582	0.3058
Galen	0.5674	0.5560	0.5891	0.5533	0.5941	0.4201	0.4131	0.5630	0.5718	0.3974
MeSH Indexing New	0.5233	0.4941	0.6131	0.5185	0.4709	0.4782	0.4349	0.4772	0.5792	0.3637
MeSH Indexing Pre	0.5296	0.5138	0.6002	0.5254	0.4896	0.4665	0.4316	0.4965	0.5673	0.3703
MeSH Indexing Ref	0.5293	0.5178	0.5946	0.5250	0.4930	0.4627	0.4303	0.5006	0.5616	0.3701
MeSH Indexing Add	0.5299	0.5103	0.5884	0.5227	0.5020	0.4503	0.4248	0.5106	0.5507	0.3673
main system	0.4554	0.6245	0.3737	0.4382	0.6420	0.2372	0.2622	0.6210	0.3596	0.2985

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3131	0.5441	0.5519	0.5154	0.3405	0.3207
MTI First Line Index	0.4672	0.7526	0.6671	0.6785	0.5192	0.4582
MeSH Indexing	0.4600	0.6958	0.7121	0.6725	0.4765	0.4870
Wishart-S1	0.3958	0.7000	0.6181	0.6204	0.4339	0.3987
Wishart-S2	0.4257	0.7785	0.6072	0.6545	0.5035	0.3953
Wishart-S3	0.3741	0.6637	0.5857	0.5939	0.4148	0.3689
Wishart-S4	0.4263	0.7566	0.6260	0.6587	0.5046	0.3942
Wishart-S5	0.4251	0.7752	0.6072	0.6538	0.5022	0.3944
cole_hce1	0.3836	0.5732	0.6462	0.5810	0.3758	0.4261
cole_hce2	0.4234	0.6273	0.6894	0.6300	0.4195	0.4638
utai_rebayct	0.2843	0.5351	0.4982	0.4910	0.2745	0.3238
MCTeamMM	0.4039	0.6524	0.6108	0.6021	0.4320	0.4121
MCTeamMM10	0.2891	0.4628	0.5540	0.4807	0.2923	0.3103
MCTeamSR	0.2855	0.5091	0.4944	0.4766	0.3203	0.2788
MCTeamSR8	0.2891	0.4628	0.5540	0.4807	0.2923	0.3103
Hippocrates	0.4713	0.7423	0.6773	0.6831	0.5104	0.4692
MCTeamMR24	0.4039	0.6524	0.6108	0.6021	0.4320	0.4121
Galen	0.4776	0.7135	0.7179	0.6923	0.4919	0.4965
MeSH Indexing New	0.4591	0.6744	0.7325	0.6712	0.4619	0.5013
MeSH Indexing Pre	0.4607	0.6936	0.7171	0.6740	0.4744	0.4911
MeSH Indexing Ref	0.4603	0.6960	0.7121	0.6726	0.4767	0.4878
MeSH Indexing Add	0.4593	0.6910	0.7102	0.6723	0.4723	0.4841
main system	0.3706	0.7920	0.4472	0.5335	0.5285	0.3124

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2587	0.2672	0.2848	0.2532	0.3787	0.3673	0.3064	0.2451	0.2740	0.1513
MTI First Line Index	0.5559	0.6066	0.5398	0.5471	0.5619	0.4982	0.4725	0.5999	0.5179	0.3935
MeSH Indexing	0.5259	0.5145	0.5904	0.5215	0.4869	0.4549	0.4237	0.4970	0.5583	0.3677
Wishart-S1	0.4660	0.5183	0.4528	0.4541	0.5193	0.3362	0.3256	0.4953	0.4401	0.3060
Wishart-S2	0.5066	0.6271	0.4533	0.5057	0.6007	0.4006	0.3985	0.6247	0.4260	0.3530
Wishart-S3	0.4139	0.4541	0.4109	0.4104	0.4759	0.2902	0.2880	0.4607	0.3758	0.2713
Wishart-S4	0.4915	0.5972	0.4589	0.4984	0.5762	0.4813	0.4640	0.5906	0.4209	0.3474
Wishart-S5	0.5039	0.6233	0.4506	0.5027	0.5974	0.3995	0.3980	0.6235	0.4228	0.3504
cole_hce1	0.4050	0.3749	0.4736	0.3992	0.2913	0.3465	0.3087	0.3749	0.4404	0.2591
cole_hce2	0.4678	0.4330	0.5385	0.4584	0.4194	0.3770	0.3492	0.4330	0.5087	0.3106
utai_rebayct	0.2898	0.2683	0.3357	0.2843	0.5407	0.1512	0.1542	0.2683	0.3151	0.1713
Panacea	0.5641	0.5509	0.5847	0.5505	0.5838	0.4232	0.4122	0.5598	0.5685	0.3949
MCTeamMM	0.4528	0.4512	0.4817	0.4444	0.5008	0.3217	0.3171	0.4512	0.4543	0.2998
MCTeamMM10	0.3047	0.2933	0.3474	0.2944	0.3121	0.2773	0.2476	0.2876	0.3239	0.1812
MCTeamSR	0.2282	0.2429	0.2392	0.2284	0.4572	0.2913	0.2814	0.2411	0.2165	0.1341
MCTeamSR8	0.1968	0.2076	0.2069	0.1908	0.3742	0.2480	0.2344	0.2030	0.1909	0.1105
Hippocrates	0.5630	0.5838	0.5474	0.5471	0.6036	0.4030	0.3984	0.5948	0.5344	0.3933
MCTeamMR24	0.4528	0.4512	0.4817	0.4444	0.5008	0.3217	0.3171	0.4512	0.4543	0.2998
Galen	0.5631	0.5502	0.5832	0.5492	0.5852	0.4174	0.4067	0.5592	0.5672	0.3938
MeSH Indexing New	0.5213	0.4964	0.6050	0.5168	0.4682	0.4670	0.4286	0.4790	0.5719	0.3631
MeSH Indexing Pre	0.5238	0.5076	0.5945	0.5191	0.4817	0.4573	0.4239	0.4904	0.5621	0.3655
MeSH Indexing Ref	0.5262	0.5150	0.5904	0.5218	0.4876	0.4549	0.4239	0.4974	0.5584	0.3679
MeSH Indexing Add	0.5237	0.5018	0.5850	0.5156	0.4935	0.4465	0.4194	0.5019	0.5474	0.3612

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3118	0.5455	0.5410	0.5119	0.3438	0.3144
MTI First Line Index	0.4626	0.7513	0.6562	0.6723	0.5172	0.4488
MeSH Indexing	0.4609	0.6831	0.7118	0.6683	0.4700	0.4919
Wishart-S1	0.3945	0.6929	0.6057	0.6087	0.4354	0.3961
Wishart-S2	0.4257	0.7743	0.5972	0.6478	0.5034	0.3920
Wishart-S3	0.3799	0.6533	0.5889	0.5938	0.4124	0.3783
Wishart-S4	0.4250	0.7543	0.6192	0.6543	0.5029	0.3913
Wishart-S5	0.4246	0.7725	0.5963	0.6469	0.5027	0.3903
cole_hce1	0.3828	0.5602	0.6420	0.5740	0.3695	0.4288
cole_hce2	0.4221	0.6117	0.6881	0.6231	0.4117	0.4671
utai_rebayct	0.2860	0.5209	0.5090	0.4916	0.2695	0.3324
MCTeamMM	0.4013	0.6351	0.6066	0.5942	0.4213	0.4137
MCTeamMM10	0.3162	0.7395	0.3951	0.4786	0.4481	0.2662
MCTeamSR	0.2742	0.3846	0.6216	0.4548	0.2378	0.3564
MCTeamSR8	0.2652	0.4028	0.5175	0.4272	0.2618	0.3005
Hippocrates	0.4621	0.7294	0.6625	0.6698	0.5011	0.4566
MCTeamMR24	0.3311	0.4075	0.7750	0.5158	0.2591	0.5005
Galen	0.4704	0.7025	0.7054	0.6822	0.4850	0.4851
MeSH Indexing New	0.4584	0.6620	0.7290	0.6655	0.4548	0.5023
MeSH Indexing Pre	0.4594	0.6874	0.7045	0.6665	0.4723	0.4863
MeSH Indexing Ref	0.4598	0.6876	0.7047	0.6672	0.4723	0.4866
MeSH Indexing Add	0.4594	0.7006	0.6909	0.6665	0.4820	0.4765

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3065	0.5442	0.5421	0.5088	0.3365	0.3138
MTI First Line Index	0.4709	0.7584	0.6679	0.6822	0.5265	0.4598
MeSH Indexing	0.4580	0.6965	0.7095	0.6731	0.4740	0.4840
Wishart-S1	0.3983	0.7122	0.6142	0.6252	0.4411	0.3967
Wishart-S2	0.4305	0.7843	0.6083	0.6590	0.5095	0.3973
Wishart-S3	0.3787	0.6638	0.5942	0.5993	0.4145	0.3767
Wishart-S4	0.4323	0.7627	0.6300	0.6640	0.5128	0.3995
Wishart-S5	0.4297	0.7822	0.6064	0.6570	0.5088	0.3962
cole_hce1	0.3808	0.5691	0.6450	0.5779	0.3726	0.4243
cole_hce2	0.4195	0.6230	0.6888	0.6273	0.4150	0.4608
utai_rebayct	0.2802	0.5265	0.4982	0.4868	0.2688	0.3229
Panacea	0.4745	0.7113	0.7165	0.6927	0.4866	0.4922
MCTeamMM	0.3990	0.6512	0.6062	0.5993	0.4253	0.4079
MCTeamMM10	0.3006	0.5141	0.5308	0.4888	0.3153	0.3223
MCTeamSR	0.2869	0.5134	0.4965	0.4785	0.3217	0.2807
MCTeamSR8	0.2587	0.4741	0.4309	0.4192	0.3056	0.2497
Hippocrates	0.4674	0.7371	0.6740	0.6813	0.5034	0.4639
MCTeamMR24	0.3990	0.6512	0.6062	0.5993	0.4253	0.4079
Galen	0.4725	0.7100	0.7141	0.6909	0.4852	0.4892
MeSH Indexing New	0.4567	0.6779	0.7262	0.6713	0.4607	0.4955
MeSH Indexing Pre	0.4572	0.6909	0.7126	0.6715	0.4699	0.4876
MeSH Indexing Ref	0.4578	0.6956	0.7086	0.6718	0.4741	0.4837
MeSH Indexing Add	0.4542	0.6857	0.7079	0.6695	0.4642	0.4817

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2649	0.2765	0.2817	0.2583	0.4013	0.3810	0.3196	0.2549	0.2758	0.1553
MTI First Line Index	0.5435	0.6076	0.5143	0.5341	0.5624	0.5006	0.4761	0.5982	0.4980	0.3801
MeSH Indexing	0.5240	0.5167	0.5802	0.5205	0.4914	0.4585	0.4285	0.4985	0.5523	0.3653
Wishart-S1	0.4591	0.5482	0.4227	0.4470	0.5454	0.3251	0.3216	0.5191	0.4115	0.3002
Wishart-S2	0.4982	0.6303	0.4348	0.4960	0.6071	0.4047	0.4064	0.6273	0.4132	0.3435
Wishart-S3	0.4122	0.4560	0.4012	0.4088	0.4894	0.2977	0.2982	0.4623	0.3718	0.2700
Wishart-S4	0.4851	0.5977	0.4422	0.4895	0.5772	0.4872	0.4710	0.5897	0.4120	0.3386
Wishart-S5	0.4961	0.6257	0.4333	0.4936	0.6031	0.4058	0.4079	0.6243	0.4116	0.3415
cole_hce1	0.3986	0.3703	0.4602	0.3939	0.2807	0.3457	0.3002	0.3703	0.4317	0.2547
cole_hce2	0.4663	0.4331	0.5302	0.4580	0.4146	0.3768	0.3493	0.4331	0.5050	0.3098
utai_rebayct	0.2902	0.2696	0.3289	0.2844	0.5660	0.1573	0.1638	0.2696	0.3143	0.1716
Panacea	0.5577	0.5531	0.5670	0.5444	0.5952	0.4127	0.4066	0.5605	0.5550	0.3893
MCTeamMM	0.4467	0.4469	0.4692	0.4395	0.4902	0.3173	0.3149	0.4469	0.4466	0.2953
MCTeamMM10	0.3047	0.2913	0.3401	0.2933	0.3212	0.2832	0.2540	0.2888	0.3225	0.1803
MCTeamSR	0.2254	0.2387	0.2295	0.2241	0.4790	0.3011	0.2914	0.2394	0.2129	0.1315
MCTeamSR8	0.1958	0.2020	0.1997	0.1874	0.3986	0.2593	0.2452	0.2028	0.1892	0.1087
Hippocrates	0.0009	0.0009	0.0009	0.0009	0.0015	0.0011	0.0009	0.0009	0.0009	0.0004
MCTeamMR24	0.3297	0.2412	0.5707	0.3278	0.3819	0.5064	0.4139	0.2357	0.5486	0.2010
Galen	0.0010	0.0009	0.0009	0.0009	0.0014	0.0012	0.0009	0.0010	0.0010	0.0005
MeSH Indexing New	0.5174	0.4890	0.6016	0.5137	0.4656	0.4752	0.4322	0.4723	0.5721	0.3587
MeSH Indexing Pre	0.5153	0.4639	0.6108	0.5068	0.4595	0.4823	0.4352	0.4639	0.5794	0.3517
MeSH Indexing Ref	0.5410	0.5619	0.5436	0.5302	0.5348	0.4440	0.4259	0.5533	0.5293	0.3749
MeSH Indexing Add	0.5164	0.4649	0.6129	0.5082	0.4552	0.4890	0.4380	0.4649	0.5806	0.3527

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3114	0.5548	0.5340	0.5122	0.3470	0.3115
MTI First Line Index	0.4606	0.7632	0.6433	0.6710	0.5254	0.4401
MeSH Indexing	0.4575	0.7021	0.6973	0.6709	0.4772	0.4779
Wishart-S1	0.3899	0.7366	0.5700	0.6050	0.4550	0.3723
Wishart-S2	0.4227	0.7889	0.5857	0.6468	0.5116	0.3829
Wishart-S3	0.3786	0.6713	0.5821	0.5983	0.4183	0.3707
Wishart-S4	0.4260	0.7674	0.6083	0.6539	0.5134	0.3871
Wishart-S5	0.4230	0.7861	0.5863	0.6463	0.5117	0.3832
cole_hce1	0.3796	0.5704	0.6341	0.5759	0.3710	0.4197
cole_hce2	0.4184	0.6257	0.6762	0.6256	0.4145	0.4556
utai_rebayct	0.2828	0.5286	0.4903	0.4862	0.2715	0.3222
Panacea	0.4717	0.7187	0.6979	0.6877	0.4914	0.4804
MCTeamMM	0.3970	0.6491	0.5925	0.5936	0.4251	0.4014
MCTeamMM10	0.3039	0.5176	0.5287	0.4914	0.3180	0.3232
MCTeamSR	0.2851	0.5038	0.4830	0.4694	0.3201	0.2766
MCTeamSR8	0.2580	0.4702	0.4289	0.4184	0.3050	0.2464
Hippocrates	0.1074	0.1150	0.1633	0.1295	0.0953	0.1302
MCTeamMR24	0.3312	0.4179	0.7661	0.5216	0.2619	0.4930
Galen	0.1079	0.1156	0.1658	0.1307	0.0953	0.1317
MeSH Indexing New	0.4564	0.6758	0.7229	0.6702	0.4585	0.4951
MeSH Indexing Pre	0.4548	0.6554	0.7365	0.6695	0.4414	0.5044
MeSH Indexing Ref	0.4599	0.7431	0.6594	0.6723	0.5046	0.4538
MeSH Indexing Add	0.4546	0.6527	0.7403	0.6698	0.4389	0.5074

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2588	0.2631	0.2868	0.2551	0.3916	0.3850	0.3186	0.2438	0.2757	0.1525
MTI First Line Index	0.5524	0.5984	0.5379	0.5436	0.5621	0.5141	0.4851	0.5939	0.5164	0.3891
MeSH Indexing	0.5165	0.4660	0.6161	0.5068	0.4700	0.4837	0.4366	0.4660	0.5793	0.3521
Wishart-S1	0.4628	0.5421	0.4314	0.4512	0.5554	0.3192	0.3169	0.5195	0.4173	0.3033
Wishart-S2	0.5061	0.6235	0.4527	0.5044	0.6065	0.4123	0.4104	0.6226	0.4264	0.3513
Wishart-S3	0.4059	0.4487	0.3996	0.4018	0.4803	0.2879	0.2877	0.4562	0.3656	0.2650
Wishart-S4	0.4827	0.5855	0.4520	0.4901	0.5779	0.4980	0.4781	0.5805	0.4131	0.3392
Wishart-S5	0.5029	0.6194	0.4496	0.5009	0.5996	0.4097	0.4080	0.6205	0.4228	0.3483
cole_hce1	0.4020	0.3742	0.4671	0.3964	0.2855	0.3507	0.3080	0.3742	0.4342	0.2566
cole_hce2	0.4020	0.3742	0.4671	0.3964	0.2855	0.3507	0.3080	0.3742	0.4342	0.2566
utai_rebayct	0.2954	0.2750	0.3374	0.2891	0.5692	0.1565	0.1615	0.2750	0.3190	0.1748
Panacea	0.5672	0.5533	0.5884	0.5528	0.6007	0.4287	0.4200	0.5626	0.5719	0.3972
MCTeamMM	0.4529	0.4542	0.4776	0.4442	0.5045	0.3267	0.3238	0.4542	0.4517	0.2996
MCTeamMM10	0.3053	0.2966	0.3439	0.2935	0.3132	0.2800	0.2515	0.2910	0.3212	0.1803
MCTeamSR	0.2097	0.2870	0.1873	0.2156	0.5268	0.2411	0.2488	0.2868	0.1652	0.1269
MCTeamSR8	0.1928	0.1998	0.2020	0.1860	0.3843	0.2539	0.2389	0.1998	0.1863	0.1077
Hippocrates	0.5658	0.5889	0.5498	0.5498	0.6186	0.4104	0.4076	0.6002	0.5352	0.3954
MCTeamMR24	0.3267	0.2352	0.5802	0.3230	0.3799	0.5144	0.4191	0.2319	0.5522	0.1972
Galen	0.5661	0.5527	0.5869	0.5515	0.6022	0.4232	0.4149	0.5617	0.5705	0.3961
MeSH Indexing New	0.5237	0.5277	0.5914	0.5232	0.4934	0.4640	0.4309	0.4907	0.5614	0.3685
MeSH Indexing Pre	0.5308	0.5496	0.5765	0.5286	0.5046	0.4625	0.4352	0.5122	0.5508	0.3738
MeSH Indexing Ref	0.5268	0.6301	0.4949	0.5251	0.5937	0.3783	0.3860	0.6089	0.4643	0.3703
MeSH Indexing Add	0.5253	0.5315	0.5887	0.5251	0.4933	0.4620	0.4301	0.4949	0.5596	0.3702
utai_rebayct_2	0.3350	0.3118	0.3858	0.3290	0.5820	0.1741	0.1841	0.3118	0.3618	0.2035

BioASQ Participants Area

Test Results for Task 1a

Evaluation Overview

+ Dry-run test

Annotated articles:6391/7864

Flat Measures

Hierarchical Measures

+ Test batch 1, week 1

Annotated articles:1592/1934

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:743/830

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:762/790

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:722/2233

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:5901/6562

Flat Measures

Hierarchical Measures

+ Test batch 1, week 6

Annotated articles:4152/4414

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:4068/4869

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:4788/5551

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:6081/7144

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:3908/4623

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:6651/8233

Flat Measures

Hierarchical Measures

+ Test batch 2, week 6

Annotated articles:6762/8381

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:6022/7578

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:3918/10139

Flat Measures