BioASQ Participants Area

Evaluation Overview

In the following table you can see the number of the articles for each test set that has been released. In parentheses, there is the number of the articles that have been annotated by the curators for each test set.

	Batch 1	Batch 2	Batch 3
Week 1	4440 (3319)	4085 (3422)	4342 (3009)
Week 2	4721 (3734)	3496 (2788)	8840 (5883)
Week 3	4802 (3884)	4524 (3274)	3702 (2860)
Week 4	3579 (2431)	5407 (3923)	4726 (3252)
Week 5	5299 (3693)	5454 (3666)	4533 (3252)

Test Results for Task 2a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Dry-run test

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2510	0.2516	0.2893	0.2499	0.3592	0.3845	0.3182	0.2296	0.2767	0.1484
MTI First Line Index	0.5624	0.6087	0.5473	0.5535	0.5675	0.4858	0.4633	0.6023	0.5274	0.3998
MeSH Indexing	0.5675	0.6155	0.5629	0.5671	0.5722	0.4671	0.4551	0.6132	0.5281	0.4116
Default MTI	0.5673	0.5721	0.5908	0.5595	0.5381	0.5273	0.4889	0.5647	0.5699	0.4040
Macro	0.1033	0.6722	0.0639	0.1149	0.6722	0.0001	0.0001	0.6722	0.0560	0.0639
Micro	0.1033	0.6722	0.0639	0.1149	0.6722	0.0001	0.0001	0.6722	0.0560	0.0639
PerExample	0.1033	0.6722	0.0639	0.1149	0.6722	0.0001	0.0001	0.6722	0.0560	0.0639
spoon-spoon	0.3678	0.3562	0.4103	0.3557	0.4423	0.3241	0.3045	0.3402	0.4002	0.2271
fork-fork	0.3755	0.4382	0.3531	0.3609	0.5331	0.2736	0.2750	0.4167	0.3416	0.2328
spork-spork	0.3716	0.3891	0.3837	0.3586	0.4775	0.2996	0.2899	0.3703	0.3729	0.2301
EO_Sys1	0.4538	0.5471	0.5060	0.4963	0.3787	0.2869	0.2420	0.4118	0.5054	0.3469
Limited sample	0.4506	0.5467	0.4992	0.4923	0.3701	0.2806	0.2370	0.4104	0.4997	0.3433
Mixing with MTI	0.5060	0.4937	0.6908	0.5550	0.4305	0.5841	0.4773	0.4041	0.6767	0.3978

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3044	0.5320	0.5566	0.5120	0.3261	0.3181
MTI First Line Index	0.4694	0.7532	0.6736	0.6829	0.5203	0.4603
MeSH Indexing	0.4784	0.7601	0.6833	0.6946	0.5313	0.4656
Default MTI	0.4782	0.7215	0.7210	0.6954	0.4989	0.4938
Macro	0.0845	0.6955	0.0521	0.0951	0.2803	0.0512
Micro	0.0845	0.6955	0.0521	0.0951	0.2803	0.0512
PerExample	0.0845	0.6955	0.0521	0.0951	0.2803	0.0512
spoon-spoon	0.3393	0.5673	0.5425	0.5198	0.3524	0.3694
fork-fork	0.3337	0.6555	0.4522	0.4967	0.4135	0.3141
spork-spork	0.3362	0.5994	0.5009	0.5098	0.3742	0.3438
EO_Sys1	0.4247	0.7087	0.6104	0.6173	0.4797	0.4214
Limited sample	0.4209	0.7075	0.6058	0.6141	0.4783	0.4162
Mixing with MTI	0.4791	0.6548	0.7983	0.6946	0.4454	0.5636

+ Test batch 1, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2549	0.2576	0.2864	0.2530	0.3769	0.3786	0.3158	0.2375	0.2750	0.1501
MTI First Line Index	0.5605	0.6213	0.5368	0.5557	0.5803	0.4745	0.4558	0.6132	0.5162	0.3994
Default MTI	0.5704	0.5895	0.5797	0.5659	0.5490	0.5166	0.4813	0.5828	0.5585	0.4082
FU System	0.4499	0.3444	0.6714	0.4398	0.3393	0.4932	0.3889	0.3441	0.6496	0.2913
FU_System_t25	0.4529	0.3467	0.6763	0.4428	0.3428	0.5008	0.3961	0.3464	0.6538	0.2937
FU_System_t15	0.5056	0.4753	0.5668	0.4980	0.4725	0.3778	0.3533	0.4757	0.5395	0.3444
Macro	0.1101	0.7798	0.0690	0.1248	0.7798	0.0001	0.0001	0.7798	0.0592	0.0690
Micro	0.1101	0.7798	0.0690	0.1248	0.7798	0.0001	0.0001	0.7798	0.0592	0.0690
PerExample	0.1101	0.7798	0.0690	0.1248	0.7798	0.0001	0.0001	0.7798	0.0592	0.0690
spoon-spoon	0.3847	0.3680	0.4449	0.3769	0.4469	0.3480	0.3264	0.3478	0.4304	0.2418
fork-fork	0.3944	0.4500	0.3846	0.3849	0.5395	0.2947	0.2987	0.4234	0.3690	0.2495
spork-spork	0.3894	0.4001	0.4174	0.3804	0.4833	0.3245	0.3152	0.3769	0.4027	0.2451
EO_Sys1	0.5270	0.5647	0.5104	0.5099	0.5234	0.2790	0.2655	0.5473	0.5083	0.3567
FU_System_k15	0.5198	0.4886	0.5845	0.5127	0.4473	0.4314	0.3963	0.4890	0.5546	0.3574
FU_System_k25	0.4645	0.3555	0.6951	0.4544	0.3119	0.5611	0.4307	0.3553	0.6707	0.3033
Limited sample	0.5214	0.5629	0.5008	0.5030	0.5111	0.2701	0.2562	0.5450	0.4997	0.3507
Mixing with MTI	0.5726	0.5089	0.6872	0.5665	0.4978	0.5728	0.5038	0.4989	0.6720	0.4070
Asclepius	0.5890	0.5882	0.6060	0.5814	0.5961	0.4434	0.4323	0.5904	0.5876	0.4241
Antinomyra SYS3	0.5346	0.5427	0.5409	0.5257	0.6215	0.2530	0.2652	0.5421	0.5272	0.3709
Antinomyra SYS4	0.5345	0.5401	0.5447	0.5262	0.5838	0.2791	0.2885	0.5399	0.5292	0.3712
L2R-n1	0.5671	0.6490	0.5422	0.5702	0.6019	0.4328	0.4324	0.6470	0.5047	0.4129
L2R-n2	0.5718	0.6361	0.5576	0.5733	0.5891	0.4482	0.4422	0.6322	0.5219	0.4159
L2R-n3	0.5659	0.6642	0.5312	0.5692	0.6144	0.4233	0.4259	0.6588	0.4960	0.4122
L2R-n4	0.0799	0.0904	0.0733	0.0778	0.0620	0.0655	0.0598	0.0895	0.0722	0.0425
L2R-n5	0.0918	0.0994	0.0830	0.0871	0.0621	0.0657	0.0600	0.1017	0.0836	0.0481

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3074	0.5517	0.5617	0.5278	0.3341	0.3140
MTI First Line Index	0.4729	0.7692	0.6634	0.6882	0.5345	0.4529
Default MTI	0.4840	0.7425	0.7077	0.7029	0.5151	0.4863
FU System	0.4094	0.5320	0.7947	0.6153	0.3445	0.5468
FU_System_t25	0.4120	0.5368	0.7974	0.6197	0.3468	0.5493
FU_System_t15	0.4394	0.6637	0.6813	0.6487	0.4430	0.4688
Macro	0.0847	0.7994	0.0546	0.1007	0.2976	0.0505
Micro	0.0847	0.7994	0.0546	0.1007	0.2976	0.0505
PerExample	0.0847	0.7994	0.0546	0.1007	0.2976	0.0505
spoon-spoon	0.3582	0.5821	0.6017	0.5598	0.3607	0.3983
fork-fork	0.3541	0.6695	0.5122	0.5449	0.4201	0.3409
spork-spork	0.3557	0.6154	0.5623	0.5537	0.3821	0.3728
EO_Sys1	0.4364	0.7310	0.6122	0.6336	0.4946	0.4253
FU_System_k15	0.4504	0.6727	0.6983	0.6615	0.4535	0.4810
FU_System_k25	0.4224	0.5474	0.8125	0.6322	0.3555	0.5625
Limited sample	0.4314	0.7306	0.6046	0.6274	0.4955	0.4175
Mixing with MTI	0.4903	0.6772	0.7954	0.7108	0.4602	0.5622
Asclepius	0.4951	0.7384	0.7298	0.7152	0.5118	0.5066
Antinomyra SYS3	0.4474	0.7281	0.6385	0.6592	0.4894	0.4363
Antinomyra SYS4	0.4504	0.7216	0.6454	0.6603	0.4887	0.4423
L2R-n1	0.4787	0.7945	0.6473	0.6887	0.5578	0.4476
L2R-n2	0.4833	0.7841	0.6650	0.6951	0.5504	0.4605
L2R-n3	0.4762	0.8064	0.6350	0.6850	0.5670	0.4384
L2R-n4	0.2235	0.3887	0.3956	0.3755	0.2385	0.2233
L2R-n5	0.2268	0.3899	0.3983	0.3774	0.2407	0.2274

+ Test batch 1, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2382	0.4054	0.2253	0.2197	0.3747	0.2919	0.2572	0.2561	0.2227	0.1288
MTI First Line Index	0.5596	0.6268	0.5267	0.5521	0.5824	0.4809	0.4605	0.6185	0.5109	0.3961
Default MTI	0.5709	0.5924	0.5755	0.5651	0.5441	0.5233	0.4836	0.5848	0.5576	0.4084
FU System	0.4541	0.3461	0.6788	0.4445	0.3438	0.5043	0.3948	0.3461	0.6598	0.2954
FU_System_t25	0.4569	0.3483	0.6837	0.4474	0.3470	0.5145	0.4031	0.3483	0.6639	0.2976
FU_System_t15	0.5232	0.4903	0.5879	0.5159	0.4388	0.4535	0.4117	0.4903	0.5608	0.3605
Macro	0.1018	0.7185	0.0610	0.1109	0.7185	0.0001	0.0001	0.7185	0.0548	0.0610
Micro	0.1018	0.7185	0.0610	0.1109	0.7185	0.0001	0.0001	0.7185	0.0548	0.0610
PerExample	0.1018	0.7185	0.0610	0.1109	0.7185	0.0001	0.0001	0.7185	0.0548	0.0610
spoon-spoon	0.3817	0.3582	0.4340	0.3684	0.4468	0.3556	0.3319	0.3453	0.4267	0.2352
fork-fork	0.3524	0.4961	0.2834	0.3381	0.4400	0.1225	0.1359	0.4836	0.2772	0.2175
spork-spork	0.1018	0.7185	0.0610	0.1109	0.7185	0.0001	0.0001	0.7185	0.0548	0.0610
EO_Sys1	0.5325	0.5814	0.5043	0.5140	0.5341	0.2864	0.2730	0.5619	0.5060	0.3607
FU_System_k15	0.5226	0.4898	0.5873	0.5153	0.4433	0.4499	0.4095	0.4898	0.5602	0.3600
FU_System_k25	0.4657	0.3550	0.6987	0.4564	0.3042	0.5730	0.4341	0.3550	0.6768	0.3049
Limited sample	0.5251	0.5788	0.4921	0.5050	0.5211	0.2779	0.2636	0.5598	0.4945	0.3528
Mixing with MTI	0.5778	0.5194	0.6807	0.5711	0.4971	0.5812	0.5083	0.5079	0.6702	0.4124
Asclepius	0.0040	0.0037	0.0035	0.0035	0.0019	0.0010	0.0007	0.0041	0.0040	0.0018
Antinomyra SYS3	0.5473	0.5985	0.5143	0.5353	0.6170	0.2607	0.2733	0.5983	0.5042	0.3801
Antinomyra SYS4	0.5351	0.5464	0.5357	0.5249	0.5868	0.2860	0.2957	0.5461	0.5246	0.3695
L2R-n1	0.5844	0.6231	0.5793	0.5808	0.5719	0.4918	0.4726	0.6156	0.5562	0.4238
L2R-n2	0.5857	0.5842	0.6151	0.5805	0.5369	0.5286	0.4905	0.5766	0.5951	0.4231
L2R-n3	0.5812	0.6417	0.5619	0.5784	0.5889	0.4748	0.4628	0.6317	0.5381	0.4216
L2R-n4	0.5770	0.6390	0.5591	0.5767	0.5918	0.4610	0.4531	0.6350	0.5287	0.4199
L2R-n5	0.5750	0.6483	0.5520	0.5749	0.6008	0.4527	0.4492	0.6412	0.5213	0.4177
Antinomyra SYS2	0.5378	0.5532	0.5345	0.5277	0.6269	0.2650	0.2786	0.5521	0.5242	0.3720
FDU_MeSHIndexing_1	0.5429	0.5453	0.5715	0.5379	0.5203	0.4441	0.4242	0.5453	0.5406	0.3811
FDU_MeSHIndexing_2	0.5430	0.5454	0.5716	0.5381	0.5162	0.4431	0.4230	0.5454	0.5406	0.3813
FDU_MeSHIndexing_3	0.5426	0.5450	0.5705	0.5374	0.5250	0.4290	0.4103	0.5450	0.5402	0.3808
FDU_MeSHIndexing_4	0.5428	0.5452	0.5706	0.5376	0.5270	0.4313	0.4126	0.5452	0.5405	0.3809
FDU_MeSHIndexing_5	0.5397	0.5420	0.5680	0.5347	0.5175	0.4355	0.4147	0.5420	0.5373	0.3783

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2498	0.6247	0.4168	0.4150	0.3262	0.2429
MTI First Line Index	0.4725	0.7789	0.6576	0.6901	0.5371	0.4494
Default MTI	0.4867	0.7494	0.7092	0.7074	0.5190	0.4876
FU System	0.4128	0.5371	0.8084	0.6264	0.3448	0.5515
FU_System_t25	0.4146	0.5414	0.8095	0.6296	0.3464	0.5539
FU_System_t15	0.4527	0.6765	0.7110	0.6717	0.4535	0.4836
Macro	0.0810	0.7389	0.0496	0.0915	0.2879	0.0482
Micro	0.0810	0.7389	0.0496	0.0915	0.2879	0.0482
PerExample	0.0810	0.7389	0.0496	0.0915	0.2879	0.0482
spoon-spoon	0.3511	0.5742	0.5858	0.5493	0.3537	0.3895
fork-fork	0.3005	0.7087	0.3565	0.4392	0.4639	0.2437
spork-spork	0.0810	0.7389	0.0496	0.0915	0.2879	0.0482
EO_Sys1	0.4366	0.7507	0.6080	0.6393	0.5062	0.4178
FU_System_k15	0.4522	0.6765	0.7101	0.6710	0.4532	0.4831
FU_System_k25	0.4244	0.5482	0.8224	0.6388	0.3547	0.5664
Limited sample	0.4301	0.7457	0.5973	0.6294	0.5043	0.4090
Mixing with MTI	0.4956	0.6914	0.7934	0.7185	0.4696	0.5604
Asclepius	0.1064	0.1082	0.1775	0.1303	0.0914	0.1345
Antinomyra SYS3	0.4532	0.7614	0.6256	0.6654	0.5241	0.4228
Antinomyra SYS4	0.4497	0.7300	0.6425	0.6632	0.4922	0.4380
L2R-n1	0.4914	0.7763	0.6962	0.7116	0.5408	0.4797
L2R-n2	0.4977	0.7473	0.7341	0.7200	0.5181	0.5097
L2R-n3	0.4871	0.7904	0.6771	0.7054	0.5522	0.4655
L2R-n4	0.4856	0.7865	0.6728	0.7026	0.5498	0.4626
L2R-n5	0.4814	0.7964	0.6599	0.6975	0.5562	0.4536
Antinomyra SYS2	0.4488	0.7424	0.6355	0.6644	0.4971	0.4330
FDU_MeSHIndexing_1	0.4652	0.7323	0.6905	0.6879	0.4941	0.4709
FDU_MeSHIndexing_2	0.4651	0.7316	0.6905	0.6875	0.4934	0.4714
FDU_MeSHIndexing_3	0.4638	0.7317	0.6876	0.6864	0.4934	0.4681
FDU_MeSHIndexing_4	0.4645	0.7323	0.6878	0.6868	0.4943	0.4687
FDU_MeSHIndexing_5	0.4608	0.7291	0.6827	0.6821	0.4897	0.4660

+ Test batch 1, week 3

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2442	0.3181	0.2433	0.2290	0.3747	0.3179	0.2743	0.2504	0.2382	0.1354
MTI First Line Index	0.5690	0.6350	0.5372	0.5615	0.5985	0.4980	0.4791	0.6284	0.5199	0.4071
Default MTI	0.5757	0.5949	0.5803	0.5689	0.5552	0.5357	0.4977	0.5908	0.5614	0.4132
FU System	0.4494	0.3438	0.6695	0.4396	0.3471	0.5015	0.3942	0.3438	0.6486	0.2909
FU_System_t25	0.4993	0.4702	0.5584	0.4917	0.4774	0.3820	0.3575	0.4702	0.5322	0.3381
FU_System_t15	0.5001	0.4709	0.5593	0.4925	0.4756	0.3845	0.3594	0.4709	0.5330	0.3389
Macro	0.0995	0.7093	0.0584	0.1065	0.7093	0.0001	0.0001	0.7093	0.0535	0.0584
Micro	0.0995	0.7093	0.0584	0.1065	0.7093	0.0001	0.0001	0.7093	0.0535	0.0584
PerExample	0.0995	0.7093	0.0584	0.1065	0.7093	0.0001	0.0001	0.7093	0.0535	0.0584
spoon-spoon	0.3810	0.3596	0.4342	0.3685	0.4499	0.3655	0.3399	0.3445	0.4262	0.2349
fork-fork	0.3402	0.4839	0.2683	0.3233	0.4327	0.1247	0.1373	0.4753	0.2649	0.2062
spork-spork	0.3384	0.5033	0.2604	0.3213	0.4537	0.1216	0.1351	0.4955	0.2569	0.2049
FU_System_k15	0.4819	0.6264	0.4186	0.4832	0.6285	0.2528	0.2710	0.6285	0.3908	0.3314
FU_System_k25	0.4962	0.5743	0.4641	0.4940	0.5774	0.2918	0.3001	0.5749	0.4364	0.3407
Asclepius	0.5876	0.5886	0.5965	0.5772	0.6025	0.4452	0.4369	0.5925	0.5828	0.4198
Antinomyra SYS3	0.5362	0.6056	0.4980	0.5258	0.6295	0.2503	0.2664	0.5949	0.4879	0.3717
Antinomyra SYS4	0.5037	0.5942	0.4528	0.4973	0.6860	0.1701	0.1949	0.5917	0.4385	0.3442
L2R-n1	0.5853	0.6003	0.5984	0.5806	0.5681	0.5145	0.4891	0.5976	0.5734	0.4236
L2R-n2	0.5858	0.5992	0.6025	0.5806	0.5629	0.5191	0.4912	0.5916	0.5801	0.4236
L2R-n3	0.5857	0.6033	0.5990	0.5808	0.5654	0.5156	0.4895	0.5956	0.5761	0.4239
L2R-n4	0.5857	0.6208	0.5878	0.5816	0.5812	0.5030	0.4852	0.6083	0.5647	0.4250
L2R-n5	0.5861	0.6168	0.5917	0.5818	0.5780	0.5073	0.4875	0.6042	0.5690	0.4252
ERIAS-SIS RUN2	0.2349	0.2382	0.2523	0.2351	0.3528	0.4032	0.3518	0.2386	0.2313	0.1386
FDU_MeSHIndexing_1	0.5387	0.5439	0.5675	0.5345	0.5201	0.4442	0.4250	0.5439	0.5335	0.3776
FDU_MeSHIndexing_2	0.5269	0.5320	0.5544	0.5226	0.5122	0.4364	0.4180	0.5320	0.5219	0.3663
FDU_MeSHIndexing_3	0.5353	0.5405	0.5632	0.5309	0.5299	0.4206	0.4050	0.5405	0.5302	0.3743
FDU_MeSHIndexing_4	0.5370	0.5422	0.5655	0.5329	0.5330	0.4285	0.4119	0.5422	0.5319	0.3763
FDU_MeSHIndexing_5	0.5347	0.5399	0.5634	0.5307	0.5170	0.4360	0.4177	0.5399	0.5296	0.3743

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2730	0.5624	0.4650	0.4499	0.3294	0.2691
MTI First Line Index	0.4782	0.7798	0.6615	0.6912	0.5437	0.4553
Default MTI	0.4890	0.7461	0.7094	0.7049	0.5202	0.4906
FU System	0.4139	0.5329	0.8028	0.6209	0.3469	0.5513
FU_System_t25	0.4377	0.6603	0.6892	0.6512	0.4405	0.4670
FU_System_t15	0.4385	0.6606	0.6898	0.6516	0.4410	0.4680
Macro	0.0794	0.7290	0.0474	0.0878	0.2866	0.0472
Micro	0.0794	0.7290	0.0474	0.0878	0.2866	0.0472
PerExample	0.0794	0.7290	0.0474	0.0878	0.2866	0.0472
spoon-spoon	0.3516	0.5668	0.5940	0.5471	0.3548	0.3925
fork-fork	0.2962	0.6951	0.3477	0.4287	0.4586	0.2401
spork-spork	0.2919	0.7096	0.3329	0.4193	0.4697	0.2318
FU_System_k15	0.4024	0.7938	0.5129	0.5948	0.5382	0.3441
FU_System_k25	0.4212	0.7518	0.5743	0.6249	0.5078	0.3851
Asclepius	0.4948	0.7401	0.7239	0.7122	0.5143	0.5030
Antinomyra SYS3	0.4463	0.7599	0.6098	0.6510	0.5271	0.4127
Antinomyra SYS4	0.4109	0.7646	0.5400	0.6078	0.5208	0.3607
L2R-n1	0.4956	0.7542	0.7145	0.7114	0.5272	0.4968
L2R-n2	0.4956	0.7530	0.7169	0.7111	0.5254	0.4999
L2R-n3	0.4954	0.7563	0.7134	0.7106	0.5284	0.4971
L2R-n4	0.4939	0.7725	0.6979	0.7083	0.5407	0.4864
L2R-n5	0.4948	0.7691	0.7018	0.7089	0.5384	0.4899
ERIAS-SIS RUN2	0.2722	0.4589	0.5124	0.4625	0.2782	0.2872
FDU_MeSHIndexing_1	0.4644	0.7202	0.6891	0.6802	0.4909	0.4719
FDU_MeSHIndexing_2	0.4565	0.7172	0.6795	0.6733	0.4841	0.4630
FDU_MeSHIndexing_3	0.4603	0.7193	0.6826	0.6762	0.4883	0.4662
FDU_MeSHIndexing_4	0.4620	0.7236	0.6817	0.6774	0.4913	0.4671
FDU_MeSHIndexing_5	0.4600	0.7165	0.6829	0.6751	0.4868	0.4673

+ Test batch 1, week 4

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2594	0.2380	0.3225	0.2586	0.3474	0.4277	0.3580	0.2250	0.3061	0.1537
MTI First Line Index	0.5678	0.6252	0.5406	0.5575	0.5693	0.4834	0.4631	0.6207	0.5233	0.4025
Hippocrates	0.5787	0.5444	0.6287	0.5665	0.5338	0.4966	0.4665	0.5505	0.6101	0.4097
Default MTI	0.5753	0.5871	0.5864	0.5660	0.5343	0.5299	0.4912	0.5844	0.5665	0.4094
FU System	0.4145	0.4150	0.4317	0.4052	0.4135	0.2708	0.2511	0.4151	0.4139	0.2658
FU_System_t25	0.5198	0.5204	0.5488	0.5113	0.4950	0.3735	0.3594	0.5205	0.5190	0.3577
FU_System_t15	0.5103	0.5110	0.5383	0.5018	0.4806	0.3580	0.3439	0.5110	0.5096	0.3489
Macro	0.3821	0.3066	0.5913	0.3773	0.1381	0.4312	0.2987	0.2833	0.5870	0.2417
Micro	0.4938	0.5261	0.4558	0.4605	0.4731	0.1412	0.1273	0.5299	0.4624	0.3160
PerExample	0.4826	0.4988	0.4755	0.4665	0.4265	0.1475	0.1281	0.4941	0.4716	0.3201
spoon-spoon	0.3838	0.3593	0.4405	0.3715	0.4280	0.3514	0.3293	0.3475	0.4285	0.2378
fork-fork	0.3529	0.4891	0.2874	0.3387	0.4109	0.1267	0.1371	0.4798	0.2791	0.2173
spork-spork	0.3517	0.5079	0.2786	0.3366	0.4333	0.1216	0.1333	0.5017	0.2708	0.2161
FU_System_k15	0.4304	0.4920	0.3998	0.4224	0.4817	0.2396	0.2377	0.4937	0.3815	0.2809
FU_System_k25	0.4249	0.3971	0.4744	0.4141	0.2917	0.3095	0.2825	0.3971	0.4569	0.2738
Asclepius	0.5941	0.5918	0.6098	0.5831	0.5883	0.4633	0.4488	0.5958	0.5923	0.4273
Antinomyra SYS3	0.5510	0.6061	0.5234	0.5387	0.6166	0.2810	0.2890	0.6000	0.5094	0.3846
Antinomyra SYS4	0.5401	0.5376	0.5524	0.5268	0.5784	0.3057	0.3099	0.5411	0.5391	0.3729
L2R-n1	0.5875	0.5939	0.6139	0.5804	0.5397	0.5136	0.4853	0.5873	0.5878	0.4233
L2R-n2	0.5869	0.5803	0.6242	0.5792	0.5283	0.5228	0.4893	0.5761	0.5982	0.4222
L2R-n3	0.5867	0.5763	0.6278	0.5789	0.5251	0.5263	0.4908	0.5717	0.6025	0.4217
L2R-n4	0.5883	0.5890	0.6182	0.5800	0.5392	0.5159	0.4884	0.5833	0.5934	0.4232
L2R-n5	0.5855	0.5923	0.6119	0.5791	0.5381	0.5127	0.4836	0.5867	0.5843	0.4221
Antinomyra SYS2	0.5330	0.5112	0.5733	0.5226	0.5089	0.3555	0.3319	0.5119	0.5559	0.3679
Acc	0.3857	0.7836	0.2510	0.3548	0.7444	0.0406	0.0492	0.8023	0.2539	0.2339
obOne	0.1992	0.2274	0.1876	0.1885	0.0356	0.0101	0.0093	0.2145	0.1859	0.1124

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3048	0.5039	0.5944	0.5209	0.3025	0.3376
MTI First Line Index	0.4744	0.7669	0.6628	0.6849	0.5350	0.4558
Hippocrates	0.4888	0.6967	0.7559	0.7039	0.4834	0.5263
Default MTI	0.4876	0.7344	0.7140	0.7005	0.5148	0.4952
FU System	0.3841	0.6073	0.6094	0.5816	0.4005	0.3999
FU_System_t25	0.4414	0.6921	0.6625	0.6499	0.4705	0.4514
FU_System_t15	0.4358	0.6858	0.6556	0.6431	0.4645	0.4455
Macro	0.3662	0.4860	0.7401	0.5551	0.3169	0.4981
Micro	0.3957	0.6889	0.5535	0.5795	0.4623	0.3796
PerExample	0.4060	0.6669	0.5902	0.5982	0.4526	0.3989
spoon-spoon	0.3500	0.5596	0.5958	0.5463	0.3490	0.3943
fork-fork	0.3047	0.6946	0.3621	0.4397	0.4606	0.2506
spork-spork	0.2999	0.7108	0.3467	0.4302	0.4736	0.2408
FU_System_k15	0.3804	0.6731	0.5431	0.5722	0.4463	0.3593
FU_System_k25	0.3871	0.5750	0.6568	0.5865	0.3798	0.4310
Asclepius	0.4961	0.7362	0.7269	0.7101	0.5146	0.5093
Antinomyra SYS3	0.4529	0.7496	0.6264	0.6543	0.5238	0.4290
Antinomyra SYS4	0.4510	0.7099	0.6562	0.6585	0.4830	0.4528
L2R-n1	0.4943	0.7405	0.7231	0.7063	0.5221	0.5058
L2R-n2	0.4957	0.7303	0.7337	0.7078	0.5146	0.5146
L2R-n3	0.4959	0.7274	0.7373	0.7082	0.5120	0.5175
L2R-n4	0.4946	0.7401	0.7245	0.7068	0.5203	0.5077
L2R-n5	0.4941	0.7402	0.7196	0.7049	0.5223	0.5050
Antinomyra SYS2	0.4555	0.6770	0.6951	0.6625	0.4685	0.4738
Acc	0.2709	0.8495	0.2646	0.3692	0.5493	0.1932
obOne	0.2307	0.3936	0.3075	0.3138	0.2786	0.2226

+ Test batch 1, week 5

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2552	0.2673	0.3167	0.2505	0.3575	0.4153	0.3450	0.2202	0.3033	0.1481
MTI First Line Index	0.5505	0.6206	0.5147	0.5415	0.5852	0.4632	0.4475	0.6160	0.4976	0.3860
Hippocrates	0.4675	0.7631	0.3394	0.4438	0.7417	0.2321	0.2605	0.7397	0.3418	0.3009
Default MTI	0.5619	0.5890	0.5639	0.5556	0.5531	0.5036	0.4724	0.5840	0.5414	0.3987
FU System	0.5339	0.4822	0.6306	0.5241	0.4731	0.5283	0.4732	0.4835	0.5961	0.3673
FU_System_t25	0.5524	0.5239	0.6176	0.5442	0.5033	0.5230	0.4795	0.5264	0.5811	0.3864
FU_System_t15	0.5452	0.4925	0.6446	0.5355	0.4630	0.5519	0.4903	0.4937	0.6087	0.3780
Sisyphus	0.4688	0.7617	0.3397	0.4457	0.7318	0.2319	0.2602	0.7474	0.3415	0.3024
Macro	0.3962	0.3246	0.5939	0.3915	0.1523	0.4260	0.2807	0.2990	0.5869	0.2523
Micro	0.4877	0.5260	0.4478	0.4570	0.4892	0.1214	0.1097	0.5272	0.4537	0.3117
PerExample	0.4791	0.4966	0.4691	0.4632	0.4515	0.1285	0.1123	0.4931	0.4658	0.3161
spoon-spoon	0.3781	0.3565	0.4333	0.3657	0.4400	0.3469	0.3229	0.3417	0.4232	0.2333
fork-fork	0.3454	0.4832	0.2799	0.3316	0.4249	0.1221	0.1345	0.4715	0.2726	0.2121
spork-spork	0.3444	0.5064	0.2720	0.3307	0.4475	0.1180	0.1314	0.4932	0.2646	0.2117
FU_System_k15	0.5463	0.4935	0.6456	0.5365	0.4752	0.5474	0.4889	0.4946	0.6099	0.3791
FU_System_k25	0.5473	0.4955	0.6459	0.5372	0.4974	0.5303	0.4793	0.4965	0.6097	0.3801
Asclepius	0.5886	0.5904	0.5970	0.5771	0.6035	0.4399	0.4308	0.5965	0.5809	0.4194
Antinomyra SYS3	0.5437	0.6139	0.5036	0.5311	0.6189	0.2457	0.2591	0.6092	0.4908	0.3761
Antinomyra SYS4	0.5308	0.5387	0.5337	0.5197	0.5794	0.2704	0.2804	0.5401	0.5218	0.3647
Antinomyra SYS1	0.5443	0.6313	0.4920	0.5311	0.6322	0.2303	0.2471	0.6312	0.4785	0.3767
L2R-n1	0.5828	0.6022	0.5987	0.5782	0.5618	0.4989	0.4760	0.5948	0.5713	0.4205
L2R-n2	0.5823	0.5992	0.5981	0.5778	0.5610	0.4955	0.4741	0.5969	0.5684	0.4200
L2R-n3	0.5823	0.5992	0.5981	0.5778	0.5610	0.4955	0.4741	0.5969	0.5684	0.4200
L2R-n4	0.5703	0.5159	0.6676	0.5617	0.4846	0.5639	0.5034	0.5088	0.6486	0.4024
L2R-n5	0.5825	0.6181	0.5841	0.5769	0.5758	0.4820	0.4685	0.6093	0.5579	0.4193
ERIAS-SIS	0.1763	0.5256	0.1252	0.1850	0.5017	0.2225	0.2369	0.4553	0.1093	0.1082
Antinomyra SYS2	0.5386	0.5478	0.5420	0.5284	0.5609	0.3026	0.2987	0.5503	0.5274	0.3725
ERIAS-SIS RUN2	0.1764	0.5245	0.1256	0.1852	0.5018	0.2228	0.2372	0.4527	0.1095	0.1083
ERIAS-SIS RUN3	0.1731	0.3501	0.1422	0.1808	0.4892	0.2163	0.2226	0.2843	0.1244	0.1053
FDU_MeSHIndexing_1	0.5315	0.5339	0.5680	0.5260	0.5050	0.4236	0.4067	0.5339	0.5290	0.3693
FDU_MeSHIndexing_2	0.5312	0.5336	0.5668	0.5254	0.5195	0.4076	0.3942	0.5336	0.5288	0.3689
FDU_MeSHIndexing_3	0.5497	0.5523	0.5866	0.5438	0.5263	0.4617	0.4410	0.5523	0.5472	0.3865
FDU_MeSHIndexing_4	0.5496	0.5522	0.5875	0.5442	0.5255	0.4657	0.4427	0.5522	0.5471	0.3867
FDU_MeSHIndexing_5	0.5544	0.5571	0.5934	0.5493	0.5299	0.5008	0.4726	0.5571	0.5517	0.3911
Acc	0.3685	0.7836	0.2375	0.3413	0.7530	0.0320	0.0402	0.8027	0.2391	0.2222
obOne	0.1935	0.2249	0.1802	0.1837	0.0294	0.0074	0.0066	0.2101	0.1794	0.1089

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2927	0.5192	0.5750	0.4997	0.3002	0.3259
MTI First Line Index	0.4673	0.7775	0.6479	0.6833	0.5376	0.4399
Hippocrates	0.3691	0.8620	0.4670	0.5678	0.5577	0.2954
Default MTI	0.4828	0.7497	0.6989	0.7016	0.5206	0.4790
FU System	0.4682	0.6692	0.7617	0.6901	0.4505	0.5233
FU_System_t25	0.4777	0.7030	0.7431	0.6999	0.4780	0.5114
FU_System_t15	0.4751	0.6749	0.7690	0.6960	0.4573	0.5310
Sisyphus	0.3709	0.8699	0.4642	0.5702	0.5624	0.2954
Macro	0.3784	0.5126	0.7441	0.5754	0.3335	0.4972
Micro	0.3973	0.7028	0.5510	0.5842	0.4672	0.3772
PerExample	0.4072	0.6777	0.5906	0.6043	0.4547	0.3965
spoon-spoon	0.3501	0.5658	0.5921	0.5463	0.3537	0.3899
fork-fork	0.3014	0.6952	0.3564	0.4347	0.4624	0.2467
spork-spork	0.2971	0.7148	0.3411	0.4254	0.4762	0.2379
FU_System_k15	0.4751	0.6767	0.7688	0.6971	0.4573	0.5309
FU_System_k25	0.4742	0.6773	0.7646	0.6944	0.4576	0.5303
Asclepius	0.4938	0.7483	0.7239	0.7161	0.5176	0.4998
Antinomyra SYS3	0.4478	0.7689	0.6145	0.6565	0.5301	0.4144
Antinomyra SYS4	0.4472	0.7224	0.6405	0.6577	0.4879	0.4372
Antinomyra SYS1	0.4427	0.7873	0.5899	0.6478	0.5406	0.4009
L2R-n1	0.4949	0.7594	0.7148	0.7131	0.5306	0.4951
L2R-n2	0.4952	0.7571	0.7154	0.7134	0.5297	0.4950
L2R-n3	0.4952	0.7571	0.7154	0.7134	0.5297	0.4950
L2R-n4	0.4935	0.6972	0.7795	0.7151	0.4766	0.5472
L2R-n5	0.4910	0.7749	0.6973	0.7097	0.5398	0.4817
ERIAS-SIS	0.1997	0.7413	0.2409	0.3299	0.4158	0.1427
Antinomyra SYS2	0.4569	0.7248	0.6627	0.6705	0.4962	0.4494
ERIAS-SIS RUN2	0.1996	0.7391	0.2426	0.3312	0.4145	0.1429
ERIAS-SIS RUN3	0.2223	0.6159	0.2939	0.3599	0.3742	0.1755
FDU_MeSHIndexing_1	0.4592	0.7199	0.6882	0.6776	0.4865	0.4689
FDU_MeSHIndexing_2	0.4576	0.7213	0.6843	0.6763	0.4862	0.4660
FDU_MeSHIndexing_3	0.4738	0.7296	0.7069	0.6934	0.4990	0.4848
FDU_MeSHIndexing_4	0.4750	0.7264	0.7116	0.6947	0.4979	0.4878
FDU_MeSHIndexing_5	0.4791	0.7294	0.7205	0.7010	0.5012	0.4930
Acc	0.2613	0.8567	0.2512	0.3562	0.5530	0.1831
obOne	0.2260	0.3965	0.3005	0.3105	0.2764	0.2159

+ Test batch 2, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2682	0.2490	0.3212	0.2643	0.3671	0.4291	0.3575	0.2365	0.3097	0.1577
MTI First Line Index	0.5674	0.6409	0.5355	0.5616	0.5996	0.4878	0.4708	0.6356	0.5124	0.4071
cole_hce1	0.4870	0.4696	0.5354	0.4799	0.4258	0.3846	0.3588	0.4696	0.5057	0.3287
cole_hce2	0.4199	0.4050	0.4692	0.4164	0.2928	0.3388	0.3049	0.4050	0.4361	0.2723
utai_rebayct	0.3110	0.2999	0.3419	0.3064	0.5550	0.1549	0.1588	0.2999	0.3230	0.1866
cole_hce_ne	0.4818	0.4646	0.5291	0.4746	0.4237	0.3781	0.3545	0.4646	0.5003	0.3244
Hippocrates	0.5845	0.5918	0.5905	0.5738	0.5825	0.4463	0.4368	0.5977	0.5718	0.4169
utai_rebayct_2	0.3129	0.3017	0.3439	0.3082	0.4849	0.1630	0.1662	0.3017	0.3249	0.1881
Default MTI	0.5783	0.6093	0.5789	0.5733	0.5709	0.5287	0.4960	0.6040	0.5546	0.4175
FU System	0.5782	0.6014	0.5865	0.5732	0.5633	0.5366	0.5007	0.5953	0.5621	0.4172
FU_System_t25	0.5648	0.5234	0.6428	0.5569	0.5004	0.5701	0.5107	0.5273	0.6080	0.3992
FU_System_t15	0.5558	0.5150	0.6291	0.5465	0.4883	0.5587	0.5007	0.5198	0.5971	0.3897
Sisyphus	0.5870	0.6205	0.5686	0.5752	0.6131	0.4217	0.4176	0.6284	0.5508	0.4190
Macro	0.3896	0.3184	0.5853	0.3838	0.1445	0.4101	0.2744	0.2937	0.5787	0.2466
Micro	0.4882	0.5299	0.4488	0.4565	0.4776	0.1197	0.1090	0.5325	0.4507	0.3120
PerExample	0.4772	0.5021	0.4678	0.4643	0.4053	0.1276	0.1100	0.4923	0.4630	0.3179
spoon-spoon	0.3886	0.3706	0.4464	0.3783	0.4496	0.3549	0.3333	0.3525	0.4330	0.2427
fork-fork	0.3560	0.4980	0.2912	0.3441	0.4373	0.1316	0.1426	0.4864	0.2807	0.2214
spork-spork	0.3553	0.5214	0.2839	0.3435	0.4595	0.1274	0.1396	0.5083	0.2731	0.2213
FU_System_k15	0.5671	0.5369	0.6350	0.5602	0.5260	0.5462	0.5013	0.5391	0.5982	0.4028
FU_System_k25	0.5523	0.5223	0.6178	0.5449	0.5123	0.5358	0.4903	0.5250	0.5825	0.3878
Asclepius	0.5917	0.5911	0.6061	0.5816	0.5960	0.4489	0.4381	0.5964	0.5869	0.4248
L2R-n1	0.5951	0.6129	0.6083	0.5891	0.5710	0.5210	0.4952	0.6085	0.5823	0.4323
L2R-n2	0.5947	0.6160	0.6049	0.5889	0.5737	0.5176	0.4935	0.6119	0.5784	0.4321
L2R-n3	0.5940	0.6260	0.5968	0.5884	0.5806	0.5092	0.4887	0.6199	0.5702	0.4317
L2R-n4	0.5925	0.6129	0.6065	0.5879	0.5630	0.5165	0.4911	0.6063	0.5793	0.4313
L2R-n5	0.5916	0.6087	0.6068	0.5870	0.5630	0.5158	0.4916	0.6064	0.5774	0.4302
ERIAS-SIS	0.2175	0.4461	0.1681	0.2233	0.4505	0.2935	0.2944	0.3967	0.1498	0.1328
ERIAS-SIS RUN2	0.1859	0.5586	0.1313	0.1953	0.5218	0.2328	0.2473	0.4819	0.1152	0.1149
ERIAS-SIS RUN3	0.1889	0.5688	0.1329	0.1982	0.5247	0.2426	0.2564	0.4934	0.1168	0.1166
FDU_MeSHIndexing_1	0.5703	0.5703	0.6105	0.5657	0.5455	0.5085	0.4802	0.5703	0.5704	0.4086
FDU_MeSHIndexing_2	0.5693	0.5692	0.6094	0.5647	0.5432	0.5062	0.4779	0.5692	0.5693	0.4077
FDU_MeSHIndexing_3	0.5671	0.5671	0.6068	0.5623	0.5438	0.4993	0.4748	0.5671	0.5672	0.4053
Acc	0.3759	0.7965	0.2463	0.3502	0.7447	0.0331	0.0414	0.8156	0.2442	0.2303
obOne	0.2004	0.2343	0.1861	0.1901	0.0312	0.0075	0.0069	0.2202	0.1838	0.1129

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3083	0.5167	0.5902	0.5251	0.3130	0.3348
MTI First Line Index	0.4793	0.7890	0.6625	0.6946	0.5506	0.4535
cole_hce1	0.4319	0.6535	0.6760	0.6396	0.4407	0.4572
cole_hce2	0.3882	0.5961	0.6298	0.5870	0.3910	0.4177
utai_rebayct	0.2972	0.5669	0.4992	0.5076	0.2948	0.3263
cole_hce_ne	0.4278	0.6501	0.6717	0.6357	0.4369	0.4520
Hippocrates	0.4905	0.7404	0.7166	0.7078	0.5158	0.4969
utai_rebayct_2	0.3016	0.5651	0.5107	0.5124	0.2986	0.3320
Default MTI	0.4926	0.7594	0.7097	0.7103	0.5327	0.4885
FU System	0.4931	0.7524	0.7179	0.7112	0.5272	0.4943
FU_System_t25	0.4872	0.6921	0.7674	0.7065	0.4766	0.5321
FU_System_t15	0.4797	0.6780	0.7631	0.6950	0.4682	0.5261
Sisyphus	0.4872	0.7632	0.6870	0.7016	0.5315	0.4783
Macro	0.3704	0.4981	0.7341	0.5599	0.3265	0.4923
Micro	0.3950	0.7019	0.5480	0.5784	0.4691	0.3762
PerExample	0.4056	0.6734	0.5838	0.5972	0.4565	0.3945
spoon-spoon	0.3583	0.5782	0.5953	0.5529	0.3627	0.4008
fork-fork	0.3066	0.7039	0.3593	0.4406	0.4666	0.2506
spork-spork	0.3019	0.7241	0.3448	0.4324	0.4786	0.2416
FU_System_k15	0.4878	0.7038	0.7535	0.7051	0.4850	0.5256
FU_System_k25	0.4783	0.6958	0.7453	0.6974	0.4760	0.5150
Asclepius	0.4961	0.7410	0.7281	0.7142	0.5147	0.5078
L2R-n1	0.4988	0.7620	0.7194	0.7162	0.5326	0.5020
L2R-n2	0.4981	0.7645	0.7156	0.7151	0.5345	0.4989
L2R-n3	0.4963	0.7725	0.7068	0.7129	0.5403	0.4919
L2R-n4	0.5006	0.7621	0.7172	0.7146	0.5364	0.5022
L2R-n5	0.5002	0.7592	0.7194	0.7156	0.5335	0.5022
ERIAS-SIS	0.2386	0.6894	0.3273	0.4073	0.4001	0.1861
ERIAS-SIS RUN2	0.2032	0.7724	0.2483	0.3415	0.4245	0.1442
ERIAS-SIS RUN3	0.2027	0.7779	0.2483	0.3432	0.4243	0.1439
FDU_MeSHIndexing_1	0.4885	0.7312	0.7271	0.7048	0.5101	0.5031
FDU_MeSHIndexing_2	0.4875	0.7318	0.7256	0.7043	0.5093	0.5020
FDU_MeSHIndexing_3	0.4865	0.7324	0.7231	0.7032	0.5093	0.5003
Acc	0.2683	0.8631	0.2574	0.3610	0.5651	0.1902
obOne	0.2293	0.3986	0.2964	0.3074	0.2858	0.2171

+ Test batch 2, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2599	0.2379	0.3264	0.2586	0.3394	0.4262	0.3506	0.2250	0.3075	0.1537
MTI First Line Index	0.5743	0.6250	0.5587	0.5673	0.5707	0.4894	0.4638	0.6176	0.5367	0.4131
cole_hce1	0.4896	0.4590	0.5523	0.4803	0.3988	0.3752	0.3449	0.4590	0.5245	0.3301
cole_hce2	0.4821	0.4520	0.5431	0.4726	0.4007	0.3585	0.3292	0.4520	0.5164	0.3232
utai_rebayct	0.3125	0.2930	0.3527	0.3062	0.5319	0.1548	0.1560	0.2930	0.3348	0.1869
cole_hce_ne	0.4823	0.4522	0.5435	0.4728	0.4000	0.3562	0.3287	0.4522	0.5167	0.3234
Hippocrates	0.5817	0.5822	0.5981	0.5716	0.5543	0.4528	0.4359	0.5845	0.5789	0.4152
utai_rebayct_2	0.3640	0.3412	0.4179	0.3593	0.4021	0.2389	0.2235	0.3413	0.3899	0.2260
Default MTI	0.5803	0.5903	0.6013	0.5734	0.5365	0.5309	0.4872	0.5829	0.5776	0.4182
FU System	0.5614	0.4988	0.6743	0.5518	0.4778	0.5690	0.5001	0.5010	0.6383	0.3952
FU_System_t25	0.5642	0.5110	0.6612	0.5552	0.4838	0.5628	0.4984	0.5140	0.6253	0.3986
FU_System_t15	0.5648	0.5116	0.6620	0.5559	0.4860	0.5623	0.4986	0.5145	0.6259	0.3992
Sisyphus	0.5845	0.6094	0.5758	0.5732	0.5843	0.4274	0.4157	0.6133	0.5584	0.4175
Macro	0.3839	0.3081	0.6079	0.3818	0.1366	0.4449	0.2986	0.2818	0.6020	0.2451
Micro	0.4978	0.5188	0.4715	0.4672	0.4479	0.1400	0.1218	0.5212	0.4764	0.3208
PerExample	0.4855	0.4948	0.4916	0.4726	0.3982	0.1476	0.1235	0.4858	0.4852	0.3248
FU_System_k15	0.5640	0.5081	0.6639	0.5548	0.4855	0.5645	0.4990	0.5114	0.6288	0.3979
FU_System_k25	0.5192	0.4108	0.7367	0.5074	0.4029	0.6165	0.5008	0.4110	0.7049	0.3520
Asclepius	0.5893	0.5777	0.6160	0.5787	0.5662	0.4582	0.4392	0.5816	0.5971	0.4221
Antinomyra SYS0	0.5950	0.5967	0.6185	0.5872	0.5580	0.4830	0.4581	0.5937	0.5962	0.4309
Antinomyra SYS3	0.5936	0.5950	0.6179	0.5862	0.5413	0.4996	0.4697	0.5923	0.5949	0.4301
Antinomyra SYS4	0.5951	0.5969	0.6205	0.5883	0.5436	0.5025	0.4718	0.5938	0.5964	0.4321
Antinomyra SYS1	0.5941	0.5960	0.6178	0.5865	0.5542	0.4857	0.4591	0.5928	0.5953	0.4306
L2R-n1	0.5955	0.5984	0.6259	0.5896	0.5474	0.5232	0.4907	0.5924	0.5986	0.4338
L2R-n2	0.5958	0.6014	0.6235	0.5899	0.5501	0.5208	0.4894	0.5956	0.5959	0.4342
L2R-n3	0.5959	0.6102	0.6169	0.5906	0.5563	0.5133	0.4849	0.6026	0.5895	0.4350
L2R-n4	0.5930	0.5824	0.6399	0.5872	0.5284	0.5371	0.4969	0.5747	0.6126	0.4309
L2R-n5	0.5950	0.6035	0.6198	0.5890	0.5499	0.5147	0.4841	0.5977	0.5924	0.4334
Antinomyra SYS2	0.5946	0.5961	0.6194	0.5874	0.5464	0.5023	0.4713	0.5933	0.5958	0.4312
FDU_MeSHIndexing_1	0.5759	0.5594	0.6346	0.5697	0.5290	0.5147	0.4792	0.5594	0.5935	0.4135
FDU_MeSHIndexing_2	0.5754	0.5589	0.6341	0.5691	0.5286	0.5114	0.4766	0.5589	0.5930	0.4129
FDU_MeSHIndexing_3	0.5755	0.5590	0.6345	0.5694	0.5264	0.5132	0.4772	0.5590	0.5931	0.4131
FDU_MeSHIndexing_4	0.5754	0.5588	0.6339	0.5690	0.5293	0.5097	0.4758	0.5588	0.5929	0.4128
FDU_MeSHIndexing_5	0.5747	0.5582	0.6337	0.5686	0.5275	0.5133	0.4775	0.5582	0.5923	0.4123
Acc	0.3950	0.7713	0.2590	0.3631	0.7394	0.0415	0.0495	0.7966	0.2626	0.2410
SNUMedinfo1	0.4612	0.5760	0.4027	0.4468	0.3992	0.3403	0.3162	0.5524	0.3958	0.3028
SNUMedinfo2	0.4691	0.4993	0.4708	0.4567	0.3264	0.3796	0.3327	0.4740	0.4643	0.3100
SNUMedinfo3	0.4741	0.5028	0.4668	0.4583	0.3536	0.3772	0.3364	0.4899	0.4593	0.3113
SNUMedinfo4	0.4681	0.4304	0.5307	0.4541	0.3082	0.4135	0.3490	0.4256	0.5200	0.3060
SNUMedinfo5	0.4536	0.3787	0.5778	0.4405	0.2856	0.4409	0.3589	0.3807	0.5610	0.2931

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3063	0.5050	0.5970	0.5202	0.3067	0.3404
MTI First Line Index	0.4814	0.7665	0.6754	0.6905	0.5342	0.4718
cole_hce1	0.4337	0.6408	0.6880	0.6364	0.4341	0.4699
cole_hce2	0.4283	0.6378	0.6790	0.6309	0.4294	0.4631
utai_rebayct	0.2947	0.5544	0.5105	0.5077	0.2856	0.3334
cole_hce_ne	0.4278	0.6374	0.6784	0.6304	0.4287	0.4630
Hippocrates	0.4880	0.7279	0.7211	0.7014	0.5068	0.5022
utai_rebayct_2	0.3502	0.5779	0.6210	0.5733	0.3437	0.3905
Default MTI	0.4913	0.7343	0.7238	0.7025	0.5144	0.5064
FU System	0.4824	0.6641	0.7843	0.6953	0.4545	0.5541
FU_System_t25	0.4842	0.6753	0.7717	0.6964	0.4633	0.5458
FU_System_t15	0.4847	0.6748	0.7728	0.6967	0.4632	0.5472
Sisyphus	0.4842	0.7504	0.6910	0.6956	0.5198	0.4835
Macro	0.3679	0.4826	0.7550	0.5569	0.3145	0.5089
Micro	0.4028	0.6884	0.5723	0.5905	0.4596	0.3949
PerExample	0.4105	0.6632	0.6044	0.6041	0.4482	0.4117
FU_System_k15	0.4834	0.6713	0.7769	0.6968	0.4598	0.5484
FU_System_k25	0.4553	0.5854	0.8399	0.6670	0.3902	0.5932
Asclepius	0.4940	0.7260	0.7353	0.7084	0.5032	0.5159
Antinomyra SYS0	0.5005	0.7481	0.7229	0.7111	0.5243	0.5110
Antinomyra SYS3	0.4980	0.7456	0.7233	0.7098	0.5197	0.5113
Antinomyra SYS4	0.4995	0.7438	0.7253	0.7100	0.5198	0.5139
Antinomyra SYS1	0.5004	0.7430	0.7244	0.7093	0.5228	0.5126
L2R-n1	0.5007	0.7430	0.7324	0.7111	0.5220	0.5180
L2R-n2	0.5004	0.7451	0.7298	0.7107	0.5237	0.5160
L2R-n3	0.4997	0.7520	0.7233	0.7101	0.5286	0.5108
L2R-n4	0.5015	0.7308	0.7439	0.7111	0.5142	0.5277
L2R-n5	0.4990	0.7472	0.7263	0.7096	0.5247	0.5130
Antinomyra SYS2	0.4982	0.7439	0.7249	0.7101	0.5177	0.5137
FDU_MeSHIndexing_1	0.4901	0.7183	0.7371	0.7008	0.4994	0.5202
FDU_MeSHIndexing_2	0.4903	0.7176	0.7364	0.7000	0.4999	0.5205
FDU_MeSHIndexing_3	0.4894	0.7165	0.7367	0.6996	0.4991	0.5191
FDU_MeSHIndexing_4	0.4893	0.7177	0.7348	0.6994	0.4987	0.5194
FDU_MeSHIndexing_5	0.4893	0.7166	0.7353	0.6989	0.4984	0.5193
Acc	0.2818	0.8450	0.2775	0.3820	0.5512	0.2041
SNUMedinfo1	0.3637	0.7072	0.4807	0.5292	0.4644	0.3373
SNUMedinfo2	0.3806	0.6435	0.5815	0.5680	0.4160	0.4015
SNUMedinfo3	0.3818	0.6468	0.5761	0.5691	0.4185	0.3982
SNUMedinfo4	0.3867	0.5821	0.6635	0.5864	0.3707	0.4549
SNUMedinfo5	0.3810	0.5271	0.7216	0.5831	0.3316	0.4947

+ Test batch 2, week 3

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2656	0.2591	0.3096	0.2623	0.3667	0.4212	0.3530	0.2384	0.2998	0.1570
MTI First Line Index	0.5670	0.6448	0.5333	0.5598	0.5924	0.4768	0.4626	0.6339	0.5129	0.4045
cole_hce1	0.4816	0.4626	0.5335	0.4733	0.4033	0.3565	0.3338	0.4626	0.5022	0.3228
cole_hce2	0.4736	0.4549	0.5243	0.4654	0.3987	0.3342	0.3133	0.4549	0.4938	0.3157
utai_rebayct	0.3112	0.2990	0.3468	0.3063	0.5437	0.1527	0.1564	0.2990	0.3245	0.1865
cole_hce_ne	0.4757	0.4569	0.5262	0.4673	0.4014	0.3359	0.3162	0.4569	0.4960	0.3174
Hippocrates	0.5898	0.6046	0.5939	0.5811	0.5917	0.4595	0.4509	0.6063	0.5743	0.4235
utai_rebayct_2	0.3518	0.3380	0.3953	0.3474	0.4089	0.2415	0.2311	0.3380	0.3668	0.2162
Default MTI	0.5771	0.6109	0.5776	0.5714	0.5581	0.5152	0.4854	0.6018	0.5543	0.4160
FU System	0.5701	0.5682	0.6143	0.5638	0.5187	0.5096	0.4797	0.5680	0.5722	0.4060
FU_System_t25	0.5529	0.4866	0.6766	0.5419	0.4649	0.5643	0.4981	0.4864	0.6405	0.3846
FU_System_t15	0.5667	0.5447	0.6311	0.5588	0.5195	0.5104	0.4764	0.5445	0.5906	0.4010
Sisyphus	0.5901	0.6392	0.5630	0.5796	0.6294	0.4245	0.4236	0.6428	0.5454	0.4228
spoon-spoon	0.3885	0.3725	0.4340	0.3745	0.4633	0.3492	0.3307	0.3585	0.4240	0.2401
fork-fork	0.3544	0.4902	0.2890	0.3398	0.4347	0.1299	0.1423	0.4827	0.2799	0.2194
spork-spork	0.3532	0.5097	0.2815	0.3382	0.4553	0.1265	0.1399	0.5026	0.2723	0.2186
FU_System_k15	0.5634	0.5243	0.6481	0.5542	0.4728	0.5473	0.4962	0.5241	0.6091	0.3967
FU_System_k25	0.5295	0.4360	0.7069	0.5172	0.3833	0.6157	0.5092	0.4356	0.6749	0.3612
Asclepius	0.5886	0.5927	0.6004	0.5785	0.5944	0.4473	0.4362	0.5959	0.5814	0.4205
Antinomyra SYS0	0.5956	0.6156	0.6024	0.5890	0.5803	0.4738	0.4566	0.6118	0.5803	0.4317
Antinomyra SYS3	0.5921	0.6127	0.5993	0.5862	0.5731	0.4749	0.4575	0.6082	0.5769	0.4291
Antinomyra SYS4	0.5884	0.6090	0.5957	0.5826	0.5658	0.4715	0.4535	0.6044	0.5733	0.4256
Antinomyra SYS1	0.5857	0.6054	0.5917	0.5791	0.5625	0.4538	0.4406	0.6016	0.5706	0.4215
L2R-n1	0.5918	0.5999	0.6174	0.5869	0.5479	0.5220	0.4939	0.5911	0.5925	0.4301
L2R-n2	0.5917	0.6027	0.6150	0.5869	0.5479	0.5219	0.4938	0.5937	0.5896	0.4303
L2R-n3	0.5928	0.6161	0.6034	0.5879	0.5634	0.5100	0.4873	0.6082	0.5781	0.4314
L2R-n4	0.5923	0.6172	0.6017	0.5876	0.5634	0.5100	0.4873	0.6094	0.5761	0.4310
L2R-n5	0.5926	0.6093	0.6099	0.5877	0.5565	0.5121	0.4869	0.6018	0.5838	0.4309
ERIAS-SIS	0.4815	0.4674	0.5085	0.4712	0.4528	0.3284	0.3172	0.4717	0.4917	0.3214
Antinomyra SYS2	0.5930	0.6138	0.6005	0.5875	0.5654	0.4880	0.4688	0.6091	0.5778	0.4303
ERIAS-SIS RUN2	0.4842	0.4699	0.5122	0.4742	0.4697	0.3179	0.3075	0.4746	0.4942	0.3236
ERIAS-SIS RUN3	0.4908	0.4813	0.5169	0.4823	0.4620	0.3542	0.3403	0.4837	0.4982	0.3312
FDU_MeSHIndexing_1	0.5708	0.5686	0.6158	0.5647	0.5444	0.4992	0.4742	0.5686	0.5731	0.4071
FDU_MeSHIndexing_2	0.5688	0.5666	0.6142	0.5630	0.5382	0.4973	0.4713	0.5665	0.5710	0.4053
FDU_MeSHIndexing_3	0.5694	0.5672	0.6142	0.5632	0.5443	0.4983	0.4730	0.5672	0.5717	0.4056
FDU_MeSHIndexing_4	0.5686	0.5663	0.6131	0.5624	0.5407	0.4929	0.4689	0.5663	0.5708	0.4047
FDU_MeSHIndexing_5	0.5698	0.5676	0.6150	0.5638	0.5419	0.4960	0.4707	0.5675	0.5720	0.4059
SNUMedinfo1	0.4689	0.4654	0.4707	0.4499	0.3796	0.3927	0.3542	0.4759	0.4621	0.3038
SNUMedinfo2	0.4525	0.5188	0.4201	0.4443	0.4441	0.3563	0.3393	0.5225	0.3990	0.2985
SNUMedinfo3	0.4677	0.5090	0.4375	0.4458	0.3833	0.3781	0.3438	0.5046	0.4359	0.3015
SNUMedinfo4	0.4667	0.4385	0.4972	0.4462	0.3402	0.4100	0.3559	0.4422	0.4940	0.3001
SNUMedinfo5	0.4555	0.3885	0.5457	0.4376	0.3081	0.4355	0.3612	0.3959	0.5363	0.2913
ERIAS-SIS RUN4	0.4948	0.4841	0.5212	0.4857	0.4776	0.3502	0.3386	0.4872	0.5026	0.3337
ERIAS-SIS RUN5	0.4966	0.4855	0.5235	0.4875	0.4884	0.3478	0.3364	0.4886	0.5048	0.3353

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3078	0.5191	0.5721	0.5153	0.3182	0.3303
MTI First Line Index	0.4740	0.7814	0.6535	0.6840	0.5444	0.4511
cole_hce1	0.4237	0.6413	0.6661	0.6260	0.4303	0.4539
cole_hce2	0.4187	0.6383	0.6559	0.6187	0.4265	0.4477
utai_rebayct	0.2912	0.5510	0.5030	0.4987	0.2861	0.3267
cole_hce_ne	0.4181	0.6382	0.6568	0.6190	0.4264	0.4470
Hippocrates	0.4939	0.7439	0.7166	0.7082	0.5194	0.4999
utai_rebayct_2	0.3380	0.5617	0.5989	0.5512	0.3353	0.3748
Default MTI	0.4876	0.7525	0.7003	0.6995	0.5263	0.4881
FU System	0.4823	0.7196	0.7244	0.6946	0.4993	0.5056
FU_System_t25	0.4761	0.6560	0.7837	0.6888	0.4458	0.5548
FU_System_t15	0.4803	0.7039	0.7370	0.6933	0.4839	0.5176
Sisyphus	0.4874	0.7727	0.6762	0.6981	0.5382	0.4731
spoon-spoon	0.3542	0.5791	0.5835	0.5477	0.3627	0.3895
fork-fork	0.3059	0.6962	0.3655	0.4432	0.4568	0.2528
spork-spork	0.3018	0.7120	0.3519	0.4353	0.4688	0.2444
FU_System_k15	0.4809	0.6849	0.7593	0.6939	0.4713	0.5330
FU_System_k25	0.4621	0.6091	0.8179	0.6738	0.4087	0.5781
Asclepius	0.4896	0.7328	0.7217	0.7052	0.5083	0.5024
Antinomyra SYS0	0.4980	0.7580	0.7098	0.7093	0.5324	0.5001
Antinomyra SYS3	0.4970	0.7579	0.7083	0.7086	0.5314	0.4985
Antinomyra SYS4	0.4941	0.7541	0.7058	0.7054	0.5288	0.4957
Antinomyra SYS1	0.4907	0.7520	0.6995	0.7013	0.5237	0.4920
L2R-n1	0.5004	0.7484	0.7245	0.7118	0.5267	0.5109
L2R-n2	0.4997	0.7492	0.7232	0.7115	0.5270	0.5093
L2R-n3	0.4973	0.7581	0.7128	0.7103	0.5311	0.5010
L2R-n4	0.4970	0.7584	0.7124	0.7103	0.5316	0.5001
L2R-n5	0.4969	0.7513	0.7166	0.7089	0.5266	0.5048
ERIAS-SIS	0.4179	0.6453	0.6423	0.6215	0.4292	0.4349
Antinomyra SYS2	0.4979	0.7561	0.7085	0.7078	0.5313	0.5002
ERIAS-SIS RUN2	0.4189	0.6506	0.6434	0.6245	0.4316	0.4346
ERIAS-SIS RUN3	0.4275	0.6583	0.6506	0.6318	0.4420	0.4407
FDU_MeSHIndexing_1	0.4852	0.7287	0.7190	0.6968	0.5047	0.5062
FDU_MeSHIndexing_2	0.4835	0.7268	0.7187	0.6960	0.5025	0.5047
FDU_MeSHIndexing_3	0.4838	0.7276	0.7172	0.6952	0.5032	0.5046
FDU_MeSHIndexing_4	0.4831	0.7267	0.7155	0.6937	0.5028	0.5036
FDU_MeSHIndexing_5	0.4840	0.7271	0.7176	0.6953	0.5037	0.5050
SNUMedinfo1	0.3829	0.6091	0.5890	0.5696	0.3968	0.4058
SNUMedinfo2	0.3716	0.6549	0.5232	0.5525	0.4313	0.3578
SNUMedinfo3	0.3728	0.6424	0.5354	0.5456	0.4239	0.3731
SNUMedinfo4	0.3817	0.5844	0.6238	0.5711	0.3777	0.4295
SNUMedinfo5	0.3792	0.5355	0.6940	0.5781	0.3384	0.4735
ERIAS-SIS RUN4	0.4284	0.6650	0.6493	0.6344	0.4438	0.4409
ERIAS-SIS RUN5	0.4291	0.6672	0.6496	0.6359	0.4451	0.4409

+ Test batch 2, week 4

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2655	0.2686	0.3133	0.2625	0.3653	0.4175	0.3489	0.2369	0.3020	0.1566
MTI First Line Index	0.5689	0.6298	0.5411	0.5615	0.5860	0.4907	0.4704	0.6261	0.5213	0.4056
cole_hce1	0.4919	0.5157	0.4993	0.4865	0.4735	0.3335	0.3283	0.5156	0.4703	0.3349
cole_hce2	0.4842	0.5075	0.4893	0.4780	0.4707	0.3160	0.3121	0.5075	0.4629	0.3277
utai_rebayct	0.3110	0.3260	0.3127	0.3059	0.6046	0.1221	0.1299	0.3260	0.2974	0.1870
cole_hce_ne	0.4847	0.5081	0.4896	0.4784	0.4732	0.3153	0.3135	0.5081	0.4634	0.3281
Hippocrates	0.5989	0.6091	0.6038	0.5903	0.6130	0.4836	0.4731	0.6126	0.5859	0.4329
utai_rebayct_2	0.3621	0.3796	0.3712	0.3595	0.4561	0.2100	0.2069	0.3796	0.3462	0.2261
Default MTI	0.5785	0.5971	0.5857	0.5716	0.5567	0.5371	0.4994	0.5937	0.5641	0.4146
FU System	0.5769	0.5804	0.6128	0.5720	0.5395	0.5192	0.4897	0.5804	0.5735	0.4142
FU_System_t25	0.5623	0.4988	0.6790	0.5527	0.4569	0.5887	0.5152	0.4988	0.6445	0.3949
FU_System_t15	0.5675	0.5171	0.6645	0.5588	0.4766	0.5740	0.5117	0.5171	0.6289	0.4009
Sisyphus	0.5992	0.6442	0.5712	0.5890	0.6497	0.4456	0.4445	0.6503	0.5554	0.4325
Macro	0.3918	0.3202	0.5864	0.3869	0.1468	0.4117	0.2693	0.2958	0.5802	0.2484
Micro	0.4906	0.5283	0.4512	0.4612	0.4682	0.1194	0.1069	0.5308	0.4561	0.3149
PerExample	0.4783	0.5026	0.4729	0.4683	0.3758	0.1293	0.1094	0.4880	0.4689	0.3208
spoon-spoon	0.3796	0.3568	0.4422	0.3699	0.4440	0.3622	0.3349	0.3400	0.4298	0.2361
fork-fork	0.3486	0.4858	0.2805	0.3339	0.4418	0.1282	0.1425	0.4777	0.2744	0.2137
spork-spork	0.3473	0.5065	0.2728	0.3325	0.4625	0.1238	0.1390	0.4983	0.2665	0.2130
FU_System_k15	0.5752	0.5579	0.6321	0.5689	0.5158	0.5377	0.4975	0.5579	0.5937	0.4110
FU_System_k25	0.5719	0.5368	0.6492	0.5643	0.4962	0.5580	0.5063	0.5368	0.6120	0.4064
Asclepius	0.5955	0.5971	0.6067	0.5856	0.6131	0.4594	0.4498	0.6010	0.5901	0.4278
Antinomyra SYS0	0.6013	0.6117	0.6109	0.5941	0.5905	0.4838	0.4643	0.6111	0.5919	0.4373
Antinomyra SYS3	0.6020	0.6123	0.6118	0.5949	0.5879	0.4857	0.4666	0.6118	0.5926	0.4381
Antinomyra SYS4	0.5951	0.6058	0.6052	0.5883	0.5746	0.4886	0.4678	0.6048	0.5858	0.4310
Antinomyra SYS1	0.5885	0.5981	0.5966	0.5803	0.5647	0.4530	0.4381	0.5981	0.5793	0.4231
L2R-n1	0.5946	0.6155	0.6082	0.5892	0.5656	0.5172	0.4925	0.6072	0.5825	0.4319
L2R-n2	0.5975	0.6112	0.6125	0.5910	0.5740	0.5216	0.4966	0.6076	0.5877	0.4338
L2R-n3	0.5950	0.6260	0.5973	0.5894	0.5866	0.5023	0.4857	0.6198	0.5720	0.4324
L2R-n4	0.5979	0.6167	0.6089	0.5915	0.5740	0.5215	0.4966	0.6126	0.5839	0.4344
L2R-n5	0.5951	0.6322	0.5931	0.5896	0.5866	0.5022	0.4857	0.6254	0.5676	0.4327
ERIAS-SIS	0.2815	0.2709	0.2915	0.2726	0.3749	0.1776	0.1568	0.2769	0.2862	0.1672
Antinomyra SYS2	0.5995	0.6099	0.6087	0.5919	0.5847	0.4829	0.4661	0.6092	0.5901	0.4349
ERIAS-SIS RUN2	0.3472	0.3325	0.3570	0.3345	0.4439	0.2500	0.2343	0.3415	0.3530	0.2140
ERIAS-SIS RUN3	0.5136	0.5002	0.5386	0.5039	0.4790	0.4040	0.3851	0.5052	0.5222	0.3506
FDU_MeSHIndexing_1	0.5781	0.5816	0.6136	0.5731	0.5565	0.5131	0.4870	0.5816	0.5747	0.4156
FDU_MeSHIndexing_2	0.5782	0.5817	0.6135	0.5731	0.5562	0.5137	0.4875	0.5817	0.5748	0.4156
FDU_MeSHIndexing_3	0.5771	0.5806	0.6124	0.5720	0.5575	0.5087	0.4837	0.5806	0.5737	0.4146
Acc	0.3708	0.7926	0.2379	0.3426	0.7320	0.0311	0.0394	0.8192	0.2396	0.2243
SNUMedinfo1	0.4628	0.5705	0.4044	0.4554	0.4624	0.3665	0.3470	0.5753	0.3870	0.3079
SNUMedinfo2	0.4597	0.5329	0.4261	0.4548	0.4403	0.3768	0.3524	0.5342	0.4034	0.3068
SNUMedinfo3	0.4747	0.5142	0.4603	0.4614	0.3691	0.4025	0.3579	0.5008	0.4513	0.3130
SNUMedinfo4	0.4709	0.4394	0.5212	0.4577	0.3233	0.4354	0.3674	0.4368	0.5108	0.3086
SNUMedinfo5	0.4585	0.3900	0.5641	0.4454	0.2988	0.4580	0.3719	0.3937	0.5489	0.2971
ERIAS-SIS RUN4	0.5130	0.4997	0.5380	0.5033	0.4792	0.4029	0.3847	0.5047	0.5216	0.3501

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3014	0.5295	0.5719	0.5154	0.3128	0.3242
MTI First Line Index	0.4760	0.7741	0.6635	0.6905	0.5391	0.4544
cole_hce1	0.4217	0.6941	0.6187	0.6274	0.4684	0.4133
cole_hce2	0.4158	0.6908	0.6101	0.6213	0.4634	0.4064
utai_rebayct	0.2827	0.5937	0.4349	0.4790	0.3042	0.2868
cole_hce_ne	0.4160	0.6886	0.6105	0.6210	0.4627	0.4070
Hippocrates	0.4995	0.7562	0.7229	0.7195	0.5274	0.5008
utai_rebayct_2	0.3375	0.6177	0.5509	0.5576	0.3626	0.3418
Default MTI	0.4890	0.7469	0.7127	0.7064	0.5211	0.4905
FU System	0.4873	0.7339	0.7260	0.7050	0.5110	0.5008
FU_System_t25	0.4818	0.6674	0.7905	0.7005	0.4568	0.5486
FU_System_t15	0.4846	0.6833	0.7767	0.7034	0.4694	0.5387
Sisyphus	0.4923	0.7840	0.6813	0.7085	0.5440	0.4742
Macro	0.3714	0.5059	0.7361	0.5683	0.3269	0.4890
Micro	0.3959	0.7018	0.5530	0.5860	0.4665	0.3748
PerExample	0.4056	0.6770	0.5904	0.6035	0.4549	0.3937
spoon-spoon	0.3502	0.5683	0.5997	0.5523	0.3491	0.3938
fork-fork	0.2997	0.6959	0.3579	0.4377	0.4568	0.2441
spork-spork	0.2948	0.7119	0.3435	0.4288	0.4673	0.2353
FU_System_k15	0.4870	0.7157	0.7452	0.7058	0.4959	0.5146
FU_System_k25	0.4866	0.6994	0.7622	0.7054	0.4829	0.5276
Asclepius	0.4966	0.7469	0.7233	0.7151	0.5199	0.5027
Antinomyra SYS0	0.5028	0.7594	0.7192	0.7180	0.5339	0.5031
Antinomyra SYS3	0.5037	0.7603	0.7203	0.7192	0.5352	0.5034
Antinomyra SYS4	0.4972	0.7574	0.7127	0.7129	0.5299	0.4960
Antinomyra SYS1	0.4895	0.7545	0.7060	0.7089	0.5222	0.4873
L2R-n1	0.4955	0.7597	0.7151	0.7116	0.5332	0.4963
L2R-n2	0.4996	0.7481	0.7320	0.7163	0.5248	0.5101
L2R-n3	0.4953	0.7605	0.7134	0.7110	0.5329	0.4958
L2R-n4	0.4993	0.7506	0.7293	0.7159	0.5272	0.5078
L2R-n5	0.4946	0.7633	0.7098	0.7101	0.5355	0.4930
ERIAS-SIS	0.3117	0.4724	0.5391	0.4866	0.3063	0.3364
Antinomyra SYS2	0.5006	0.7597	0.7141	0.7157	0.5330	0.4992
ERIAS-SIS RUN2	0.3414	0.5179	0.5764	0.5277	0.3368	0.3671
ERIAS-SIS RUN3	0.4390	0.6778	0.6727	0.6558	0.4540	0.4503
FDU_MeSHIndexing_1	0.4878	0.7381	0.7214	0.7045	0.5139	0.4990
FDU_MeSHIndexing_2	0.4880	0.7374	0.7215	0.7043	0.5137	0.4994
FDU_MeSHIndexing_3	0.4873	0.7388	0.7189	0.7035	0.5140	0.4981
Acc	0.2612	0.8601	0.2503	0.3553	0.5544	0.1832
SNUMedinfo1	0.3689	0.7030	0.4912	0.5499	0.4602	0.3338
SNUMedinfo2	0.3738	0.6715	0.5284	0.5647	0.4365	0.3557
SNUMedinfo3	0.3798	0.6554	0.5677	0.5713	0.4218	0.3855
SNUMedinfo4	0.3868	0.5930	0.6532	0.5917	0.3742	0.4415
SNUMedinfo5	0.3832	0.5432	0.7116	0.5924	0.3391	0.4793
ERIAS-SIS RUN4	0.4386	0.6780	0.6721	0.6556	0.4538	0.4499

+ Test batch 2, week 5

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2679	0.2515	0.3107	0.2618	0.3732	0.4096	0.3429	0.2411	0.3013	0.1561
MTI First Line Index	0.5693	0.6353	0.5311	0.5593	0.5938	0.4735	0.4594	0.6322	0.5177	0.4045
cole_hce1	0.4863	0.5434	0.4661	0.4818	0.4989	0.3060	0.3092	0.5434	0.4401	0.3310
cole_hce2	0.4805	0.5368	0.4591	0.4752	0.4931	0.2884	0.2925	0.5368	0.4348	0.3255
utai_rebayct	0.2981	0.3331	0.2854	0.2948	0.5879	0.0998	0.1080	0.3331	0.2698	0.1786
cole_hce_ne	0.4799	0.5362	0.4582	0.4745	0.4872	0.2874	0.2915	0.5362	0.4343	0.3248
Hippocrates	0.5991	0.6137	0.5974	0.5900	0.6151	0.4697	0.4631	0.6176	0.5817	0.4332
utai_rebayct_2	0.3464	0.3870	0.3353	0.3443	0.4518	0.1784	0.1790	0.3870	0.3135	0.2149
Default MTI	0.5810	0.6039	0.5783	0.5720	0.5604	0.5205	0.4890	0.6012	0.5621	0.4171
FU System	0.5183	0.6107	0.4834	0.5225	0.5889	0.4857	0.4709	0.6113	0.4499	0.3688
FU_System_t25	0.5490	0.6075	0.5288	0.5482	0.5775	0.5020	0.4808	0.6079	0.5005	0.3931
FU_System_t15	0.5677	0.6042	0.5586	0.5630	0.5685	0.5148	0.4880	0.6042	0.5354	0.4078
Sisyphus	0.5933	0.6280	0.5712	0.5830	0.6347	0.4440	0.4437	0.6344	0.5571	0.4270
Macro	0.3965	0.3246	0.5751	0.3888	0.1493	0.4060	0.2703	0.3019	0.5772	0.2511
Micro	0.4918	0.5344	0.4436	0.4600	0.4906	0.1158	0.1063	0.5403	0.4513	0.3150
PerExample	0.4835	0.5103	0.4615	0.4674	0.4475	0.1224	0.1083	0.5076	0.4616	0.3203
spoon-spoon	0.3917	0.3695	0.4365	0.3772	0.4535	0.3505	0.3301	0.3586	0.4316	0.2420
fork-fork	0.3492	0.4984	0.2785	0.3356	0.4284	0.1191	0.1321	0.4852	0.2727	0.2147
spork-spork	0.3476	0.5191	0.2708	0.3341	0.4509	0.1151	0.1291	0.5058	0.2648	0.2137
FU_System_k15	0.5808	0.5965	0.5849	0.5719	0.5551	0.5279	0.4930	0.5932	0.5688	0.4165
FU_System_k25	0.5766	0.6009	0.5743	0.5694	0.5620	0.5220	0.4911	0.6001	0.5549	0.4141
Asclepius	0.5928	0.5974	0.5990	0.5827	0.6111	0.4488	0.4423	0.6019	0.5840	0.4257
Antinomyra SYS0	0.5967	0.6136	0.5965	0.5871	0.5907	0.4708	0.4555	0.6133	0.5810	0.4302
Antinomyra SYS3	0.5957	0.5960	0.6106	0.5873	0.5713	0.4735	0.4525	0.5965	0.5949	0.4300
Antinomyra SYS4	0.5922	0.6107	0.5929	0.5838	0.5803	0.4811	0.4640	0.6086	0.5766	0.4271
Antinomyra SYS1	0.6022	0.6194	0.6028	0.5929	0.5934	0.4650	0.4468	0.6189	0.5863	0.4360
L2R-n1	0.5831	0.6446	0.5513	0.5745	0.5864	0.4783	0.4677	0.6419	0.5341	0.4189
L2R-n2	0.5876	0.6316	0.5650	0.5782	0.5849	0.4980	0.4820	0.6325	0.5486	0.4226
L2R-n3	0.5843	0.6568	0.5439	0.5757	0.6029	0.4694	0.4635	0.6551	0.5273	0.4206
L2R-n4	0.5927	0.6271	0.5877	0.5857	0.5770	0.4963	0.4790	0.6205	0.5672	0.4297
L2R-n5	0.5976	0.6095	0.6067	0.5896	0.5734	0.5175	0.4938	0.6080	0.5876	0.4340
ERIAS-SIS	0.3153	0.3064	0.3234	0.3062	0.4047	0.2008	0.1847	0.3119	0.3188	0.1910
Antinomyra SYS2	0.5908	0.6081	0.5911	0.5818	0.5710	0.4620	0.4426	0.6072	0.5753	0.4244
ERIAS-SIS RUN2	0.3523	0.3405	0.3592	0.3403	0.4496	0.2501	0.2370	0.3485	0.3561	0.2174
ERIAS-SIS RUN3	0.5160	0.5061	0.5369	0.5066	0.4915	0.4036	0.3874	0.5105	0.5217	0.3536
Acc	0.3759	0.7997	0.2396	0.3466	0.7519	0.0306	0.0389	0.8215	0.2437	0.2262
SNUMedinfo1	0.4119	0.3446	0.5312	0.4012	0.2464	0.4739	0.3525	0.3376	0.5282	0.2600
SNUMedinfo2	0.4186	0.5995	0.3359	0.4065	0.4283	0.3570	0.3326	0.5710	0.3304	0.2670
SNUMedinfo3	0.4306	0.5213	0.3839	0.4181	0.3670	0.3895	0.3449	0.4986	0.3789	0.2756
SNUMedinfo4	0.4326	0.5289	0.3777	0.4194	0.3902	0.3849	0.3460	0.5173	0.3717	0.2768
SNUMedinfo5	0.4361	0.4656	0.4193	0.4238	0.3565	0.4109	0.3558	0.4652	0.4104	0.2796
ERIAS-SIS RUN4	0.5158	0.5059	0.5366	0.5063	0.4927	0.4025	0.3866	0.5102	0.5214	0.3534
ERIAS-SIS RUN5	0.5120	0.5022	0.5328	0.5027	0.4849	0.3981	0.3820	0.5066	0.5177	0.3498

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3118	0.5303	0.5780	0.5282	0.3225	0.3311
MTI First Line Index	0.4803	0.7869	0.6518	0.6900	0.5498	0.4527
cole_hce1	0.4187	0.7191	0.5790	0.6153	0.4874	0.3939
cole_hce2	0.4132	0.7156	0.5728	0.6096	0.4842	0.3874
utai_rebayct	0.2764	0.6036	0.3890	0.4494	0.3165	0.2655
cole_hce_ne	0.4116	0.7151	0.5700	0.6082	0.4833	0.3854
Hippocrates	0.5023	0.7603	0.7135	0.7174	0.5332	0.4995
utai_rebayct_2	0.3323	0.6357	0.5138	0.5436	0.3757	0.3207
Default MTI	0.4955	0.7570	0.7019	0.7069	0.5323	0.4919
FU System	0.4532	0.7765	0.6446	0.6845	0.5309	0.4177
FU_System_t25	0.4757	0.7669	0.6732	0.6969	0.5333	0.4537
FU_System_t15	0.4881	0.7608	0.6916	0.7042	0.5319	0.4768
Sisyphus	0.4953	0.7749	0.6876	0.7094	0.5433	0.4784
Macro	0.3761	0.5130	0.7236	0.5694	0.3345	0.4870
Micro	0.3948	0.7071	0.5394	0.5801	0.4717	0.3692
PerExample	0.4064	0.6878	0.5746	0.6012	0.4619	0.3884
spoon-spoon	0.3590	0.5824	0.5917	0.5573	0.3636	0.3937
fork-fork	0.3000	0.7049	0.3468	0.4308	0.4707	0.2407
spork-spork	0.2956	0.7212	0.3336	0.4225	0.4826	0.2322
FU_System_k15	0.4958	0.7509	0.7101	0.7086	0.5269	0.4971
FU_System_k25	0.4935	0.7558	0.7022	0.7073	0.5299	0.4891
Asclepius	0.4977	0.7494	0.7172	0.7144	0.5228	0.5005
Antinomyra SYS0	0.5010	0.7665	0.7035	0.7132	0.5369	0.4958
Antinomyra SYS3	0.5039	0.7547	0.7197	0.7179	0.5287	0.5077
Antinomyra SYS4	0.4977	0.7647	0.6990	0.7095	0.5354	0.4920
Antinomyra SYS1	0.5042	0.7678	0.7087	0.7167	0.5397	0.4999
L2R-n1	0.4873	0.7916	0.6654	0.7000	0.5562	0.4611
L2R-n2	0.4945	0.7725	0.6893	0.7070	0.5419	0.4819
L2R-n3	0.4877	0.7925	0.6647	0.7000	0.5557	0.4619
L2R-n4	0.4971	0.7742	0.6944	0.7087	0.5451	0.4878
L2R-n5	0.5046	0.7513	0.7251	0.7169	0.5260	0.5147
ERIAS-SIS	0.3298	0.4990	0.5548	0.5082	0.3266	0.3521
Antinomyra SYS2	0.4947	0.7617	0.6983	0.7080	0.5313	0.4901
ERIAS-SIS RUN2	0.3473	0.5263	0.5786	0.5333	0.3449	0.3699
ERIAS-SIS RUN3	0.4465	0.6878	0.6712	0.6605	0.4646	0.4542
Acc	0.2638	0.8700	0.2491	0.3557	0.5645	0.1843
SNUMedinfo1	0.3604	0.4968	0.7089	0.5586	0.3098	0.4772
SNUMedinfo2	0.3450	0.7137	0.4401	0.5092	0.4831	0.2968
SNUMedinfo3	0.3629	0.6543	0.5200	0.5435	0.4355	0.3474
SNUMedinfo4	0.3636	0.6610	0.5105	0.5434	0.4405	0.3416
SNUMedinfo5	0.3726	0.6102	0.5774	0.5665	0.3973	0.3835
ERIAS-SIS RUN4	0.4461	0.6871	0.6707	0.6599	0.4640	0.4540
ERIAS-SIS RUN5	0.4445	0.6863	0.6685	0.6583	0.4627	0.4519

+ Test batch 3, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2635	0.2446	0.3163	0.2594	0.3662	0.4271	0.3536	0.2324	0.3041	0.1545
MTI First Line Index	0.5654	0.6260	0.5356	0.5560	0.5881	0.4823	0.4645	0.6195	0.5201	0.4008
cole_hce1	0.4790	0.5164	0.4693	0.4718	0.4815	0.3094	0.3096	0.5164	0.4465	0.3235
cole_hce2	0.4706	0.5075	0.4596	0.4629	0.4818	0.2912	0.2927	0.5075	0.4388	0.3156
utai_rebayct	0.2923	0.3152	0.2839	0.2864	0.5725	0.0988	0.1055	0.3152	0.2726	0.1735
cole_hce_ne	0.4735	0.5106	0.4631	0.4659	0.4813	0.2928	0.2942	0.5106	0.4415	0.3177
Hippocrates	0.5865	0.5861	0.5932	0.5729	0.5931	0.4721	0.4607	0.5911	0.5820	0.4175
utai_rebayct_2	0.3420	0.3688	0.3390	0.3383	0.4417	0.1823	0.1818	0.3688	0.3188	0.2109
Default MTI	0.5719	0.5881	0.5786	0.5627	0.5543	0.5235	0.4895	0.5834	0.5608	0.4062
Sisyphus	0.5814	0.5987	0.5676	0.5661	0.6111	0.4468	0.4414	0.6066	0.5583	0.4114
Macro	0.3873	0.3128	0.5829	0.3809	0.1465	0.4254	0.2872	0.2901	0.5828	0.2449
Micro	0.4831	0.5059	0.4458	0.4487	0.4705	0.1294	0.1158	0.5140	0.4556	0.3059
PerExample	0.4747	0.4906	0.4654	0.4579	0.4338	0.1360	0.1179	0.4841	0.4657	0.3128
Asclepius	0.5826	0.5734	0.5970	0.5683	0.5929	0.4537	0.4400	0.5786	0.5867	0.4127
Antinomyra SYS0	0.5980	0.6047	0.6044	0.5883	0.5863	0.4747	0.4575	0.6145	0.5824	0.4312
Antinomyra SYS3	0.6001	0.5932	0.6235	0.5899	0.5548	0.5299	0.5006	0.5930	0.6074	0.4328
Antinomyra SYS4	0.5948	0.5879	0.6183	0.5848	0.5537	0.5296	0.4989	0.5877	0.6020	0.4275
Antinomyra SYS1	0.5992	0.6060	0.6063	0.5898	0.5816	0.4840	0.4652	0.6157	0.5835	0.4327
L2R-n1	0.5838	0.6184	0.5771	0.5741	0.5718	0.4977	0.4804	0.6135	0.5568	0.4180
L2R-n2	0.5843	0.6009	0.5878	0.5735	0.5637	0.5135	0.4899	0.6003	0.5691	0.4172
L2R-n3	0.5841	0.6134	0.5775	0.5733	0.5742	0.4995	0.4820	0.6117	0.5589	0.4173
L2R-n4	0.5868	0.6077	0.5953	0.5779	0.5685	0.5031	0.4831	0.6004	0.5738	0.4215
L2R-n5	0.5885	0.5890	0.6096	0.5784	0.5603	0.5195	0.4931	0.5868	0.5901	0.4219
ERIAS-SIS	0.5029	0.5723	0.4585	0.4854	0.5492	0.3284	0.3312	0.5690	0.4506	0.3368
Antinomyra SYS2	0.5985	0.5901	0.6204	0.5870	0.5611	0.5089	0.4800	0.5914	0.6057	0.4299
ERIAS-SIS RUN2	0.5061	0.5604	0.4836	0.4842	0.5021	0.3602	0.3510	0.5326	0.4822	0.3356
ERIAS-SIS RUN3	0.5093	0.5523	0.4860	0.4903	0.5185	0.3587	0.3532	0.5440	0.4788	0.3409
Acc	0.3789	0.7747	0.2433	0.3461	0.7475	0.0371	0.0455	0.7999	0.2482	0.2273
SNUMedinfo1	0.4960	0.5177	0.4800	0.4776	0.4133	0.4132	0.3740	0.5194	0.4746	0.3285
SNUMedinfo2	0.4882	0.5783	0.4280	0.4673	0.4429	0.3756	0.3516	0.5685	0.4277	0.3206
SNUMedinfo3	0.4969	0.5022	0.4983	0.4772	0.3822	0.4281	0.3783	0.4967	0.4971	0.3280
SNUMedinfo4	0.4854	0.4300	0.5582	0.4681	0.3344	0.4712	0.3929	0.4327	0.5526	0.3183
SNUMedinfo5	0.4883	0.4667	0.5181	0.4732	0.3796	0.4373	0.3840	0.4720	0.5059	0.3234
ERIAS-SIS RUN4	0.5097	0.5527	0.4839	0.4903	0.5199	0.3573	0.3522	0.5441	0.4793	0.3410
ERIAS-SIS RUN5	0.5093	0.5707	0.4702	0.4930	0.5473	0.3475	0.3490	0.5689	0.4610	0.3432
VaneSSa@UA 0	0.4676	0.4817	0.4734	0.4583	0.5235	0.3160	0.3150	0.4817	0.4544	0.3118

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3091	0.5183	0.5862	0.5241	0.3131	0.3357
MTI First Line Index	0.4737	0.7774	0.6581	0.6869	0.5362	0.4526
cole_hce1	0.4167	0.7020	0.5958	0.6183	0.4724	0.4008
cole_hce2	0.4094	0.6976	0.5848	0.6103	0.4664	0.3927
utai_rebayct	0.2755	0.5852	0.4044	0.4548	0.3054	0.2722
cole_hce_ne	0.4094	0.6945	0.5838	0.6080	0.4659	0.3933
Hippocrates	0.4923	0.7401	0.7204	0.7094	0.5143	0.5002
utai_rebayct_2	0.3319	0.6163	0.5276	0.5439	0.3653	0.3281
Default MTI	0.4837	0.7458	0.7084	0.7032	0.5129	0.4874
Sisyphus	0.4860	0.7555	0.6953	0.7028	0.5243	0.4795
Macro	0.3711	0.4971	0.7386	0.5625	0.3238	0.4961
Micro	0.3929	0.6828	0.5533	0.5781	0.4549	0.3776
PerExample	0.4021	0.6692	0.5886	0.5985	0.4469	0.3945
Asclepius	0.4877	0.7315	0.7214	0.7056	0.5037	0.5014
Antinomyra SYS0	0.4986	0.7575	0.7163	0.7154	0.5277	0.5000
Antinomyra SYS3	0.5020	0.7479	0.7348	0.7201	0.5191	0.5156
Antinomyra SYS4	0.5009	0.7473	0.7330	0.7188	0.5195	0.5129
Antinomyra SYS1	0.5019	0.7598	0.7192	0.7184	0.5318	0.5021
L2R-n1	0.4875	0.7669	0.6949	0.7030	0.5365	0.4800
L2R-n2	0.4911	0.7450	0.7192	0.7077	0.5183	0.4991
L2R-n3	0.4893	0.7554	0.7072	0.7054	0.5265	0.4899
L2R-n4	0.4905	0.7585	0.7076	0.7060	0.5287	0.4920
L2R-n5	0.4945	0.7346	0.7355	0.7114	0.5099	0.5132
ERIAS-SIS	0.4183	0.7468	0.5723	0.6161	0.5018	0.3876
Antinomyra SYS2	0.5025	0.7482	0.7328	0.7190	0.5207	0.5149
ERIAS-SIS RUN2	0.4182	0.7326	0.5965	0.6172	0.4869	0.4057
ERIAS-SIS RUN3	0.4252	0.7279	0.6057	0.6280	0.4867	0.4097
Acc	0.2676	0.8528	0.2609	0.3637	0.5517	0.1901
SNUMedinfo1	0.4000	0.6620	0.6028	0.6004	0.4312	0.4086
SNUMedinfo2	0.3834	0.7110	0.5277	0.5693	0.4660	0.3600
SNUMedinfo3	0.4003	0.6463	0.6229	0.6008	0.4195	0.4246
SNUMedinfo4	0.4020	0.5787	0.6998	0.6068	0.3727	0.4786
SNUMedinfo5	0.4031	0.6144	0.6551	0.6076	0.3988	0.4440
ERIAS-SIS RUN4	0.4253	0.7286	0.6038	0.6277	0.4875	0.4085
ERIAS-SIS RUN5	0.4253	0.7451	0.5868	0.6267	0.5017	0.3976
VaneSSa@UA 0	0.4065	0.6850	0.5939	0.6108	0.4493	0.3990

+ Test batch 3, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2668	0.2559	0.3089	0.2632	0.3843	0.4187	0.3484	0.2414	0.2983	0.1570
MTI First Line Index	0.5550	0.6315	0.5125	0.5455	0.6072	0.4726	0.4625	0.6270	0.4978	0.3923
cole_hce1	0.4767	0.5040	0.4778	0.4710	0.4760	0.3120	0.3129	0.5040	0.4523	0.3214
cole_hce2	0.4700	0.4969	0.4699	0.4638	0.4732	0.2921	0.2935	0.4969	0.4459	0.3157
utai_rebayct	0.2950	0.3118	0.2937	0.2900	0.5939	0.1182	0.1281	0.3118	0.2798	0.1760
cole_hce_ne	0.4701	0.4970	0.4686	0.4634	0.4689	0.2916	0.2943	0.4970	0.4460	0.3156
Panacea	0.5869	0.6294	0.5565	0.5742	0.6292	0.4430	0.4451	0.6376	0.5436	0.4190
Hippocrates	0.5855	0.6019	0.5790	0.5743	0.6163	0.4515	0.4499	0.6072	0.5653	0.4184
Galen	0.5857	0.6047	0.5767	0.5743	0.6162	0.4513	0.4497	0.6104	0.5629	0.4184
utai_rebayct_2	0.3465	0.3663	0.3493	0.3429	0.4455	0.1965	0.1949	0.3663	0.3287	0.2140
Default MTI	0.5669	0.5988	0.5598	0.5592	0.5732	0.5118	0.4845	0.5935	0.5425	0.4044
Sisyphus	0.5869	0.6295	0.5565	0.5743	0.6291	0.4429	0.4450	0.6376	0.5436	0.4190
Macro	0.3934	0.3306	0.5526	0.3839	0.1521	0.3711	0.2379	0.3058	0.5516	0.2472
Micro	0.4753	0.5165	0.4241	0.4386	0.4862	0.0990	0.0919	0.5275	0.4325	0.2984
PerExample	0.4643	0.4915	0.4438	0.4483	0.3969	0.1032	0.0918	0.4857	0.4447	0.3053
Antinomyra SYS0	0.6079	0.6266	0.6059	0.6007	0.5906	0.5189	0.5041	0.6361	0.5821	0.4441
Antinomyra SYS3	0.6078	0.6265	0.6058	0.6006	0.5880	0.5230	0.5064	0.6359	0.5820	0.4440
Antinomyra SYS4	0.6068	0.6257	0.6053	0.5999	0.5924	0.5191	0.5045	0.6349	0.5811	0.4434
Antinomyra SYS1	0.6073	0.6262	0.6056	0.6003	0.5960	0.5179	0.5041	0.6354	0.5815	0.4438
L2R-n1	0.6052	0.6278	0.6099	0.5979	0.5990	0.5237	0.5045	0.6222	0.5891	0.4422
L2R-n2	0.6076	0.6210	0.6148	0.5991	0.5993	0.5321	0.5097	0.6203	0.5954	0.4437
L2R-n3	0.6068	0.6317	0.6053	0.5988	0.6079	0.5191	0.5022	0.6299	0.5854	0.4435
L2R-n4	0.5841	0.6007	0.5930	0.5763	0.5770	0.5056	0.4845	0.5956	0.5731	0.4206
L2R-n5	0.5845	0.6126	0.5810	0.5763	0.5936	0.4943	0.4793	0.6089	0.5620	0.4209
ERIAS-SIS	0.4793	0.5801	0.4167	0.4655	0.5903	0.2944	0.3113	0.5874	0.4047	0.3193
Antinomyra SYS2	0.6079	0.6266	0.6058	0.6006	0.5850	0.5198	0.5043	0.6361	0.5821	0.4442
ERIAS-SIS RUN2	0.4991	0.5222	0.5053	0.4802	0.4783	0.3841	0.3655	0.4963	0.5019	0.3306
ERIAS-SIS RUN3	0.4887	0.5571	0.4469	0.4738	0.5530	0.3329	0.3410	0.5561	0.4359	0.3257
Acc	0.3519	0.7550	0.2208	0.3193	0.7633	0.0254	0.0336	0.8040	0.2253	0.2076
SNUMedinfo2	0.5292	0.5497	0.5368	0.5127	0.4819	0.4472	0.4196	0.5251	0.5334	0.3582
SNUMedinfo3	0.5291	0.5495	0.5369	0.5127	0.4815	0.4472	0.4196	0.5249	0.5335	0.3581
SNUMedinfo4	0.4955	0.4576	0.5980	0.4877	0.4278	0.4934	0.4408	0.4260	0.5920	0.3331
SNUMedinfo5	0.5059	0.4763	0.5894	0.4963	0.4350	0.4869	0.4385	0.4461	0.5844	0.3414

BioASQ Participants Area

Evaluation Overview

Test Results for Task 2a

+ Dry-run test

Annotated articles:2703/3186

Flat Measures

Hierarchical Measures

+ Test batch 1, week 1

Annotated articles:3319/4440

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:3734/4721

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:3884/4802

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:2431/3579

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:3693/5299

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:3422/4085

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:2788/3496

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:3274/4524

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:3923/5407

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:3666/5454

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:3009/4342

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:5883/8840

Flat Measures