BioASQ Participants Area

Test Results for Task 5a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Dry Run 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6329	0.6857	0.6080	0.6233	0.6516	0.5606	0.5422	0.6764	0.5946	0.4721
Default MTI	0.6361	0.6407	0.6587	0.6281	0.6104	0.5978	0.5582	0.6294	0.6429	0.4749

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5035	0.7954	0.7055	0.7268	0.5551	0.4882
Default MTI	0.5102	0.7594	0.7538	0.7366	0.5281	0.5238

+ Dry Run 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6108	0.6727	0.5861	0.6043	0.6242	0.5350	0.5144	0.6620	0.5669	0.4506
Default MTI	0.6190	0.6317	0.6376	0.6132	0.5812	0.5785	0.5354	0.6211	0.6169	0.4591
MZ1	0.4745	0.6931	0.3881	0.4632	0.4074	0.1988	0.2048	0.6559	0.3717	0.3179

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4916	0.7870	0.6916	0.7142	0.5471	0.4740
Default MTI	0.5039	0.7508	0.7413	0.7259	0.5254	0.5139
MZ1	0.3600	0.7892	0.4669	0.5468	0.5194	0.3056

+ Test batch 1, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6121	0.6687	0.5834	0.6011	0.6248	0.5377	0.5164	0.6618	0.5692	0.4473
Default MTI	0.6226	0.6347	0.6343	0.6134	0.5889	0.5775	0.5379	0.6275	0.6177	0.4592
auth1	0.6254	0.6160	0.6440	0.6125	0.6057	0.5136	0.4970	0.6219	0.6291	0.4572
auth2	0.6324	0.6226	0.6502	0.6188	0.6133	0.5004	0.4832	0.6288	0.6361	0.4643
auth3	0.6335	0.6485	0.6236	0.6182	0.6317	0.4817	0.4733	0.6562	0.6123	0.4643
Search system 1	0.2707	0.2650	0.3065	0.2683	0.4201	0.4192	0.3510	0.2505	0.2944	0.1616
Search system 2	0.2570	0.2409	0.3111	0.2555	0.4039	0.4235	0.3465	0.2255	0.2988	0.1524
Search system 3	0.2103	0.2941	0.1801	0.2064	0.4012	0.2578	0.2572	0.2851	0.1665	0.1219
Search system 4	0.2430	0.2094	0.3595	0.2468	0.2936	0.4872	0.3661	0.1865	0.3485	0.1457
Search system 5	0.2559	0.2297	0.3349	0.2562	0.3136	0.4583	0.3629	0.2112	0.3244	0.1523
MZ1	0.4649	0.4888	0.4863	0.4595	0.5764	0.1913	0.2019	0.4678	0.4621	0.3101
DeepMeSH1	0.6607	0.7099	0.6346	0.6504	0.6687	0.5312	0.5283	0.7099	0.6178	0.4994
DeepMeSH2	0.6578	0.7240	0.6211	0.6494	0.6827	0.5166	0.5182	0.7280	0.5999	0.4984
DeepMeSH3	0.6498	0.6978	0.6238	0.6386	0.6465	0.5081	0.5033	0.6991	0.6070	0.4865
DeepMeSH4	0.6428	0.7083	0.6052	0.6335	0.6557	0.4791	0.4821	0.7139	0.5846	0.4809
DeepMeSH5	0.6607	0.7099	0.6346	0.6504	0.6687	0.5312	0.5283	0.7099	0.6178	0.4994

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4888	0.7844	0.6886	0.7113	0.5447	0.4711
Default MTI	0.5026	0.7552	0.7373	0.7258	0.5265	0.5101
auth1	0.5041	0.7430	0.7439	0.7250	0.5203	0.5158
auth2	0.5081	0.7489	0.7452	0.7287	0.5254	0.5190
auth3	0.5039	0.7732	0.7178	0.7256	0.5398	0.4979
Search system 1	0.2980	0.5086	0.5359	0.4994	0.2954	0.3332
Search system 2	0.2936	0.4923	0.5413	0.4929	0.2848	0.3372
Search system 3	0.2602	0.5733	0.3739	0.4243	0.3450	0.2320
Search system 4	0.2823	0.4368	0.5998	0.4819	0.2476	0.3757
Search system 5	0.2864	0.4564	0.5805	0.4885	0.2597	0.3605
MZ1	0.3872	0.6450	0.6050	0.5943	0.4274	0.3906
DeepMeSH1	0.5269	0.8156	0.7234	0.7472	0.5787	0.5089
DeepMeSH2	0.5234	0.8261	0.7080	0.7432	0.5871	0.4966
DeepMeSH3	0.5183	0.8089	0.7142	0.7387	0.5712	0.5002
DeepMeSH4	0.5111	0.8186	0.6950	0.7323	0.5758	0.4840
DeepMeSH5	0.5269	0.8156	0.7234	0.7472	0.5787	0.5089

+ Test batch 1, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6125	0.6703	0.5854	0.6025	0.6266	0.5433	0.5199	0.6598	0.5715	0.4488
Default MTI	0.6230	0.6358	0.6390	0.6161	0.5844	0.5833	0.5386	0.6231	0.6230	0.4623
auth1	0.6272	0.6219	0.6456	0.6165	0.6150	0.5117	0.4979	0.6235	0.6310	0.4611
auth2	0.6346	0.6289	0.6523	0.6232	0.6250	0.4996	0.4849	0.6309	0.6384	0.4684
auth3	0.6355	0.6565	0.6251	0.6229	0.6446	0.4813	0.4750	0.6592	0.6135	0.4684
Search system 1	0.2752	0.2699	0.3108	0.2719	0.4233	0.4245	0.3567	0.2546	0.2994	0.1640
Search system 2	0.2612	0.2449	0.3154	0.2589	0.4080	0.4288	0.3521	0.2291	0.3038	0.1545
Search system 3	0.2127	0.2983	0.1809	0.2084	0.4071	0.2599	0.2605	0.2902	0.1678	0.1237
Search system 4	0.2469	0.2113	0.3665	0.2498	0.2971	0.4935	0.3733	0.1893	0.3549	0.1476
Search system 5	0.2587	0.2313	0.3400	0.2585	0.3156	0.4621	0.3674	0.2131	0.3290	0.1538
MZ1	0.5546	0.5565	0.5610	0.5390	0.5302	0.3582	0.3272	0.5566	0.5527	0.3841
DeepMeSH1	0.6620	0.7133	0.6366	0.6533	0.6732	0.5345	0.5307	0.7091	0.6208	0.5019
DeepMeSH2	0.6644	0.7011	0.6496	0.6563	0.6635	0.5524	0.5415	0.6995	0.6326	0.5050
DeepMeSH3	0.6524	0.7027	0.6270	0.6433	0.6507	0.5159	0.5112	0.6994	0.6113	0.4905
DeepMeSH4	0.6490	0.6808	0.6371	0.6401	0.6267	0.5192	0.5092	0.6806	0.6202	0.4867
DeepMeSH5	0.6594	0.7287	0.6220	0.6522	0.6905	0.5246	0.5265	0.7289	0.6020	0.5010
Sequencer	0.0001	0.0002	0.0001	0.0001	0.0002	0.0001	0.0001	0.0002	0.0001	0.0001

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4918	0.7868	0.6907	0.7130	0.5457	0.4758
Default MTI	0.5073	0.7559	0.7428	0.7289	0.5275	0.5188
auth1	0.5099	0.7493	0.7467	0.7300	0.5253	0.5219
auth2	0.5140	0.7555	0.7492	0.7345	0.5304	0.5251
auth3	0.5103	0.7791	0.7198	0.7298	0.5464	0.5042
Search system 1	0.2985	0.5102	0.5369	0.4995	0.2955	0.3359
Search system 2	0.2940	0.4940	0.5426	0.4932	0.2846	0.3401
Search system 3	0.2617	0.5779	0.3719	0.4240	0.3480	0.2332
Search system 4	0.2830	0.4366	0.6030	0.4819	0.2474	0.3811
Search system 5	0.2870	0.4557	0.5828	0.4880	0.2594	0.3649
MZ1	0.4472	0.6966	0.6670	0.6596	0.4729	0.4530
DeepMeSH1	0.5311	0.8176	0.7260	0.7496	0.5803	0.5158
DeepMeSH2	0.5359	0.8088	0.7387	0.7537	0.5760	0.5264
DeepMeSH3	0.5227	0.8108	0.7173	0.7416	0.5722	0.5072
DeepMeSH4	0.5234	0.7960	0.7298	0.7432	0.5622	0.5150
DeepMeSH5	0.5289	0.8292	0.7107	0.7462	0.5897	0.5042
Sequencer	0.1002	0.1506	0.0906	0.1008	0.1497	0.0877

+ Test batch 1, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6223	0.6807	0.5917	0.6123	0.6475	0.5496	0.5318	0.6728	0.5788	0.4610
Default MTI	0.6342	0.6432	0.6496	0.6266	0.6097	0.5894	0.5509	0.6343	0.6340	0.4749
auth1	0.6223	0.6126	0.6429	0.6112	0.6213	0.5123	0.5010	0.6157	0.6291	0.4548
auth2	0.6307	0.6210	0.6512	0.6194	0.6342	0.4977	0.4865	0.6240	0.6375	0.4637
auth3	0.6288	0.6514	0.6166	0.6167	0.6547	0.4768	0.4737	0.6550	0.6046	0.4615
Search system 1	0.2843	0.2756	0.3175	0.2810	0.4391	0.4267	0.3576	0.2637	0.3083	0.1709
Search system 2	0.2699	0.2502	0.3219	0.2677	0.4242	0.4305	0.3529	0.2374	0.3126	0.1611
Search system 3	0.2143	0.2986	0.1816	0.2094	0.4172	0.2601	0.2620	0.2923	0.1691	0.1243
Search system 4	0.2553	0.2155	0.3740	0.2583	0.3114	0.4954	0.3723	0.1964	0.3644	0.1536
Search system 5	0.2672	0.2356	0.3475	0.2670	0.3305	0.4670	0.3694	0.2207	0.3384	0.1599
DeepMeSH1	0.6615	0.7145	0.6347	0.6531	0.6804	0.5373	0.5367	0.7106	0.6188	0.5016
DeepMeSH2	0.6639	0.7024	0.6482	0.6561	0.6710	0.5539	0.5467	0.7000	0.6313	0.5049
DeepMeSH3	0.6478	0.6991	0.6206	0.6382	0.6597	0.5102	0.5098	0.6965	0.6054	0.4847
DeepMeSH4	0.6439	0.6766	0.6309	0.6349	0.6330	0.5154	0.5081	0.6761	0.6146	0.4807
DeepMeSH5	0.6577	0.7281	0.6197	0.6507	0.6983	0.5243	0.5297	0.7273	0.6002	0.4992

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4981	0.7932	0.6965	0.7212	0.5517	0.4805
Default MTI	0.5140	0.7609	0.7497	0.7362	0.5332	0.5246
auth1	0.4986	0.7396	0.7432	0.7243	0.5125	0.5113
auth2	0.5045	0.7481	0.7470	0.7305	0.5191	0.5167
auth3	0.4982	0.7754	0.7089	0.7229	0.5363	0.4897
Search system 1	0.3025	0.5114	0.5423	0.5059	0.2984	0.3375
Search system 2	0.2983	0.4953	0.5478	0.4997	0.2880	0.3417
Search system 3	0.2602	0.5693	0.3697	0.4206	0.3472	0.2310
Search system 4	0.2878	0.4368	0.6102	0.4881	0.2509	0.3823
Search system 5	0.2921	0.4562	0.5902	0.4944	0.2635	0.3667
DeepMeSH1	0.5255	0.8160	0.7229	0.7476	0.5765	0.5079
DeepMeSH2	0.5296	0.8070	0.7362	0.7520	0.5714	0.5184
DeepMeSH3	0.5124	0.8067	0.7117	0.7370	0.5635	0.4950
DeepMeSH4	0.5130	0.7925	0.7239	0.7385	0.5530	0.5037
DeepMeSH5	0.5205	0.8270	0.7076	0.7438	0.5834	0.4943

+ Test batch 1, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6180	0.6650	0.5918	0.6050	0.6119	0.5509	0.5252	0.6575	0.5829	0.4526
Default MTI	0.6216	0.6247	0.6411	0.6119	0.5703	0.5891	0.5428	0.6148	0.6286	0.4594
MZ1	0.4940	0.6352	0.4053	0.4774	0.6168	0.2575	0.2551	0.6495	0.3985	0.3299
DeepMeSH1	0.6633	0.7058	0.6416	0.6525	0.6552	0.5462	0.5374	0.7025	0.6282	0.5025
DeepMeSH2	0.6655	0.6923	0.6584	0.6563	0.6410	0.5639	0.5471	0.6895	0.6432	0.5059
DeepMeSH3	0.6514	0.6924	0.6300	0.6399	0.6360	0.5191	0.5095	0.6898	0.6170	0.4877
DeepMeSH4	0.6474	0.6672	0.6431	0.6369	0.6076	0.5267	0.5095	0.6671	0.6289	0.4839
DeepMeSH5	0.6617	0.7200	0.6300	0.6527	0.6660	0.5318	0.5280	0.7198	0.6122	0.5024
Sequencer	0.0941	0.0935	0.0954	0.0883	0.0014	0.0013	0.0010	0.0920	0.0964	0.0494
RACAI-DumbClassifier	0.2470	0.7319	0.1433	0.2262	0.5979	0.0012	0.0015	0.7681	0.1472	0.1381

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4876	0.7813	0.6980	0.7156	0.5365	0.4748
Default MTI	0.5002	0.7454	0.7492	0.7270	0.5158	0.5161
MZ1	0.3763	0.7716	0.4894	0.5745	0.5095	0.3182
DeepMeSH1	0.5235	0.8090	0.7319	0.7494	0.5693	0.5115
DeepMeSH2	0.5284	0.7981	0.7480	0.7543	0.5623	0.5249
DeepMeSH3	0.5126	0.8003	0.7215	0.7394	0.5581	0.5010
DeepMeSH4	0.5126	0.7804	0.7355	0.7396	0.5440	0.5105
DeepMeSH5	0.5211	0.8191	0.7196	0.7471	0.5762	0.5016
Sequencer	0.1673	0.2326	0.2264	0.2152	0.1795	0.1730
RACAI-DumbClassifier	0.1580	0.7852	0.1266	0.2113	0.4559	0.0983

+ Test batch 1, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6150	0.6753	0.5894	0.6088	0.6376	0.5331	0.5172	0.6658	0.5715	0.4556
Default MTI	0.6267	0.6386	0.6437	0.6212	0.5967	0.5721	0.5362	0.6299	0.6235	0.4684
auth1	0.6241	0.6122	0.6484	0.6133	0.6152	0.5095	0.4969	0.6167	0.6317	0.4571
auth2	0.6317	0.6200	0.6562	0.6209	0.6257	0.4970	0.4836	0.6243	0.6394	0.4653
auth3	0.6319	0.6498	0.6263	0.6205	0.6440	0.4763	0.4713	0.6543	0.6110	0.4654
auth4	0.6204	0.6184	0.6374	0.6073	0.6492	0.4812	0.4787	0.6142	0.6267	0.4514
auth5	0.6321	0.6249	0.6514	0.6209	0.6142	0.5125	0.4991	0.6294	0.6349	0.4655
Search system 1	0.2781	0.2688	0.3156	0.2758	0.4284	0.4115	0.3476	0.2581	0.3014	0.1670
Search system 2	0.2642	0.2436	0.3199	0.2623	0.4135	0.4152	0.3435	0.2326	0.3057	0.1571
Search system 3	0.2140	0.2933	0.1859	0.2102	0.3970	0.2484	0.2493	0.2876	0.1704	0.1250
Search system 4	0.2497	0.2105	0.3679	0.2525	0.2997	0.4758	0.3629	0.1928	0.3545	0.1496
Search system 5	0.2621	0.2312	0.3445	0.2625	0.3183	0.4487	0.3593	0.2171	0.3306	0.1567
MZ1	0.5502	0.6075	0.5206	0.5387	0.5932	0.3032	0.3014	0.6047	0.5048	0.3833
DeepMeSH1	0.6629	0.7154	0.6366	0.6550	0.6809	0.5266	0.5272	0.7138	0.6187	0.5037
DeepMeSH2	0.6633	0.7014	0.6496	0.6567	0.6651	0.5417	0.5356	0.7010	0.6294	0.5054
DeepMeSH3	0.6503	0.7030	0.6227	0.6414	0.6604	0.5025	0.5036	0.7021	0.6056	0.4884
DeepMeSH4	0.6464	0.6797	0.6338	0.6381	0.6328	0.5093	0.5032	0.6813	0.6150	0.4843
DeepMeSH5	0.6578	0.7275	0.6219	0.6521	0.6909	0.5122	0.5177	0.7291	0.5993	0.5007
Sequencer	0.1072	0.1117	0.1064	0.1014	0.0019	0.0014	0.0013	0.1076	0.1069	0.0572

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4956	0.7916	0.6912	0.7167	0.5517	0.4764
Default MTI	0.5114	0.7593	0.7432	0.7318	0.5329	0.5205
auth1	0.5051	0.7397	0.7487	0.7270	0.5170	0.5199
auth2	0.5097	0.7463	0.7516	0.7319	0.5229	0.5235
auth3	0.5054	0.7730	0.7209	0.7279	0.5406	0.5001
auth4	0.5018	0.7518	0.7309	0.7201	0.5256	0.5101
auth5	0.5095	0.7492	0.7508	0.7328	0.5225	0.5236
Search system 1	0.3017	0.5039	0.5365	0.4984	0.2979	0.3369
Search system 2	0.2978	0.4877	0.5420	0.4920	0.2877	0.3413
Search system 3	0.2627	0.5646	0.3739	0.4209	0.3467	0.2351
Search system 4	0.2871	0.4311	0.6005	0.4801	0.2510	0.3800
Search system 5	0.2914	0.4504	0.5814	0.4867	0.2634	0.3651
MZ1	0.4359	0.7458	0.6161	0.6499	0.5029	0.4127
DeepMeSH1	0.5293	0.8195	0.7246	0.7503	0.5827	0.5104
DeepMeSH2	0.5323	0.8084	0.7381	0.7536	0.5755	0.5203
DeepMeSH3	0.5175	0.8123	0.7136	0.7407	0.5719	0.4981
DeepMeSH4	0.5176	0.7950	0.7248	0.7400	0.5595	0.5068
DeepMeSH5	0.5241	0.8290	0.7101	0.7461	0.5875	0.4977
Sequencer	0.1692	0.2425	0.2216	0.2152	0.1868	0.1730

+ Test batch 2, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6180	0.6787	0.5917	0.6109	0.6511	0.5405	0.5241	0.6693	0.5741	0.4595
Default MTI	0.6274	0.6399	0.6463	0.6218	0.6104	0.5808	0.5446	0.6309	0.6239	0.4691
auth1	0.6217	0.6082	0.6502	0.6118	0.6214	0.5169	0.5029	0.6118	0.6320	0.4554
auth2	0.6297	0.6157	0.6573	0.6190	0.6310	0.5041	0.4905	0.6196	0.6400	0.4631
auth3	0.6306	0.6464	0.6269	0.6188	0.6512	0.4842	0.4796	0.6507	0.6116	0.4639
auth4	0.6296	0.6213	0.6520	0.6192	0.6185	0.5195	0.5046	0.6250	0.6343	0.4633
auth5	0.6173	0.6140	0.6383	0.6050	0.6586	0.4880	0.4862	0.6085	0.6264	0.4486
iria-1	0.4527	0.4050	0.5221	0.4430	0.3765	0.3710	0.3372	0.4094	0.5063	0.2975
iria-2	0.4099	0.3710	0.4691	0.4003	0.2849	0.3356	0.2941	0.3719	0.4565	0.2629
iria-3	0.4685	0.4179	0.5419	0.4588	0.4294	0.3761	0.3474	0.4233	0.5245	0.3105
Search system 1	0.2766	0.2672	0.3114	0.2733	0.4398	0.4182	0.3540	0.2562	0.3006	0.1655
Search system 2	0.2631	0.2434	0.3152	0.2605	0.4228	0.4213	0.3493	0.2316	0.3045	0.1561
Search system 3	0.2092	0.2877	0.1798	0.2044	0.4043	0.2470	0.2483	0.2837	0.1657	0.1212
Search system 4	0.2484	0.2090	0.3638	0.2506	0.3105	0.4824	0.3695	0.1915	0.3534	0.1484
Search system 5	0.2602	0.2288	0.3396	0.2596	0.3289	0.4555	0.3657	0.2151	0.3292	0.1549
MZ1	0.5393	0.5654	0.5386	0.5284	0.5925	0.3102	0.3068	0.5618	0.5185	0.3738
DeepMeSH1	0.6602	0.7158	0.6365	0.6541	0.6926	0.5316	0.5338	0.7102	0.6169	0.5031
DeepMeSH2	0.6587	0.7139	0.6345	0.6525	0.6896	0.5274	0.5295	0.7091	0.6150	0.5012
DeepMeSH3	0.6474	0.6991	0.6246	0.6401	0.6717	0.5019	0.5040	0.6950	0.6060	0.4868
DeepMeSH4	0.6394	0.6897	0.6161	0.6312	0.6624	0.4837	0.4873	0.6864	0.5985	0.4770
DeepMeSH5	0.6596	0.6974	0.6485	0.6535	0.6731	0.5437	0.5375	0.6962	0.6266	0.5019
Sequencer	0.1036	0.1005	0.1058	0.0970	0.0013	0.0011	0.0010	0.0998	0.1077	0.0548
MZ2	0.5481	0.6032	0.5227	0.5370	0.6022	0.3038	0.3022	0.6001	0.5044	0.3819
MZ3	0.4056	0.3963	0.4388	0.4013	0.5130	0.2453	0.2404	0.3945	0.4173	0.2595

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4988	0.7929	0.6977	0.7212	0.5525	0.4821
Default MTI	0.5109	0.7584	0.7482	0.7332	0.5305	0.5223
auth1	0.5038	0.7371	0.7541	0.7283	0.5128	0.5217
auth2	0.5080	0.7426	0.7561	0.7320	0.5182	0.5247
auth3	0.5050	0.7691	0.7264	0.7291	0.5364	0.5026
auth4	0.5065	0.7461	0.7551	0.7331	0.5162	0.5234
auth5	0.4993	0.7495	0.7349	0.7211	0.5202	0.5102
iria-1	0.4010	0.5702	0.6721	0.6006	0.3831	0.4457
iria-2	0.3753	0.5387	0.6376	0.5659	0.3580	0.4201
iria-3	0.4092	0.5820	0.6830	0.6123	0.3920	0.4530
Search system 1	0.3013	0.5071	0.5364	0.5007	0.2976	0.3350
Search system 2	0.2977	0.4921	0.5417	0.4949	0.2879	0.3395
Search system 3	0.2607	0.5682	0.3718	0.4206	0.3468	0.2322
Search system 4	0.2874	0.4346	0.6048	0.4849	0.2504	0.3797
Search system 5	0.2914	0.4530	0.5853	0.4907	0.2622	0.3650
MZ1	0.4305	0.7143	0.6424	0.6502	0.4737	0.4277
DeepMeSH1	0.5297	0.8173	0.7284	0.7510	0.5805	0.5128
DeepMeSH2	0.5282	0.8160	0.7269	0.7496	0.5785	0.5114
DeepMeSH3	0.5181	0.8076	0.7197	0.7416	0.5681	0.5024
DeepMeSH4	0.5108	0.8019	0.7128	0.7351	0.5604	0.4955
DeepMeSH5	0.5306	0.8045	0.7413	0.7531	0.5699	0.5221
Sequencer	0.1717	0.2349	0.2364	0.2212	0.1806	0.1811
MZ2	0.4339	0.7426	0.6250	0.6530	0.4985	0.4142
MZ3	0.3660	0.5665	0.6587	0.5898	0.3510	0.4094

+ Test batch 2, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6226	0.6793	0.6002	0.6154	0.6400	0.5444	0.5264	0.6705	0.5811	0.4625
Default MTI	0.6308	0.6379	0.6532	0.6242	0.5980	0.5868	0.5471	0.6294	0.6323	0.4698
auth1	0.6243	0.6088	0.6541	0.6131	0.6126	0.5138	0.4988	0.6121	0.6370	0.4567
auth2	0.6316	0.6160	0.6615	0.6202	0.6237	0.5014	0.4862	0.6192	0.6444	0.4644
auth3	0.6320	0.6435	0.6332	0.6199	0.6425	0.4823	0.4760	0.6475	0.6173	0.4647
auth4	0.6318	0.6207	0.6569	0.6203	0.6114	0.5169	0.5007	0.6241	0.6398	0.4644
auth5	0.6208	0.6126	0.6465	0.6075	0.6439	0.4890	0.4823	0.6075	0.6346	0.4512
iria-1	0.4839	0.4313	0.5591	0.4732	0.3829	0.4076	0.3647	0.4350	0.5451	0.3232
Search system 1	0.2802	0.2703	0.3166	0.2759	0.4337	0.4176	0.3544	0.2592	0.3049	0.1673
Search system 2	0.2667	0.2454	0.3213	0.2627	0.4167	0.4223	0.3504	0.2341	0.3097	0.1575
Search system 3	0.2169	0.2998	0.1869	0.2130	0.4082	0.2530	0.2555	0.2938	0.1719	0.1267
Search system 4	0.2516	0.2117	0.3713	0.2527	0.3014	0.4854	0.3716	0.1934	0.3600	0.1496
Search system 5	0.2635	0.2321	0.3455	0.2621	0.3207	0.4555	0.3673	0.2175	0.3342	0.1564
MZ1	0.4678	0.5572	0.4269	0.4596	0.6535	0.1271	0.1500	0.5489	0.4076	0.3131
DeepMeSH1	0.6651	0.7139	0.6446	0.6574	0.6820	0.5360	0.5356	0.7107	0.6250	0.5062
DeepMeSH2	0.6634	0.7130	0.6426	0.6557	0.6806	0.5322	0.5320	0.7093	0.6230	0.5041
DeepMeSH3	0.6519	0.6985	0.6317	0.6430	0.6657	0.5074	0.5064	0.6958	0.6132	0.4897
DeepMeSH4	0.6438	0.6897	0.6240	0.6345	0.6595	0.4906	0.4917	0.6868	0.6058	0.4804
DeepMeSH5	0.6642	0.6981	0.6565	0.6568	0.6659	0.5484	0.5393	0.6962	0.6350	0.5049
Sequencer	0.5410	0.5378	0.5704	0.5330	0.4558	0.3530	0.3326	0.5210	0.5626	0.3808
MZ2	0.4623	0.5019	0.4472	0.4530	0.6507	0.1298	0.1522	0.5000	0.4299	0.3072
MZ3	0.4054	0.3963	0.4364	0.4010	0.5080	0.2446	0.2401	0.3951	0.4163	0.2593
MZ4	0.4532	0.4590	0.4712	0.4445	0.5194	0.2154	0.2156	0.4538	0.4526	0.2987

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4996	0.7908	0.7038	0.7228	0.5515	0.4858
Default MTI	0.5104	0.7569	0.7526	0.7342	0.5280	0.5250
auth1	0.5038	0.7356	0.7535	0.7259	0.5127	0.5237
auth2	0.5072	0.7416	0.7563	0.7303	0.5175	0.5260
auth3	0.5034	0.7648	0.7279	0.7265	0.5335	0.5042
auth4	0.5065	0.7442	0.7562	0.7313	0.5159	0.5259
auth5	0.5009	0.7466	0.7385	0.7206	0.5192	0.5161
iria-1	0.4196	0.5885	0.7007	0.6228	0.3988	0.4696
Search system 1	0.3014	0.5106	0.5399	0.5024	0.2981	0.3378
Search system 2	0.2975	0.4948	0.5451	0.4961	0.2881	0.3422
Search system 3	0.2620	0.5727	0.3762	0.4260	0.3475	0.2338
Search system 4	0.2856	0.4368	0.6055	0.4843	0.2493	0.3824
Search system 5	0.2895	0.4559	0.5862	0.4907	0.2612	0.3668
MZ1	0.3609	0.7323	0.4884	0.5533	0.4846	0.3154
DeepMeSH1	0.5295	0.8150	0.7313	0.7506	0.5793	0.5151
DeepMeSH2	0.5280	0.8145	0.7290	0.7491	0.5786	0.5130
DeepMeSH3	0.5185	0.8047	0.7207	0.7400	0.5680	0.5039
DeepMeSH4	0.5123	0.7993	0.7155	0.7343	0.5617	0.4983
DeepMeSH5	0.5299	0.8031	0.7425	0.7517	0.5694	0.5233
Sequencer	0.4650	0.7001	0.7037	0.6800	0.4801	0.4808
MZ2	0.3678	0.6880	0.5169	0.5630	0.4483	0.3389
MZ3	0.3674	0.5667	0.6561	0.5899	0.3532	0.4094
MZ4	0.3677	0.6462	0.5537	0.5689	0.4118	0.3627

+ Test batch 2, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6212	0.6862	0.5870	0.6139	0.6443	0.5372	0.5224	0.6786	0.5727	0.4613
Default MTI	0.6312	0.6472	0.6391	0.6253	0.6083	0.5768	0.5437	0.6408	0.6218	0.4717
auth1	0.6303	0.6263	0.6440	0.6210	0.6288	0.5128	0.5027	0.6310	0.6296	0.4646
auth2	0.6375	0.6335	0.6512	0.6281	0.6371	0.5014	0.4905	0.6382	0.6368	0.4722
auth3	0.6372	0.6631	0.6209	0.6264	0.6576	0.4815	0.4785	0.6683	0.6088	0.4715
auth4	0.6375	0.6389	0.6458	0.6279	0.6254	0.5150	0.5042	0.6435	0.6316	0.4721
auth5	0.6412	0.6336	0.6590	0.6320	0.6247	0.5345	0.5194	0.6386	0.6438	0.4763
iria-1	0.4844	0.4379	0.5498	0.4758	0.3888	0.3999	0.3649	0.4421	0.5356	0.3249
MZ1	0.4573	0.5559	0.4127	0.4502	0.6603	0.1371	0.1594	0.5442	0.3944	0.3042
DeepMeSH1	0.6649	0.7278	0.6299	0.6583	0.6994	0.5282	0.5319	0.7249	0.6140	0.5071
DeepMeSH2	0.6632	0.7273	0.6275	0.6564	0.6944	0.5223	0.5269	0.7241	0.6117	0.5050
DeepMeSH3	0.6524	0.7136	0.6183	0.6452	0.6770	0.4996	0.5034	0.7109	0.6028	0.4917
DeepMeSH4	0.6457	0.7063	0.6105	0.6376	0.6713	0.4827	0.4878	0.7040	0.5963	0.4835
DeepMeSH5	0.6645	0.7128	0.6407	0.6582	0.6796	0.5404	0.5372	0.7124	0.6226	0.5065
Sequencer	0.5387	0.5473	0.5539	0.5323	0.4668	0.3482	0.3323	0.5293	0.5484	0.3793
MZ2	0.4514	0.4999	0.4311	0.4439	0.6597	0.1397	0.1616	0.4959	0.4142	0.2985
MZ3	0.4023	0.4016	0.4219	0.3984	0.5160	0.2481	0.2427	0.4005	0.4041	0.2565
MZ4	0.4418	0.4554	0.4543	0.4350	0.5336	0.2225	0.2251	0.4478	0.4360	0.2899
SanMatrix1	0.0968	0.6713	0.0565	0.1030	0.6713	0.0001	0.0001	0.6713	0.0521	0.0565
Sequencer _T	0.5645	0.5907	0.5652	0.5594	0.5150	0.3633	0.3535	0.5713	0.5579	0.4056
SanMatrix2	0.0076	0.0063	0.0088	0.0071	0.0018	0.0013	0.0001	0.0063	0.0097	0.0037

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4975	0.8004	0.6893	0.7207	0.5572	0.4745
Default MTI	0.5110	0.7661	0.7371	0.7323	0.5368	0.5145
auth1	0.5057	0.7492	0.7409	0.7291	0.5236	0.5124
auth2	0.5107	0.7546	0.7425	0.7326	0.5295	0.5163
auth3	0.5052	0.7807	0.7124	0.7284	0.5461	0.4923
auth4	0.5095	0.7575	0.7420	0.7334	0.5292	0.5145
auth5	0.5144	0.7537	0.7521	0.7373	0.5307	0.5224
iria-1	0.4183	0.5937	0.6864	0.6207	0.4026	0.4589
MZ1	0.3546	0.7284	0.4819	0.5464	0.4806	0.3077
DeepMeSH1	0.5281	0.8257	0.7143	0.7474	0.5887	0.5020
DeepMeSH2	0.5261	0.8258	0.7117	0.7458	0.5874	0.5001
DeepMeSH3	0.5176	0.8163	0.7051	0.7379	0.5772	0.4923
DeepMeSH4	0.5115	0.8116	0.6984	0.7321	0.5723	0.4862
DeepMeSH5	0.5291	0.8152	0.7254	0.7498	0.5799	0.5103
Sequencer	0.4627	0.7098	0.6876	0.6783	0.4867	0.4676
MZ2	0.3628	0.6882	0.5088	0.5572	0.4478	0.3305
MZ3	0.3643	0.5715	0.6416	0.5860	0.3555	0.3987
MZ4	0.3621	0.6448	0.5446	0.5622	0.4099	0.3534
SanMatrix1	0.0730	0.6896	0.0634	0.1148	0.2789	0.0428
Sequencer _T	0.4798	0.7459	0.6921	0.6978	0.5165	0.4740
SanMatrix2	0.1232	0.1078	0.2119	0.1381	0.0942	0.1901

+ Test batch 2, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6158	0.6830	0.5923	0.6129	0.6361	0.5384	0.5209	0.6706	0.5693	0.4614
Default MTI	0.6291	0.6441	0.6478	0.6255	0.6003	0.5757	0.5400	0.6360	0.6225	0.4726
auth1	0.6192	0.6136	0.6414	0.6098	0.6105	0.5136	0.4989	0.6178	0.6205	0.4533
auth2	0.6273	0.6216	0.6490	0.6174	0.6232	0.5010	0.4863	0.6259	0.6287	0.4615
auth3	0.6258	0.6489	0.6187	0.6154	0.6436	0.4811	0.4752	0.6536	0.6003	0.4600
auth4	0.6277	0.6271	0.6448	0.6180	0.6098	0.5166	0.5013	0.6312	0.6242	0.4622
auth5	0.6304	0.6211	0.6561	0.6208	0.6093	0.5326	0.5129	0.6259	0.6350	0.4647
iria-1	0.4959	0.4470	0.5675	0.4857	0.4243	0.4105	0.3751	0.4519	0.5494	0.3337
Search system 1	0.2803	0.2747	0.3169	0.2787	0.4238	0.4184	0.3529	0.2614	0.3023	0.1692
Search system 2	0.2665	0.2496	0.3213	0.2654	0.4075	0.4227	0.3488	0.2356	0.3068	0.1594
Search system 3	0.2175	0.3033	0.1885	0.2149	0.4071	0.2541	0.2559	0.2966	0.1717	0.1281
Search system 4	0.2529	0.2175	0.3701	0.2568	0.2992	0.4852	0.3700	0.1960	0.3564	0.1525
Search system 5	0.2643	0.2375	0.3443	0.2653	0.3181	0.4568	0.3654	0.2201	0.3308	0.1587
DeepMeSH1	0.6613	0.7230	0.6342	0.6571	0.6854	0.5335	0.5347	0.7187	0.6123	0.5061
DeepMeSH2	0.6596	0.7210	0.6319	0.6549	0.6808	0.5277	0.5292	0.7171	0.6106	0.5038
DeepMeSH3	0.6484	0.7080	0.6211	0.6426	0.6704	0.5032	0.5051	0.7049	0.6003	0.4895
DeepMeSH4	0.6417	0.6988	0.6144	0.6351	0.6595	0.4876	0.4900	0.6970	0.5945	0.4815
DeepMeSH5	0.6599	0.7057	0.6455	0.6557	0.6672	0.5449	0.5381	0.7042	0.6208	0.5043
SanMatrix1	0.1228	0.0769	0.2859	0.1186	0.0173	0.0048	0.0006	0.0769	0.3039	0.0654

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5020	0.7973	0.6937	0.7214	0.5563	0.4841
Default MTI	0.5142	0.7647	0.7469	0.7365	0.5349	0.5232
auth1	0.5016	0.7440	0.7437	0.7264	0.5168	0.5136
auth2	0.5055	0.7502	0.7477	0.7317	0.5219	0.5165
auth3	0.5012	0.7742	0.7170	0.7265	0.5380	0.4941
auth4	0.5060	0.7537	0.7470	0.7329	0.5220	0.5174
auth5	0.5103	0.7497	0.7566	0.7361	0.5240	0.5237
iria-1	0.4255	0.6094	0.7031	0.6364	0.4103	0.4679
Search system 1	0.3040	0.5154	0.5416	0.5054	0.3014	0.3388
Search system 2	0.3000	0.4997	0.5470	0.4994	0.2912	0.3432
Search system 3	0.2652	0.5794	0.3785	0.4294	0.3506	0.2374
Search system 4	0.2900	0.4445	0.6053	0.4889	0.2553	0.3829
Search system 5	0.2937	0.4639	0.5865	0.4955	0.2669	0.3677
DeepMeSH1	0.5314	0.8265	0.7248	0.7544	0.5867	0.5095
DeepMeSH2	0.5296	0.8257	0.7231	0.7528	0.5843	0.5084
DeepMeSH3	0.5190	0.8173	0.7155	0.7445	0.5735	0.4979
DeepMeSH4	0.5146	0.8122	0.7104	0.7397	0.5689	0.4935
DeepMeSH5	0.5318	0.8134	0.7359	0.7550	0.5758	0.5185
SanMatrix1	0.1830	0.1897	0.4538	0.2600	0.1303	0.3332

+ Test batch 2, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6195	0.6765	0.5911	0.6101	0.6314	0.5487	0.5270	0.6697	0.5763	0.4573
Default MTI	0.6281	0.6344	0.6475	0.6206	0.5929	0.5938	0.5493	0.6282	0.6281	0.4678
auth1	0.6237	0.6119	0.6459	0.6118	0.6073	0.5200	0.5020	0.6178	0.6297	0.4557
auth2	0.6314	0.6194	0.6536	0.6193	0.6173	0.5103	0.4914	0.6255	0.6375	0.4638
auth3	0.6309	0.6471	0.6235	0.6180	0.6381	0.4914	0.4814	0.6539	0.6095	0.4630
auth4	0.6319	0.6255	0.6486	0.6199	0.6079	0.5231	0.5043	0.6314	0.6323	0.4644
auth5	0.6341	0.6183	0.6596	0.6220	0.6060	0.5413	0.5168	0.6254	0.6430	0.4667
iria-1	0.5010	0.4653	0.5496	0.4898	0.4371	0.3985	0.3689	0.4719	0.5341	0.3375
DeepMeSH1	0.6620	0.7148	0.6342	0.6533	0.6780	0.5404	0.5366	0.7134	0.6174	0.5022
DeepMeSH2	0.6609	0.7139	0.6325	0.6517	0.6791	0.5375	0.5335	0.7132	0.6157	0.5005
DeepMeSH3	0.6491	0.6997	0.6212	0.6391	0.6602	0.5077	0.5036	0.6994	0.6056	0.4860
DeepMeSH4	0.6429	0.6946	0.6133	0.6321	0.6529	0.4901	0.4887	0.6945	0.5985	0.4781
DeepMeSH5	0.6622	0.7008	0.6466	0.6537	0.6611	0.5536	0.5421	0.7019	0.6268	0.5021

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4947	0.7896	0.6936	0.7171	0.5464	0.4789
Default MTI	0.5088	0.7530	0.7467	0.7294	0.5258	0.5235
auth1	0.5013	0.7363	0.7504	0.7254	0.5124	0.5169
auth2	0.5060	0.7430	0.7530	0.7301	0.5182	0.5208
auth3	0.5009	0.7667	0.7231	0.7261	0.5331	0.4977
auth4	0.5055	0.7467	0.7525	0.7317	0.5177	0.5202
auth5	0.5112	0.7426	0.7616	0.7346	0.5200	0.5287
iria-1	0.4251	0.6207	0.6848	0.6338	0.4212	0.4537
DeepMeSH1	0.5268	0.8160	0.7232	0.7472	0.5767	0.5102
DeepMeSH2	0.5252	0.8143	0.7216	0.7454	0.5751	0.5088
DeepMeSH3	0.5156	0.8060	0.7162	0.7385	0.5647	0.4996
DeepMeSH4	0.5107	0.8030	0.7097	0.7333	0.5618	0.4934
DeepMeSH5	0.5279	0.8052	0.7359	0.7494	0.5688	0.5192

+ Test batch 3, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.5483	0.6087	0.5233	0.5407	0.5721	0.5054	0.4765	0.5959	0.5077	0.3889
Default MTI	0.6307	0.6482	0.6332	0.6215	0.6056	0.5803	0.5444	0.6428	0.6191	0.4685
auth1	0.6236	0.6111	0.6430	0.6115	0.6186	0.5078	0.4968	0.6162	0.6312	0.4557
auth2	0.6311	0.6185	0.6506	0.6188	0.6300	0.4945	0.4837	0.6236	0.6388	0.4636
auth3	0.6316	0.6492	0.6196	0.6179	0.6502	0.4747	0.4720	0.6550	0.6098	0.4635
auth4	0.6311	0.6238	0.6447	0.6184	0.6164	0.5110	0.4996	0.6289	0.6333	0.4633
auth5	0.6356	0.6197	0.6598	0.6243	0.6152	0.5303	0.5133	0.6249	0.6468	0.4691
Search system 1	0.2752	0.2714	0.3037	0.2714	0.4411	0.4133	0.3539	0.2575	0.2955	0.1639
Search system 2	0.2623	0.2473	0.3079	0.2590	0.4256	0.4178	0.3502	0.2332	0.2996	0.1548
Search system 3	0.2096	0.2957	0.1764	0.2047	0.4158	0.2482	0.2531	0.2880	0.1647	0.1210
Search system 4	0.2483	0.2134	0.3567	0.2502	0.3125	0.4835	0.3741	0.1923	0.3503	0.1480
Search system 5	0.2600	0.2334	0.3324	0.2590	0.3297	0.4540	0.3680	0.2162	0.3259	0.1543
DeepMeSH1	0.6634	0.7253	0.6260	0.6533	0.6831	0.5273	0.5293	0.7209	0.6144	0.5023
DeepMeSH2	0.6614	0.7212	0.6245	0.6508	0.6815	0.5240	0.5256	0.7179	0.6131	0.4996
DeepMeSH3	0.6512	0.7059	0.6173	0.6401	0.6642	0.4971	0.4992	0.7025	0.6069	0.4876
DeepMeSH4	0.6437	0.6970	0.6102	0.6321	0.6554	0.4796	0.4821	0.6946	0.5998	0.4788
DeepMeSH5	0.6621	0.7053	0.6384	0.6524	0.6632	0.5418	0.5347	0.7048	0.6243	0.5011

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4493	0.7411	0.6634	0.6773	0.4951	0.4400
Default MTI	0.5116	0.7650	0.7378	0.7317	0.5384	0.5160
auth1	0.5039	0.7384	0.7473	0.7258	0.5158	0.5185
auth2	0.5084	0.7447	0.7492	0.7301	0.5213	0.5222
auth3	0.5040	0.7696	0.7187	0.7255	0.5393	0.4983
auth4	0.5067	0.7481	0.7489	0.7313	0.5191	0.5208
auth5	0.5133	0.7450	0.7599	0.7360	0.5230	0.5298
Search system 1	0.2988	0.5094	0.5290	0.4966	0.2974	0.3324
Search system 2	0.2949	0.4948	0.5345	0.4911	0.2876	0.3365
Search system 3	0.2606	0.5730	0.3703	0.4217	0.3477	0.2315
Search system 4	0.2849	0.4367	0.5957	0.4806	0.2508	0.3769
Search system 5	0.2885	0.4554	0.5768	0.4867	0.2624	0.3613
DeepMeSH1	0.5278	0.8248	0.7172	0.7478	0.5874	0.5050
DeepMeSH2	0.5261	0.8223	0.7159	0.7460	0.5852	0.5035
DeepMeSH3	0.5187	0.8128	0.7106	0.7390	0.5751	0.4980
DeepMeSH4	0.5121	0.8065	0.7050	0.7328	0.5689	0.4913
DeepMeSH5	0.5297	0.8104	0.7309	0.7501	0.5768	0.5156

+ Test batch 3, week 2

BioASQ Participants Area

Test Results for Task 5a

+ Dry Run 1

Annotated articles:11372/13000

Flat Measures

Hierarchical Measures

+ Dry Run 2

Annotated articles:6179/6326

Flat Measures

Hierarchical Measures

+ Test batch 1, week 1

Annotated articles:6661/6880

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:6599/7457

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:9656/10319

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:4697/7523

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:6659/7940

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:7080/7431

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:6357/6746

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:5479/5944

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:6526/6986

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:5492/6055

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:7356/9233

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:5304/7816

Flat Measures