BioASQ Participants Area

Test Results for Task 3a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Test batch 1, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2484	0.2738	0.2846	0.2416	0.3717	0.3854	0.3195	0.2297	0.2706	0.1433
MTI First Line Index	0.5668	0.6415	0.5316	0.5598	0.6069	0.4837	0.4720	0.6367	0.5108	0.4056
Default MTI	0.5748	0.5921	0.5833	0.5673	0.5659	0.5265	0.4925	0.5879	0.5623	0.4114
it is a test submit	0.3091	0.2749	0.3769	0.3051	0.2942	0.3093	0.2578	0.2749	0.3530	0.1870
USI 10 neighbours	0.5147	0.5508	0.5082	0.5043	0.5073	0.3734	0.3708	0.5379	0.4935	0.3514
USI 20 neighbours	0.4944	0.5305	0.4990	0.4902	0.4878	0.3640	0.3572	0.5135	0.4766	0.3374
Abstract framework	0.5187	0.5600	0.5152	0.5080	0.5048	0.3763	0.3722	0.5368	0.5019	0.3555
auth1	0.5896	0.5744	0.6184	0.5788	0.5930	0.4788	0.4672	0.5793	0.6004	0.4220
auth2	0.5979	0.6373	0.5747	0.5860	0.6124	0.4657	0.4621	0.6436	0.5582	0.4304
auth4	0.5849	0.6554	0.5395	0.5710	0.7061	0.3677	0.3980	0.6585	0.5262	0.4168
MeSHLabeler	0.6266	0.6882	0.5978	0.6194	0.6356	0.5064	0.5081	0.6889	0.5746	0.4648
MeSH Now BF	0.5516	0.5671	0.5622	0.5439	0.5446	0.4767	0.4494	0.5643	0.5394	0.3885
MeSH Now HR	0.1879	0.1081	0.7510	0.1856	0.0790	0.6198	0.1992	0.1077	0.7393	0.1041
MeSHLabeler-1	0.6268	0.6879	0.5986	0.6197	0.6563	0.4934	0.4991	0.6887	0.5750	0.4650
MeSHLabeler-2	0.6164	0.7354	0.5522	0.6089	0.6826	0.4648	0.4845	0.7337	0.5314	0.4552
MeSHLabeler-3	0.6173	0.7327	0.5548	0.6094	0.6964	0.4535	0.4743	0.7308	0.5344	0.4558
MeSHLabeler-4	0.6268	0.6904	0.5966	0.6194	0.6587	0.4916	0.4978	0.6906	0.5737	0.4648

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.2867	0.5323	0.5339	0.4839	0.3131	0.3037
MTI First Line Index	0.4756	0.7885	0.6552	0.6899	0.5478	0.4486
Default MTI	0.4884	0.7485	0.7138	0.7069	0.5198	0.4916
it is a test submit	0.3310	0.5234	0.6201	0.5466	0.3215	0.3661
USI 10 neighbours	0.4449	0.7178	0.6529	0.6549	0.4889	0.4403
USI 20 neighbours	0.4480	0.6999	0.6721	0.6583	0.4889	0.4451
Abstract framework	0.4437	0.7251	0.6486	0.6520	0.4902	0.4408
auth1	0.4958	0.7269	0.7418	0.7138	0.5032	0.5177
auth2	0.4936	0.7722	0.6906	0.7067	0.5370	0.4841
auth4	0.4774	0.8008	0.6415	0.6864	0.5532	0.4472
MeSHLabeler	0.5147	0.8098	0.7013	0.7288	0.5745	0.4943
MeSH Now BF	0.4597	0.7023	0.6998	0.6762	0.4775	0.4770
MeSH Now HR	0.2200	0.2255	0.9130	0.3525	0.1372	0.6139
MeSHLabeler-1	0.5145	0.8099	0.7024	0.7294	0.5736	0.4947
MeSHLabeler-2	0.4988	0.8434	0.6527	0.7106	0.5952	0.4558
MeSHLabeler-3	0.4998	0.8422	0.6552	0.7117	0.5942	0.4580
MeSHLabeler-4	0.5139	0.8119	0.7001	0.7285	0.5748	0.4929

+ Test batch 1, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.1083	0.4188	0.0951	0.1125	0.1403	0.0771	0.0754	0.1451	0.0865	0.0637
MTI First Line Index	0.5552	0.6252	0.5174	0.5460	0.5867	0.4690	0.4545	0.6230	0.5008	0.3909
Default MTI	0.5618	0.5826	0.5620	0.5532	0.5421	0.5152	0.4805	0.5786	0.5460	0.3967
TextCategorisation1	0.5604	0.6500	0.5099	0.5484	0.6202	0.4373	0.4343	0.6461	0.4948	0.3943
TextCategorisation2	0.5608	0.6464	0.5126	0.5489	0.6160	0.4406	0.4364	0.6427	0.4974	0.3946
TextCategorisation3	0.5608	0.6476	0.5124	0.5488	0.6162	0.4405	0.4363	0.6429	0.4973	0.3946
TextCategorisation4	0.5608	0.6427	0.5134	0.5491	0.6148	0.4410	0.4366	0.6418	0.4979	0.3948
TextCategorisation5	0.5604	0.6511	0.5096	0.5481	0.6204	0.4372	0.4341	0.6463	0.4946	0.3942
spoon-spoon	0.4628	0.4871	0.4613	0.4541	0.4912	0.2962	0.2920	0.4825	0.4446	0.3071
fork-fork	0.4183	0.6442	0.3168	0.4006	0.6759	0.1602	0.1792	0.6499	0.3085	0.2664
it is a test submit	0.3021	0.2697	0.3650	0.2975	0.2771	0.3325	0.2768	0.2697	0.3434	0.1813
qaiiit system 1	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
USI 10 neighbours	0.5114	0.5471	0.5041	0.5020	0.5065	0.3877	0.3797	0.5368	0.4883	0.3483
USI 20 neighbours	0.4846	0.5139	0.4963	0.4819	0.4642	0.3747	0.3589	0.4973	0.4725	0.3289
Abstract framework	0.5159	0.5577	0.5090	0.5054	0.5051	0.3900	0.3819	0.5384	0.4951	0.3522
auth1	0.5866	0.5709	0.6154	0.5761	0.5793	0.4993	0.4786	0.5770	0.5966	0.4185
auth2	0.5931	0.6189	0.5803	0.5811	0.6026	0.4805	0.4696	0.6260	0.5635	0.4244
auth3	0.5954	0.6121	0.5906	0.5832	0.6559	0.4536	0.4550	0.6181	0.5743	0.4266
auth4	0.5806	0.6538	0.5336	0.5674	0.6864	0.3832	0.3993	0.6581	0.5195	0.4128
auth5	0.5476	0.4620	0.6852	0.5356	0.5004	0.5659	0.4919	0.4639	0.6682	0.3777
MeSHLabeler	0.6190	0.6849	0.5866	0.6118	0.6555	0.4837	0.4843	0.6868	0.5633	0.4561
MeSH Now BF	0.4361	0.4520	0.4541	0.4329	0.4302	0.3602	0.3248	0.4391	0.4332	0.2915
MeSH Now HR	0.1800	0.1032	0.7212	0.1776	0.0851	0.5921	0.2272	0.1031	0.7106	0.0991
MeSHLabeler-1	0.6193	0.6851	0.5888	0.6131	0.6445	0.4967	0.4974	0.6856	0.5646	0.4574
MeSHLabeler-2	0.6189	0.6860	0.5856	0.6117	0.6568	0.4846	0.4860	0.6875	0.5627	0.4563
MeSHLabeler-3	0.6190	0.6853	0.5876	0.6125	0.6414	0.4964	0.4971	0.6864	0.5637	0.4569
MeSHLabeler-4	0.6194	0.6855	0.5887	0.6133	0.6452	0.4967	0.4976	0.6860	0.5645	0.4575
USI baseline	0.5624	0.5804	0.5640	0.5542	0.5701	0.4640	0.4407	0.5847	0.5417	0.3984
iria-1	0.4628	0.5122	0.4487	0.4587	0.4327	0.2903	0.2912	0.5122	0.4221	0.3113

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1541	0.5787	0.2174	0.2262	0.2850	0.1398
MTI First Line Index	0.4672	0.7726	0.6445	0.6777	0.5377	0.4391
Default MTI	0.4788	0.7369	0.6963	0.6928	0.5146	0.4754
TextCategorisation1	0.4625	0.7911	0.6366	0.6778	0.5439	0.4314
TextCategorisation2	0.4630	0.7880	0.6400	0.6787	0.5414	0.4337
TextCategorisation3	0.4626	0.7885	0.6396	0.6782	0.5409	0.4334
TextCategorisation4	0.4641	0.7859	0.6424	0.6806	0.5407	0.4351
TextCategorisation5	0.4621	0.7913	0.6359	0.6770	0.5434	0.4311
spoon-spoon	0.3963	0.6653	0.5758	0.5896	0.4337	0.3952
fork-fork	0.3283	0.7944	0.3563	0.4576	0.5427	0.2563
it is a test submit	0.3287	0.5126	0.6081	0.5341	0.3218	0.3613
qaiiit system 1	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
USI 10 neighbours	0.4430	0.7112	0.6456	0.6493	0.4864	0.4375
USI 20 neighbours	0.4450	0.6826	0.6741	0.6519	0.4820	0.4457
Abstract framework	0.4416	0.7187	0.6387	0.6455	0.4882	0.4364
auth1	0.4925	0.7183	0.7394	0.7083	0.4993	0.5138
auth2	0.4912	0.7569	0.6933	0.7019	0.5270	0.4867
auth3	0.4930	0.7652	0.6974	0.7077	0.5275	0.4903
auth4	0.4754	0.7967	0.6338	0.6801	0.5534	0.4433
auth5	0.4626	0.6181	0.7986	0.6765	0.4114	0.5638
MeSHLabeler	0.5058	0.8024	0.6882	0.7173	0.5697	0.4825
MeSH Now BF	0.3817	0.5990	0.5991	0.5717	0.3992	0.3987
MeSH Now HR	0.2175	0.2160	0.8964	0.3394	0.1352	0.6112
MeSHLabeler-1	0.5069	0.8023	0.6896	0.7180	0.5691	0.4843
MeSHLabeler-2	0.5058	0.8040	0.6875	0.7175	0.5698	0.4823
MeSHLabeler-3	0.5060	0.8025	0.6890	0.7179	0.5682	0.4835
MeSHLabeler-4	0.5070	0.8025	0.6894	0.7180	0.5694	0.4842
USI baseline	0.4677	0.7095	0.6917	0.6773	0.4895	0.4765
iria-1	0.4090	0.6911	0.5798	0.6027	0.4709	0.3891

+ Test batch 1, week 3

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.1547	0.3777	0.1431	0.1495	0.2418	0.1588	0.1464	0.1920	0.1295	0.0860
MTI First Line Index	0.5688	0.6368	0.5312	0.5573	0.5934	0.4707	0.4583	0.6368	0.5140	0.4024
Default MTI	0.5758	0.5961	0.5764	0.5643	0.5570	0.5142	0.4836	0.5936	0.5590	0.4078
TextCategorisation1	0.5691	0.6534	0.5133	0.5513	0.6169	0.4224	0.4212	0.6568	0.5020	0.3976
TextCategorisation2	0.5689	0.6497	0.5152	0.5511	0.6121	0.4245	0.4224	0.6530	0.5040	0.3973
TextCategorisation3	0.5690	0.6521	0.5149	0.5510	0.6126	0.4243	0.4223	0.6533	0.5039	0.3972
TextCategorisation4	0.5690	0.6471	0.5161	0.5518	0.6110	0.4252	0.4229	0.6522	0.5046	0.3978
TextCategorisation5	0.5691	0.6559	0.5130	0.5511	0.6173	0.4222	0.4210	0.6571	0.5018	0.3975
spoon-spoon	0.4681	0.4907	0.4673	0.4566	0.4903	0.2952	0.2917	0.4863	0.4512	0.3100
fork-fork	0.4233	0.6470	0.3205	0.4048	0.6666	0.1536	0.1714	0.6517	0.3134	0.2698
it is a test submit	0.3069	0.2731	0.3756	0.3016	0.2661	0.3337	0.2754	0.2731	0.3503	0.1848
USI 10 neighbours	0.5239	0.5618	0.5126	0.5115	0.5089	0.3810	0.3735	0.5526	0.4979	0.3591
USI 20 neighbours	0.5731	0.6304	0.5471	0.5643	0.5910	0.4688	0.4585	0.6338	0.5229	0.4082
Abstract framework	0.5279	0.5688	0.5194	0.5150	0.5041	0.3821	0.3739	0.5513	0.5065	0.3628
auth1	0.5944	0.5774	0.6200	0.5805	0.5736	0.4870	0.4695	0.5858	0.6034	0.4234
auth2	0.5936	0.5944	0.5991	0.5787	0.5907	0.4735	0.4625	0.6044	0.5832	0.4218
auth3	0.6046	0.5538	0.6714	0.5897	0.6030	0.5274	0.5023	0.5606	0.6562	0.4316
auth4	0.5856	0.5915	0.5834	0.5695	0.6571	0.4109	0.4158	0.6006	0.5712	0.4131
auth5	0.5969	0.5733	0.6277	0.5817	0.6332	0.4694	0.4600	0.5811	0.6137	0.4243
MeSHLabeler	0.6204	0.6992	0.5780	0.6094	0.6673	0.4746	0.4804	0.7012	0.5562	0.4543
MeSH Now BF	0.5998	0.6203	0.5964	0.5868	0.5962	0.4612	0.4547	0.6197	0.5812	0.4305
MeSH Now HR	0.2154	0.1237	0.8556	0.2119	0.0923	0.7567	0.3089	0.1233	0.8516	0.1209
MeSHLabeler-1	0.6217	0.6963	0.5810	0.6110	0.6634	0.4791	0.4836	0.6996	0.5595	0.4557
MeSHLabeler-2	0.6227	0.6156	0.6445	0.6107	0.5877	0.5383	0.5139	0.6218	0.6236	0.4538
MeSHLabeler-3	0.6259	0.6869	0.5939	0.6145	0.6514	0.4941	0.4936	0.6843	0.5767	0.4590
MeSHLabeler-4	0.6276	0.6846	0.5989	0.6169	0.6469	0.4998	0.4976	0.6823	0.5811	0.4617
USI baseline	0.5946	0.6596	0.5612	0.5853	0.6317	0.4189	0.4255	0.6671	0.5363	0.4292
iria-1	0.4672	0.5151	0.4550	0.4613	0.4264	0.2860	0.2850	0.5151	0.4275	0.3143

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1957	0.5872	0.2979	0.2995	0.3249	0.1821
MTI First Line Index	0.4726	0.7781	0.6495	0.6811	0.5446	0.4463
Default MTI	0.4834	0.7465	0.7003	0.6971	0.5212	0.4826
TextCategorisation1	0.4623	0.7950	0.6348	0.6777	0.5444	0.4314
TextCategorisation2	0.4630	0.7929	0.6380	0.6789	0.5430	0.4334
TextCategorisation3	0.4623	0.7946	0.6372	0.6781	0.5422	0.4328
TextCategorisation4	0.4638	0.7907	0.6394	0.6797	0.5428	0.4344
TextCategorisation5	0.4616	0.7967	0.6339	0.6767	0.5436	0.4309
spoon-spoon	0.3957	0.6705	0.5800	0.5935	0.4341	0.3966
fork-fork	0.3324	0.7988	0.3618	0.4644	0.5416	0.2616
it is a test submit	0.3293	0.5133	0.6148	0.5369	0.3207	0.3655
USI 10 neighbours	0.4488	0.7244	0.6467	0.6539	0.4988	0.4404
USI 20 neighbours	0.4766	0.7703	0.6654	0.6882	0.5383	0.4569
Abstract framework	0.4475	0.7289	0.6443	0.6516	0.4993	0.4411
auth1	0.4948	0.7214	0.7381	0.7087	0.5045	0.5151
auth2	0.4871	0.7458	0.7079	0.7054	0.5096	0.4957
auth3	0.5050	0.7191	0.7731	0.7249	0.4919	0.5511
auth4	0.4823	0.7575	0.6891	0.6999	0.5149	0.4819
auth5	0.4962	0.7388	0.7338	0.7155	0.5048	0.5178
MeSHLabeler	0.5016	0.8127	0.6756	0.7112	0.5759	0.4740
MeSH Now BF	0.4995	0.7556	0.7116	0.7081	0.5323	0.5028
MeSH Now HR	0.2505	0.2397	0.9622	0.3742	0.1577	0.6705
MeSHLabeler-1	0.5025	0.8101	0.6799	0.7134	0.5724	0.4768
MeSHLabeler-2	0.5110	0.7506	0.7468	0.7269	0.5278	0.5253
MeSHLabeler-3	0.5078	0.8046	0.6911	0.7179	0.5706	0.4868
MeSHLabeler-4	0.5103	0.8036	0.6960	0.7209	0.5701	0.4907
USI baseline	0.4924	0.7856	0.6747	0.7011	0.5563	0.4710
iria-1	0.4083	0.6905	0.5774	0.5996	0.4690	0.3913

+ Test batch 1, week 4

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.0937	0.5985	0.0747	0.1098	0.0709	0.0240	0.0252	0.1638	0.0656	0.0609
MTI First Line Index	0.5712	0.6309	0.5378	0.5616	0.5630	0.4788	0.4572	0.6264	0.5249	0.4054
Default MTI	0.5743	0.5807	0.5891	0.5672	0.5203	0.5309	0.4897	0.5747	0.5739	0.4098
TextCategorisation1	0.5865	0.7517	0.4921	0.5673	0.7125	0.3719	0.3828	0.7329	0.4889	0.4139
TextCategorisation2	0.5956	0.7213	0.5216	0.5783	0.6751	0.4035	0.4084	0.7015	0.5174	0.4238
TextCategorisation3	0.5976	0.7083	0.5328	0.5808	0.6622	0.4152	0.4178	0.6884	0.5280	0.4257
TextCategorisation4	0.5990	0.6913	0.5454	0.5830	0.6350	0.4338	0.4322	0.6719	0.5403	0.4275
TextCategorisation5	0.6009	0.6727	0.5616	0.5862	0.6215	0.4523	0.4464	0.6538	0.5559	0.4301
spoon-spoon	0.4767	0.4954	0.4712	0.4652	0.4851	0.3096	0.3003	0.4911	0.4631	0.3167
fork-fork	0.4406	0.6603	0.3305	0.4189	0.6663	0.1703	0.1842	0.6622	0.3302	0.2802
it is a test submit	0.3065	0.2730	0.3692	0.3029	0.2612	0.3493	0.2878	0.2730	0.3493	0.1848
USI 10 neighbours	0.5149	0.5455	0.5116	0.5047	0.4882	0.3784	0.3661	0.5317	0.4991	0.3507
USI 20 neighbours	0.5762	0.6257	0.5536	0.5712	0.5627	0.4737	0.4561	0.6263	0.5335	0.4143
Abstract framework	0.5212	0.5549	0.5216	0.5107	0.4860	0.3834	0.3703	0.5323	0.5105	0.3565
auth1	0.5974	0.5989	0.6039	0.5857	0.5858	0.4796	0.4632	0.6024	0.5924	0.4288
auth2	0.6046	0.6534	0.5673	0.5906	0.6136	0.4647	0.4557	0.6596	0.5580	0.4349
auth3	0.6094	0.6101	0.6174	0.5978	0.6170	0.4681	0.4544	0.6121	0.6068	0.4405
auth4	0.5981	0.6361	0.5684	0.5837	0.6384	0.4187	0.4172	0.6410	0.5606	0.4281
auth5	0.6094	0.6101	0.6174	0.5978	0.6170	0.4681	0.4544	0.6121	0.6068	0.4405
MeSHLabeler	0.6266	0.6046	0.6629	0.6174	0.5645	0.5590	0.5255	0.6066	0.6479	0.4600
MeSH Now BF	0.6012	0.6150	0.6061	0.5911	0.5745	0.4769	0.4630	0.6073	0.5952	0.4340
MeSH Now HR	0.2172	0.1247	0.8602	0.2147	0.0965	0.7649	0.3529	0.1244	0.8554	0.1221
MeSHLabeler-1	0.6251	0.6030	0.6618	0.6157	0.5621	0.5572	0.5238	0.6050	0.6466	0.4583
MeSHLabeler-2	0.6233	0.6688	0.6060	0.6158	0.6028	0.4990	0.4878	0.6586	0.5916	0.4599
MeSHLabeler-3	0.6342	0.6678	0.6187	0.6243	0.6219	0.5123	0.4996	0.6663	0.6051	0.4684
MeSHLabeler-4	0.6327	0.6675	0.6164	0.6224	0.6210	0.5096	0.4974	0.6660	0.6025	0.4666
USI baseline	0.5992	0.6648	0.5618	0.5925	0.6253	0.4308	0.4332	0.6688	0.5427	0.4355
iria-1	0.4652	0.5760	0.4096	0.4623	0.4838	0.2509	0.2578	0.5761	0.3901	0.3143

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1142	0.6801	0.1316	0.1546	0.2770	0.0946
MTI First Line Index	0.4772	0.7807	0.6647	0.6932	0.5444	0.4515
Default MTI	0.4881	0.7370	0.7246	0.7087	0.5126	0.4940
TextCategorisation1	0.4592	0.8612	0.5733	0.6555	0.5984	0.4007
TextCategorisation2	0.4730	0.8403	0.6129	0.6779	0.5852	0.4275
TextCategorisation3	0.4779	0.8320	0.6272	0.6841	0.5809	0.4376
TextCategorisation4	0.4829	0.8194	0.6439	0.6916	0.5729	0.4495
TextCategorisation5	0.4868	0.8069	0.6620	0.6985	0.5620	0.4615
spoon-spoon	0.4009	0.6772	0.5870	0.6031	0.4375	0.3991
fork-fork	0.3394	0.8049	0.3744	0.4777	0.5463	0.2667
it is a test submit	0.3289	0.5305	0.6166	0.5511	0.3231	0.3581
USI 10 neighbours	0.4462	0.7141	0.6633	0.6603	0.4860	0.4439
USI 20 neighbours	0.4831	0.7692	0.6831	0.7017	0.5387	0.4622
Abstract framework	0.4459	0.7207	0.6613	0.6593	0.4866	0.4465
auth1	0.4991	0.7481	0.7241	0.7161	0.5220	0.5042
auth2	0.4959	0.7847	0.6819	0.7091	0.5506	0.4754
auth3	0.5070	0.7592	0.7322	0.7259	0.5316	0.5121
auth4	0.4937	0.7828	0.6850	0.7097	0.5473	0.4751
auth5	0.5070	0.7592	0.7322	0.7259	0.5316	0.5121
MeSHLabeler	0.5215	0.7485	0.7740	0.7431	0.5261	0.5428
MeSH Now BF	0.5066	0.7559	0.7276	0.7182	0.5323	0.5158
MeSH Now HR	0.2516	0.2446	0.9665	0.3825	0.1580	0.6669
MeSHLabeler-1	0.5226	0.7485	0.7739	0.7428	0.5276	0.5441
MeSHLabeler-2	0.5123	0.7981	0.7113	0.7297	0.5670	0.4964
MeSHLabeler-3	0.5207	0.7954	0.7240	0.7381	0.5669	0.5075
MeSHLabeler-4	0.5197	0.7956	0.7222	0.7364	0.5665	0.5071
USI baseline	0.5018	0.7935	0.6835	0.7129	0.5653	0.4766
iria-1	0.3977	0.7439	0.5103	0.5789	0.5109	0.3475

+ Test batch 1, week 5

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.0912	0.5681	0.0686	0.1041	0.0677	0.0180	0.0196	0.1786	0.0613	0.0579
MTI First Line Index	0.5593	0.6332	0.5241	0.5514	0.5931	0.4674	0.4564	0.6273	0.5047	0.3966
Default MTI	0.5635	0.5820	0.5712	0.5562	0.5418	0.5136	0.4802	0.5771	0.5506	0.4000
TextCategorisation1	0.5574	0.7786	0.4509	0.5375	0.7473	0.3394	0.3637	0.7548	0.4419	0.3877
TextCategorisation2	0.5836	0.7079	0.5191	0.5693	0.6714	0.4138	0.4247	0.6885	0.5064	0.4156
TextCategorisation3	0.5847	0.6984	0.5256	0.5708	0.6628	0.4216	0.4298	0.6802	0.5128	0.4168
TextCategorisation4	0.5877	0.6611	0.5547	0.5750	0.6255	0.4582	0.4565	0.6419	0.5419	0.4196
TextCategorisation5	0.5845	0.6310	0.5724	0.5724	0.5974	0.4727	0.4627	0.6115	0.5598	0.4162
spoon-spoon	0.4630	0.4810	0.4658	0.4535	0.5022	0.3025	0.3004	0.4770	0.4498	0.3065
fork-fork	0.4171	0.6371	0.3153	0.3977	0.6700	0.1567	0.1778	0.6457	0.3080	0.2642
it is a test submit	0.3004	0.2669	0.3685	0.2967	0.2734	0.3254	0.2680	0.2669	0.3436	0.1806
USI 10 neighbours	0.5133	0.5547	0.5086	0.5067	0.5103	0.3832	0.3773	0.5395	0.4896	0.3535
USI 20 neighbours	0.5622	0.6174	0.5405	0.5571	0.5775	0.4731	0.4609	0.6191	0.5148	0.4015
Abstract framework	0.5174	0.5624	0.5146	0.5103	0.5079	0.3849	0.3790	0.5395	0.4971	0.3573
auth1	0.5884	0.5919	0.5989	0.5782	0.5962	0.4573	0.4493	0.5951	0.5819	0.4207
auth2	0.5751	0.6501	0.5288	0.5640	0.6193	0.4370	0.4358	0.6560	0.5120	0.4078
auth3	0.5881	0.5903	0.5962	0.5758	0.6455	0.4444	0.4483	0.5968	0.5796	0.4181
auth4	0.5652	0.6169	0.5288	0.5513	0.6908	0.3719	0.3911	0.6277	0.5140	0.3958
MeSHLabeler	0.6164	0.6013	0.6528	0.6088	0.5780	0.5494	0.5198	0.6032	0.6301	0.4508
MeSH Now BF	0.5882	0.6137	0.5896	0.5799	0.5891	0.4534	0.4477	0.6051	0.5722	0.4228
MeSH Now HR	0.2140	0.1227	0.8536	0.2109	0.0935	0.7526	0.2961	0.1225	0.8478	0.1199
MeSHLabeler-1	0.6200	0.6671	0.6065	0.6136	0.6340	0.4980	0.4941	0.6655	0.5804	0.4574
MeSHLabeler-2	0.6119	0.6675	0.5954	0.6068	0.6179	0.4842	0.4824	0.6621	0.5689	0.4507
MeSHLabeler-3	0.6199	0.6867	0.5878	0.6122	0.6567	0.4870	0.4894	0.6846	0.5664	0.4565
MeSHLabeler-4	0.6209	0.6799	0.5965	0.6145	0.6467	0.4930	0.4927	0.6788	0.5721	0.4587
USI baseline	0.5882	0.6588	0.5550	0.5824	0.6308	0.4159	0.4248	0.6615	0.5295	0.4258
iria-1	0.4522	0.4749	0.4560	0.4466	0.3903	0.3001	0.2922	0.4749	0.4315	0.3012

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1073	0.6464	0.1156	0.1407	0.2722	0.0850
MTI First Line Index	0.4689	0.7861	0.6478	0.6836	0.5431	0.4413
Default MTI	0.4785	0.7434	0.7001	0.6962	0.5148	0.4785
TextCategorisation1	0.4313	0.8798	0.5274	0.6217	0.5952	0.3666
TextCategorisation2	0.4675	0.8345	0.6127	0.6744	0.5736	0.4262
TextCategorisation3	0.4699	0.8278	0.6205	0.6776	0.5703	0.4317
TextCategorisation4	0.4783	0.8032	0.6538	0.6905	0.5524	0.4556
TextCategorisation5	0.4805	0.7838	0.6750	0.6954	0.5375	0.4700
spoon-spoon	0.3913	0.6665	0.5867	0.5967	0.4241	0.3961
fork-fork	0.3258	0.7922	0.3626	0.4633	0.5298	0.2573
it is a test submit	0.3260	0.5202	0.6122	0.5406	0.3186	0.3600
USI 10 neighbours	0.4420	0.7198	0.6494	0.6541	0.4893	0.4360
USI 20 neighbours	0.4728	0.7695	0.6656	0.6891	0.5346	0.4521
Abstract framework	0.4419	0.7254	0.6475	0.6526	0.4915	0.4374
auth1	0.4907	0.7432	0.7201	0.7100	0.5143	0.4990
auth2	0.4675	0.7870	0.6474	0.6868	0.5366	0.4414
auth3	0.4779	0.7596	0.7020	0.7081	0.5016	0.4861
auth4	0.4528	0.7892	0.6266	0.6753	0.5174	0.4296
MeSHLabeler	0.5130	0.7467	0.7586	0.7323	0.5225	0.5343
MeSH Now BF	0.4922	0.7569	0.7124	0.7085	0.5262	0.4962
MeSH Now HR	0.2468	0.2444	0.9620	0.3801	0.1551	0.6627
MeSHLabeler-1	0.5092	0.7959	0.7070	0.7249	0.5624	0.4961
MeSHLabeler-2	0.5032	0.7964	0.6960	0.7178	0.5625	0.4863
MeSHLabeler-3	0.5059	0.8109	0.6882	0.7199	0.5747	0.4822
MeSHLabeler-4	0.5087	0.8056	0.6975	0.7239	0.5702	0.4893
USI baseline	0.4881	0.7907	0.6737	0.7027	0.5549	0.4651
iria-1	0.4026	0.6627	0.6027	0.6043	0.4423	0.4001

+ Test batch 2, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.0859	0.5160	0.0688	0.1008	0.0618	0.0210	0.0217	0.1473	0.0606	0.0565
MTI First Line Index	0.5614	0.6243	0.5314	0.5528	0.5733	0.4805	0.4631	0.6192	0.5135	0.3987
Default MTI	0.5664	0.5757	0.5815	0.5582	0.5294	0.5281	0.4902	0.5706	0.5622	0.4024
TextCategorisation1	0.5721	0.7465	0.4829	0.5595	0.7274	0.3729	0.3887	0.7357	0.4680	0.4064
TextCategorisation2	0.5889	0.6890	0.5352	0.5757	0.6645	0.4298	0.4342	0.6766	0.5213	0.4210
TextCategorisation3	0.5909	0.6820	0.5426	0.5778	0.6574	0.4367	0.4390	0.6696	0.5287	0.4230
TextCategorisation4	0.5894	0.6417	0.5693	0.5774	0.6079	0.4639	0.4553	0.6281	0.5552	0.4214
TextCategorisation5	0.5858	0.6099	0.5889	0.5739	0.5812	0.4804	0.4631	0.5968	0.5752	0.4169
spoon-spoon	0.4645	0.4806	0.4702	0.4561	0.4920	0.3115	0.3041	0.4751	0.4543	0.3079
fork-fork	0.4174	0.6363	0.3168	0.3992	0.6643	0.1594	0.1763	0.6418	0.3092	0.2643
it is a test submit	0.3010	0.2663	0.3695	0.2980	0.2611	0.3281	0.2718	0.2663	0.3460	0.1825
qaiiit system 1	0.2462	0.2036	0.3176	0.2399	0.0828	0.0878	0.0665	0.2028	0.3130	0.1438
USI 10 neighbours	0.5046	0.5399	0.5037	0.4973	0.4888	0.3726	0.3615	0.5248	0.4858	0.3446
USI 20 neighbours	0.5652	0.6174	0.5451	0.5602	0.5722	0.4735	0.4597	0.6191	0.5200	0.4047
Abstract framework	0.5085	0.5486	0.5104	0.5004	0.4855	0.3754	0.3637	0.5240	0.4939	0.3480
auth1	0.5852	0.5853	0.5982	0.5756	0.5880	0.4646	0.4520	0.5883	0.5820	0.4185
auth2	0.5895	0.6135	0.5786	0.5789	0.5951	0.4591	0.4494	0.6186	0.5629	0.4224
auth3	0.6019	0.6037	0.6130	0.5917	0.6051	0.4804	0.4688	0.6066	0.5972	0.4345
auth4	0.5890	0.6297	0.5648	0.5779	0.6366	0.4287	0.4304	0.6340	0.5500	0.4218
MeSHLabeler	0.6154	0.6744	0.5923	0.6087	0.6171	0.4977	0.4907	0.6740	0.5662	0.4532
MeSH Now BF	0.5838	0.6043	0.5916	0.5765	0.5752	0.4576	0.4448	0.5955	0.5725	0.4196
MeSH Now HR	0.2122	0.1217	0.8519	0.2097	0.0933	0.7549	0.3253	0.1213	0.8448	0.1190
MeSHLabeler-1	0.6196	0.6717	0.6009	0.6127	0.6163	0.5057	0.4971	0.6716	0.5750	0.4571
MeSHLabeler-2	0.6083	0.6568	0.5977	0.6031	0.6013	0.4905	0.4809	0.6494	0.5722	0.4469
MeSHLabeler-3	0.6218	0.6651	0.6057	0.6148	0.6273	0.5015	0.4931	0.6658	0.5832	0.4590
MeSHLabeler-4	0.6193	0.6581	0.6097	0.6120	0.6150	0.5074	0.4950	0.6575	0.5853	0.4559
USI baseline	0.5842	0.6537	0.5523	0.5799	0.6242	0.4151	0.4181	0.6574	0.5257	0.4234
iria-1	0.4557	0.4763	0.4617	0.4510	0.3950	0.3037	0.2934	0.4763	0.4369	0.3047
IIIT system 2	0.3152	0.2617	0.4047	0.3071	0.1230	0.1336	0.1011	0.2601	0.3998	0.1907
MeSH-UK0	0.5885	0.5761	0.6132	0.5788	0.5558	0.4862	0.4669	0.5818	0.5954	0.4223
MeSH-UK3	0.5658	0.6107	0.5408	0.5544	0.5369	0.4287	0.4147	0.6168	0.5225	0.3997
MeSH-UK4	0.5662	0.5747	0.5664	0.5541	0.5084	0.4568	0.4307	0.5834	0.5499	0.3988
Dexstr system	0.3634	0.3798	0.3655	0.3580	0.3338	0.3181	0.2832	0.3798	0.3484	0.2258
iria-2	0.4078	0.3272	0.5573	0.3991	0.2978	0.4687	0.3766	0.3294	0.5353	0.2585
iria-3	0.4269	0.3266	0.6269	0.4163	0.2989	0.4746	0.3789	0.3288	0.6086	0.2724

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1126	0.6102	0.1306	0.1522	0.2632	0.0925
MTI First Line Index	0.4689	0.7767	0.6559	0.6844	0.5348	0.4466
Default MTI	0.4793	0.7367	0.7111	0.6985	0.5064	0.4870
TextCategorisation1	0.4506	0.8610	0.5663	0.6503	0.5943	0.3913
TextCategorisation2	0.4702	0.8193	0.6264	0.6786	0.5653	0.4341
TextCategorisation3	0.4726	0.8141	0.6344	0.6819	0.5614	0.4403
TextCategorisation4	0.4777	0.7867	0.6646	0.6903	0.5394	0.4627
TextCategorisation5	0.4786	0.7641	0.6839	0.6921	0.5215	0.4784
spoon-spoon	0.3897	0.6676	0.5893	0.5995	0.4201	0.3937
fork-fork	0.3247	0.7921	0.3628	0.4637	0.5250	0.2562
it is a test submit	0.3242	0.5155	0.6121	0.5392	0.3134	0.3601
qaiiit system 1	0.2713	0.3813	0.5156	0.4195	0.2408	0.3367
USI 10 neighbours	0.4355	0.7082	0.6499	0.6497	0.4773	0.4328
USI 20 neighbours	0.4721	0.7659	0.6672	0.6881	0.5299	0.4533
Abstract framework	0.4332	0.7155	0.6455	0.6465	0.4770	0.4325
auth1	0.4846	0.7334	0.7170	0.7040	0.5032	0.4950
auth2	0.4845	0.7556	0.6924	0.7010	0.5204	0.4800
auth3	0.4956	0.7497	0.7261	0.7168	0.5155	0.5056
auth4	0.4816	0.7728	0.6761	0.6987	0.5322	0.4659
MeSHLabeler	0.5011	0.7977	0.6926	0.7165	0.5587	0.4847
MeSH Now BF	0.4892	0.7498	0.7147	0.7065	0.5164	0.4979
MeSH Now HR	0.2435	0.2451	0.9623	0.3817	0.1525	0.6599
MeSHLabeler-1	0.5054	0.7965	0.7016	0.7205	0.5596	0.4918
MeSHLabeler-2	0.4990	0.7880	0.6989	0.7149	0.5494	0.4890
MeSHLabeler-3	0.5079	0.7947	0.7050	0.7239	0.5578	0.4954
MeSHLabeler-4	0.5072	0.7900	0.7113	0.7239	0.5531	0.4996
USI baseline	0.4860	0.7872	0.6746	0.7026	0.5481	0.4639
iria-1	0.4004	0.6620	0.6011	0.6038	0.4376	0.3962
IIIT system 2	0.3192	0.4557	0.5939	0.4954	0.2866	0.3894
MeSH-UK0	0.4876	0.7260	0.7295	0.7077	0.4959	0.5075
MeSH-UK3	0.4637	0.7529	0.6482	0.6717	0.5186	0.4471
MeSH-UK4	0.4686	0.7221	0.6805	0.6783	0.4971	0.4700
Dexstr system	0.3359	0.5849	0.5511	0.5444	0.3452	0.3506
iria-2	0.3821	0.5155	0.7673	0.5958	0.3296	0.4906
iria-3	0.3916	0.5135	0.7852	0.6002	0.3271	0.5265

+ Test batch 2, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.0969	0.5925	0.0760	0.1136	0.0739	0.0290	0.0296	0.1613	0.0693	0.0631
MTI First Line Index	0.5857	0.6331	0.5663	0.5764	0.5779	0.4862	0.4656	0.6259	0.5503	0.4209
Default MTI	0.5875	0.5806	0.6191	0.5791	0.5343	0.5376	0.4942	0.5747	0.6010	0.4221
TextCategorisation1	0.6063	0.7416	0.5290	0.5914	0.7256	0.3785	0.3956	0.7309	0.5180	0.4383
TextCategorisation2	0.6159	0.6824	0.5831	0.6029	0.6569	0.4356	0.4371	0.6688	0.5708	0.4480
TextCategorisation3	0.6165	0.6745	0.5900	0.6036	0.6486	0.4434	0.4425	0.6607	0.5779	0.4484
TextCategorisation4	0.6136	0.6369	0.6176	0.6017	0.6038	0.4747	0.4616	0.6223	0.6052	0.4455
TextCategorisation5	0.6068	0.6062	0.6361	0.5956	0.5766	0.4898	0.4670	0.5908	0.6237	0.4383
spoon-spoon	0.4940	0.5077	0.5041	0.4872	0.4982	0.3066	0.3011	0.5005	0.4877	0.3361
fork-fork	0.4589	0.6535	0.3641	0.4406	0.6618	0.1674	0.1854	0.6528	0.3537	0.2995
it is a test submit	0.3164	0.2797	0.3868	0.3138	0.2666	0.3447	0.2816	0.2797	0.3644	0.1931
qaiiit system 1	0.3465	0.2878	0.4463	0.3392	0.1312	0.1448	0.1097	0.2862	0.4390	0.2143
USI 10 neighbours	0.5328	0.5492	0.5449	0.5236	0.5012	0.3810	0.3698	0.5364	0.5293	0.3683
USI 20 neighbours	0.5884	0.6268	0.5783	0.5831	0.5791	0.4787	0.4643	0.6267	0.5545	0.4266
Abstract framework	0.5379	0.5553	0.5553	0.5285	0.4972	0.3847	0.3723	0.5355	0.5402	0.3735
auth1	0.6149	0.6159	0.6303	0.6070	0.6050	0.4788	0.4666	0.6148	0.6150	0.4508
auth2	0.6185	0.6535	0.5987	0.6080	0.6340	0.4549	0.4516	0.6567	0.5846	0.4525
auth3	0.6251	0.6563	0.6108	0.6150	0.6670	0.4418	0.4457	0.6566	0.5964	0.4599
auth4	0.6101	0.6829	0.5641	0.5972	0.6940	0.3889	0.4053	0.6820	0.5520	0.4433
MeSHLabeler	0.6455	0.6906	0.6254	0.6366	0.6368	0.5084	0.5041	0.6911	0.6055	0.4832
MeSH Now BF	0.2175	0.1247	0.8756	0.2151	0.0929	0.7626	0.3142	0.1243	0.8685	0.1223
MeSHLabeler-1	0.6448	0.6991	0.6186	0.6369	0.6423	0.5033	0.5022	0.6987	0.5986	0.4837
MeSHLabeler-2	0.6489	0.6910	0.6328	0.6418	0.6515	0.5049	0.5002	0.6900	0.6124	0.4881
MeSHLabeler-3	0.6456	0.6888	0.6274	0.6364	0.6495	0.5045	0.5003	0.6870	0.6090	0.4828
MeSHLabeler-4	0.6476	0.6922	0.6299	0.6407	0.6491	0.5024	0.4983	0.6911	0.6093	0.4870
USI baseline	0.6144	0.6687	0.5897	0.6082	0.6382	0.4284	0.4329	0.6713	0.5665	0.4522
iria-1	0.4863	0.5075	0.4900	0.4822	0.4093	0.2977	0.2921	0.5075	0.4667	0.3323
MeSH-UK0	0.6195	0.6049	0.6471	0.6104	0.5783	0.4995	0.4804	0.6089	0.6305	0.4542
MeSH-UK1	0.6227	0.6089	0.6520	0.6147	0.5782	0.5089	0.4843	0.6121	0.6338	0.4586
MeSH-UK3	0.6190	0.6312	0.6209	0.6085	0.5836	0.4793	0.4638	0.6338	0.6050	0.4529
MeSH-UK4	0.6280	0.6503	0.6235	0.6185	0.6233	0.4772	0.4667	0.6517	0.6061	0.4633
Dexstr system	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
iria-2	0.4835	0.4162	0.5896	0.4753	0.3636	0.4230	0.3711	0.4176	0.5741	0.3242
iria-3	0.4848	0.4160	0.5922	0.4763	0.3650	0.4218	0.3699	0.4180	0.5768	0.3251

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1182	0.6816	0.1424	0.1646	0.2775	0.0976
MTI First Line Index	0.4866	0.7707	0.6866	0.6998	0.5415	0.4731
Default MTI	0.4939	0.7279	0.7421	0.7107	0.5088	0.5141
TextCategorisation1	0.4823	0.8518	0.6097	0.6807	0.6027	0.4329
TextCategorisation2	0.4999	0.8083	0.6719	0.7049	0.5699	0.4798
TextCategorisation3	0.5012	0.8022	0.6795	0.7069	0.5652	0.4854
TextCategorisation4	0.5051	0.7766	0.7116	0.7146	0.5448	0.5087
TextCategorisation5	0.5041	0.7521	0.7318	0.7142	0.5252	0.5237
spoon-spoon	0.4195	0.6840	0.6165	0.6229	0.4478	0.4266
fork-fork	0.3606	0.8026	0.4088	0.5054	0.5456	0.2949
it is a test submit	0.3305	0.5218	0.6278	0.5510	0.3191	0.3664
qaiiit system 1	0.3420	0.4678	0.6151	0.5131	0.3067	0.4167
USI 10 neighbours	0.4586	0.7034	0.6775	0.6618	0.4877	0.4682
USI 20 neighbours	0.4895	0.7621	0.6942	0.7028	0.5358	0.4795
Abstract framework	0.4591	0.7083	0.6766	0.6612	0.4880	0.4721
auth1	0.5095	0.7446	0.7403	0.7220	0.5275	0.5224
auth2	0.5057	0.7760	0.7022	0.7153	0.5498	0.4965
auth3	0.5109	0.7779	0.7118	0.7209	0.5528	0.5041
auth4	0.4927	0.8011	0.6612	0.6982	0.5669	0.4647
MeSHLabeler	0.5262	0.8017	0.7207	0.7362	0.5764	0.5140
MeSH Now BF	0.2467	0.2369	0.9691	0.3728	0.1541	0.6700
MeSHLabeler-1	0.5259	0.8084	0.7143	0.7361	0.5819	0.5085
MeSHLabeler-2	0.5315	0.8041	0.7282	0.7428	0.5772	0.5219
MeSHLabeler-3	0.5261	0.8020	0.7233	0.7379	0.5731	0.5160
MeSHLabeler-4	0.5305	0.8043	0.7253	0.7413	0.5782	0.5191
USI baseline	0.5114	0.7876	0.7054	0.7214	0.5652	0.4963
iria-1	0.4289	0.6801	0.6144	0.6222	0.4678	0.4239
MeSH-UK0	0.5115	0.7334	0.7541	0.7243	0.5168	0.5354
MeSH-UK1	0.5179	0.7413	0.7534	0.7278	0.5264	0.5383
MeSH-UK3	0.5074	0.7557	0.7192	0.7144	0.5340	0.5126
MeSH-UK4	0.5153	0.7751	0.7174	0.7223	0.5495	0.5140
Dexstr system	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
iria-2	0.4314	0.5829	0.7428	0.6348	0.3952	0.5053
iria-3	0.4316	0.5802	0.7497	0.6361	0.3934	0.5075

+ Test batch 2, week 3

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.0903	0.5279	0.0728	0.1059	0.0774	0.0295	0.0307	0.1477	0.0651	0.0594
MTI First Line Index	0.5660	0.6396	0.5264	0.5576	0.5980	0.4625	0.4508	0.6359	0.5100	0.4022
Default MTI	0.5734	0.5944	0.5744	0.5654	0.5536	0.5109	0.4794	0.5898	0.5578	0.4087
TextCategorisation1	0.5748	0.7546	0.4777	0.5603	0.7384	0.3545	0.3753	0.7476	0.4669	0.4065
TextCategorisation2	0.5931	0.7028	0.5293	0.5785	0.6775	0.4142	0.4234	0.6912	0.5194	0.4231
TextCategorisation3	0.5951	0.6953	0.5368	0.5805	0.6706	0.4226	0.4295	0.6837	0.5269	0.4249
TextCategorisation4	0.5978	0.6578	0.5667	0.5843	0.6290	0.4554	0.4522	0.6457	0.5566	0.4280
TextCategorisation5	0.5942	0.6281	0.5847	0.5813	0.6014	0.4715	0.4601	0.6143	0.5753	0.4243
it is a test submit	0.3016	0.2715	0.3607	0.2975	0.2741	0.3261	0.2725	0.2715	0.3393	0.1812
qaiiit system 1	0.3399	0.2851	0.4272	0.3301	0.1752	0.1446	0.1112	0.2848	0.4215	0.2071
USI 10 neighbours	0.5186	0.5593	0.5077	0.5086	0.5043	0.3778	0.3699	0.5482	0.4921	0.3546
USI 20 neighbours	0.5725	0.6362	0.5405	0.5671	0.5952	0.4625	0.4536	0.6397	0.5180	0.4110
Abstract framework	0.5246	0.5698	0.5163	0.5138	0.5015	0.3817	0.3735	0.5489	0.5023	0.3601
auth1	0.5925	0.6015	0.5928	0.5809	0.6127	0.4540	0.4493	0.6066	0.5789	0.4233
auth2	0.5934	0.6445	0.5560	0.5800	0.6401	0.4306	0.4342	0.6526	0.5441	0.4234
auth3	0.6005	0.6517	0.5653	0.5876	0.6771	0.4152	0.4267	0.6562	0.5536	0.4312
auth4	0.5813	0.6767	0.5161	0.5646	0.7015	0.3612	0.3827	0.6799	0.5077	0.4100
MeSHLabeler	0.6240	0.6854	0.5922	0.6155	0.6421	0.4908	0.4914	0.6894	0.5700	0.4602
MeSH Now BF	0.5906	0.6158	0.5866	0.5811	0.5908	0.4490	0.4435	0.6108	0.5718	0.4240
MeSH Now HR	0.2189	0.1260	0.8497	0.2159	0.0990	0.7500	0.3121	0.1257	0.8444	0.1231
MeSHLabeler-1	0.6229	0.6896	0.5894	0.6145	0.6428	0.4875	0.4886	0.6921	0.5662	0.4590
MeSHLabeler-2	0.6248	0.6809	0.5974	0.6161	0.6521	0.4856	0.4851	0.6831	0.5756	0.4598
MeSHLabeler-3	0.6258	0.6835	0.5960	0.6173	0.6576	0.4838	0.4836	0.6869	0.5747	0.4612
MeSHLabeler-4	0.6227	0.6879	0.5898	0.6142	0.6406	0.4877	0.4887	0.6911	0.5666	0.4586
USI baseline	0.5897	0.6661	0.5479	0.5833	0.6402	0.4084	0.4181	0.6727	0.5249	0.4268
iria-1	0.4647	0.5197	0.4436	0.4598	0.4268	0.2760	0.2763	0.5197	0.4202	0.3118
IIIT system 2	0.4140	0.3474	0.5230	0.4029	0.3333	0.3342	0.2918	0.3473	0.5124	0.2642
IIIT system 3	0.4414	0.4483	0.4381	0.4258	0.4900	0.2506	0.2437	0.4515	0.4319	0.2825
IIIT system 4	0.4363	0.4134	0.4645	0.4210	0.4564	0.2787	0.2603	0.4158	0.4589	0.2779
QA tam from IIIT Hyd	0.3602	0.2659	0.6004	0.3555	0.2593	0.3864	0.2861	0.2593	0.5895	0.2247
MeSH-UK0	0.5961	0.5912	0.6079	0.5839	0.5881	0.4730	0.4621	0.6002	0.5921	0.4264
MeSH-UK1	0.6036	0.5987	0.6156	0.5914	0.6061	0.4834	0.4708	0.6077	0.5995	0.4344
MeSH-UK3	0.6040	0.6758	0.5574	0.5907	0.6750	0.4201	0.4301	0.6839	0.5409	0.4354
MeSH-UK4	0.6081	0.6425	0.5863	0.5951	0.6478	0.4496	0.4506	0.6514	0.5703	0.4393
Dexstr system	0.4112	0.4386	0.4109	0.4072	0.4088	0.3527	0.3258	0.4386	0.3871	0.2643
iria-2	0.4696	0.4121	0.5536	0.4592	0.3685	0.4140	0.3670	0.4162	0.5385	0.3100
iria-3	0.4698	0.4111	0.5565	0.4594	0.3671	0.4141	0.3658	0.4149	0.5414	0.3102
testLee15	0.3016	0.2790	0.3496	0.2978	0.2819	0.3164	0.2698	0.2790	0.3282	0.1816
testLee16	0.3016	0.2790	0.3496	0.2978	0.2819	0.3164	0.2698	0.2790	0.3282	0.1816
testLee17	0.1691	0.1557	0.1968	0.1647	0.3759	0.2504	0.2108	0.1530	0.1890	0.0933
IIIT system 5	0.3602	0.2659	0.6004	0.3555	0.2593	0.3864	0.2861	0.2593	0.5895	0.2247
IIIT system 11	0.3602	0.2659	0.6004	0.3555	0.2593	0.3864	0.2861	0.2593	0.5895	0.2247

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.1243	0.6344	0.1506	0.1731	0.2746	0.1038
MTI First Line Index	0.4756	0.7864	0.6466	0.6855	0.5508	0.4437
Default MTI	0.4878	0.7504	0.7014	0.7027	0.5249	0.4827
TextCategorisation1	0.4566	0.8662	0.5574	0.6485	0.6082	0.3911
TextCategorisation2	0.4785	0.8304	0.6164	0.6786	0.5837	0.4342
TextCategorisation3	0.4812	0.8248	0.6245	0.6821	0.5792	0.4408
TextCategorisation4	0.4901	0.7994	0.6587	0.6952	0.5590	0.4667
TextCategorisation5	0.4912	0.7784	0.6799	0.6993	0.5410	0.4820
it is a test submit	0.3268	0.5275	0.6078	0.5447	0.3213	0.3558
qaiiit system 1	0.3459	0.4884	0.6051	0.5198	0.3183	0.4084
USI 10 neighbours	0.4492	0.7248	0.6507	0.6581	0.4961	0.4416
USI 20 neighbours	0.4824	0.7803	0.6625	0.6945	0.5485	0.4547
Abstract framework	0.4482	0.7326	0.6475	0.6566	0.4968	0.4427
auth1	0.4965	0.7484	0.7121	0.7100	0.5239	0.4970
auth2	0.4896	0.7849	0.6671	0.7005	0.5465	0.4670
auth3	0.4946	0.7881	0.6734	0.7042	0.5532	0.4716
auth4	0.4713	0.8101	0.6173	0.6750	0.5639	0.4294
MeSHLabeler	0.5130	0.8084	0.6937	0.7241	0.5755	0.4903
MeSH Now BF	0.4963	0.7568	0.7100	0.7096	0.5297	0.4967
MeSH Now HR	0.2550	0.2486	0.9620	0.3863	0.1610	0.6664
MeSHLabeler-1	0.5094	0.8080	0.6883	0.7198	0.5744	0.4863
MeSHLabeler-2	0.5134	0.8038	0.6990	0.7247	0.5716	0.4939
MeSHLabeler-3	0.5141	0.8075	0.6974	0.7259	0.5742	0.4929
MeSHLabeler-4	0.5101	0.8073	0.6899	0.7206	0.5736	0.4873
USI baseline	0.4914	0.7940	0.6666	0.7022	0.5623	0.4623
iria-1	0.4087	0.6988	0.5687	0.6010	0.4731	0.3860
IIIT system 2	0.3832	0.5434	0.6743	0.5812	0.3499	0.4571
IIIT system 3	0.3985	0.6826	0.5513	0.5824	0.4453	0.3880
IIIT system 4	0.4020	0.6496	0.5915	0.5931	0.4215	0.4137
QA tam from IIIT Hyd	0.3598	0.4481	0.7740	0.5481	0.2880	0.5229
MeSH-UK0	0.4984	0.7404	0.7235	0.7127	0.5146	0.5092
MeSH-UK1	0.4986	0.7422	0.7212	0.7130	0.5157	0.5086
MeSH-UK3	0.4887	0.8021	0.6505	0.6945	0.5633	0.4583
MeSH-UK4	0.4961	0.7761	0.6847	0.7064	0.5426	0.4833
Dexstr system	0.3752	0.6281	0.6240	0.6034	0.3856	0.3917
iria-2	0.4224	0.5955	0.7193	0.6333	0.3952	0.4812
iria-3	0.4220	0.5923	0.7242	0.6326	0.3928	0.4840
testLee15	0.3256	0.5374	0.5951	0.5446	0.3272	0.3463
testLee16	0.3256	0.5374	0.5951	0.5446	0.3272	0.3463
testLee17	0.2404	0.4131	0.4524	0.4097	0.2337	0.2721
IIIT system 5	0.3598	0.4481	0.7740	0.5481	0.2880	0.5229
IIIT system 11	0.3598	0.4481	0.7740	0.5481	0.2880	0.5229

+ Test batch 2, week 4

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2658	0.2539	0.3157	0.2645	0.3555	0.4155	0.3516	0.2350	0.3058	0.1578
MTI First Line Index	0.5799	0.6491	0.5486	0.5722	0.5870	0.4827	0.4663	0.6408	0.5295	0.4183
Default MTI	0.5889	0.6039	0.6030	0.5837	0.5441	0.5358	0.4998	0.5956	0.5823	0.4294
spoon-spoon	0.4742	0.4826	0.4830	0.4632	0.4914	0.3171	0.3097	0.4811	0.4675	0.3155
fork-fork	0.4336	0.6478	0.3281	0.4107	0.6666	0.1672	0.1847	0.6564	0.3237	0.2754
it is a test submit	0.3007	0.2635	0.3768	0.2972	0.2595	0.3501	0.2869	0.2635	0.3500	0.1809
qaiiit system 1	0.3055	0.2316	0.4657	0.2981	0.1237	0.1905	0.1258	0.2307	0.4519	0.1823
USI 10 neighbours	0.5237	0.5654	0.5244	0.5176	0.4934	0.3835	0.3753	0.5443	0.5046	0.3635
USI 20 neighbours	0.5866	0.6413	0.5687	0.5839	0.5852	0.4862	0.4738	0.6414	0.5404	0.4306
Abstract framework	0.5262	0.5734	0.5293	0.5194	0.4890	0.3845	0.3748	0.5426	0.5108	0.3658
auth1	0.6023	0.6075	0.6109	0.5921	0.5964	0.4770	0.4673	0.6077	0.5970	0.4359
auth2	0.6027	0.6462	0.5753	0.5908	0.6247	0.4502	0.4483	0.6492	0.5624	0.4353
auth3	0.6095	0.6356	0.5982	0.5984	0.6523	0.4375	0.4419	0.6355	0.5856	0.4429
auth5	0.6033	0.6411	0.5794	0.5910	0.6205	0.4569	0.4538	0.6448	0.5668	0.4355
MeSHLabeler	0.6356	0.6876	0.6158	0.6287	0.6205	0.5139	0.5080	0.6842	0.5935	0.4749
MeSH Now BF	0.6039	0.6246	0.6073	0.5955	0.5803	0.4631	0.4534	0.6153	0.5929	0.4389
MeSH Now HR	0.2109	0.1205	0.8629	0.2078	0.0932	0.7710	0.3322	0.1202	0.8593	0.1181
MeSHLabeler-1	0.6352	0.6937	0.6108	0.6289	0.6256	0.5088	0.5048	0.6911	0.5877	0.4749
MeSHLabeler-2	0.6408	0.6868	0.6231	0.6345	0.6475	0.5142	0.5085	0.6850	0.6020	0.4804
MeSHLabeler-3	0.6417	0.6885	0.6228	0.6350	0.6466	0.5154	0.5099	0.6858	0.6029	0.4812
MeSHLabeler-4	0.6403	0.6882	0.6218	0.6341	0.6480	0.5143	0.5093	0.6859	0.6004	0.4800
USI baseline	0.6037	0.6715	0.5709	0.5984	0.6253	0.4227	0.4289	0.6747	0.5463	0.4424
iria-1	0.4663	0.4818	0.4814	0.4612	0.3835	0.3114	0.2997	0.4818	0.4518	0.3136
IIIT system 2	0.2913	0.2099	0.6418	0.3028	0.1650	0.4847	0.2986	0.1880	0.6473	0.1852
IIIT system 3	0.4927	0.4338	0.5847	0.4810	0.4286	0.4295	0.3846	0.4314	0.5743	0.3285
MeSH-UK0	0.6152	0.6026	0.6428	0.6055	0.5842	0.5096	0.4911	0.6064	0.6243	0.4494
MeSH-UK1	0.6207	0.6554	0.6082	0.6100	0.6345	0.4726	0.4689	0.6549	0.5899	0.4554
MeSH-UK2	0.6198	0.6698	0.5962	0.6091	0.6346	0.4726	0.4689	0.6684	0.5778	0.4545
MeSH-UK3	0.6205	0.6652	0.6014	0.6103	0.6345	0.4726	0.4689	0.6636	0.5826	0.4557
MeSH-UK4	0.6119	0.6594	0.5897	0.6009	0.6261	0.4614	0.4592	0.6576	0.5722	0.4458
Dexstr system	0.3986	0.4119	0.4087	0.3924	0.4020	0.3413	0.3186	0.4119	0.3863	0.2530
iria-2	0.4770	0.4109	0.5852	0.4692	0.3752	0.4319	0.3807	0.4134	0.5639	0.3187
iria-3	0.4805	0.4116	0.5915	0.4720	0.3733	0.4303	0.3777	0.4147	0.5711	0.3206
testLee15	0.2705	0.1904	0.5217	0.2708	0.2224	0.5578	0.3925	0.1856	0.4988	0.1604
testLee16	0.2745	0.2272	0.4041	0.2763	0.2718	0.3828	0.3111	0.2119	0.3898	0.1650
pseudo n-grams	0.4007	0.3971	0.4117	0.3871	0.2870	0.3577	0.2888	0.4038	0.3976	0.2501

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3077	0.5230	0.5896	0.5268	0.3141	0.3310
MTI First Line Index	0.4866	0.7875	0.6693	0.6974	0.5581	0.4624
Default MTI	0.5009	0.7500	0.7277	0.7159	0.5317	0.5062
spoon-spoon	0.3996	0.6651	0.6028	0.6044	0.4279	0.4102
fork-fork	0.3345	0.7960	0.3727	0.4734	0.5309	0.2661
it is a test submit	0.3243	0.5136	0.6231	0.5408	0.3118	0.3668
qaiiit system 1	0.3209	0.4165	0.6654	0.4912	0.2678	0.4412
USI 10 neighbours	0.4501	0.7218	0.6643	0.6634	0.4949	0.4482
USI 20 neighbours	0.4947	0.7771	0.6878	0.7067	0.5536	0.4754
Abstract framework	0.4487	0.7289	0.6586	0.6598	0.4954	0.4484
auth1	0.5019	0.7480	0.7309	0.7181	0.5257	0.5090
auth2	0.4942	0.7812	0.6880	0.7091	0.5463	0.4783
auth3	0.5014	0.7752	0.7032	0.7154	0.5445	0.4931
auth5	0.4948	0.7780	0.6906	0.7093	0.5432	0.4815
MeSHLabeler	0.5205	0.8039	0.7153	0.7326	0.5772	0.5056
MeSH Now BF	0.5065	0.7610	0.7279	0.7207	0.5383	0.5113
MeSH Now HR	0.2439	0.2394	0.9679	0.3744	0.1526	0.6719
MeSHLabeler-1	0.5186	0.8079	0.7104	0.7316	0.5804	0.4998
MeSHLabeler-2	0.5262	0.8073	0.7225	0.7406	0.5806	0.5104
MeSHLabeler-3	0.5269	0.8094	0.7216	0.7409	0.5812	0.5105
MeSHLabeler-4	0.5256	0.8081	0.7223	0.7406	0.5809	0.5092
USI baseline	0.5020	0.7966	0.6896	0.7160	0.5669	0.4793
iria-1	0.4125	0.6650	0.6178	0.6124	0.4466	0.4171
IIIT system 2	0.3006	0.3655	0.8232	0.4842	0.2180	0.5477
IIIT system 3	0.4286	0.6368	0.7084	0.6465	0.4091	0.4864
MeSH-UK0	0.5122	0.7436	0.7531	0.7284	0.5239	0.5306
MeSH-UK1	0.5104	0.7867	0.7092	0.7212	0.5551	0.5025
MeSH-UK2	0.5096	0.7912	0.7052	0.7203	0.5626	0.4955
MeSH-UK3	0.5101	0.7896	0.7072	0.7210	0.5601	0.4985
MeSH-UK4	0.5022	0.7846	0.7013	0.7151	0.5533	0.4899
Dexstr system	0.3574	0.5970	0.6152	0.5784	0.3611	0.3846
iria-2	0.4256	0.5931	0.7437	0.6403	0.3915	0.4984
iria-3	0.4256	0.5930	0.7479	0.6419	0.3906	0.5004
testLee15	0.3143	0.4126	0.7772	0.5220	0.2453	0.4758
testLee16	0.3182	0.4730	0.6298	0.5138	0.2830	0.4028
pseudo n-grams	0.3577	0.5453	0.6118	0.5514	0.3606	0.3865

+ Test batch 2, week 5

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
IIIT system 2	0.2533	0.1724	0.5737	0.2547	0.3022	0.3691	0.2847	0.1628	0.5704	0.1499
MeSH-UK0	0.6081	0.6618	0.5817	0.5975	0.6341	0.4502	0.4493	0.6694	0.5571	0.4428
MeSH-UK1	0.6100	0.6428	0.5960	0.5977	0.6339	0.4503	0.4493	0.6525	0.5727	0.4428
MeSH-UK2	0.6140	0.6279	0.6140	0.6013	0.6216	0.4695	0.4626	0.6377	0.5921	0.4461
MeSH-UK3	0.6081	0.6618	0.5817	0.5975	0.6341	0.4502	0.4493	0.6694	0.5571	0.4428
MeSH-UK4	0.6125	0.6467	0.5988	0.6014	0.6217	0.4694	0.4626	0.6545	0.5755	0.4462
Dexstr system	0.4261	0.4428	0.4334	0.4189	0.4056	0.3581	0.3260	0.4428	0.4106	0.2753
iria-2	0.4790	0.4106	0.5832	0.4670	0.3807	0.4345	0.3812	0.4155	0.5655	0.3175
iria-3	0.4814	0.4112	0.5907	0.4699	0.3827	0.4343	0.3805	0.4158	0.5717	0.3200
testLee15	0.3378	0.2638	0.4908	0.3303	0.2736	0.4917	0.3673	0.2649	0.4662	0.2045
pseudo n-grams	0.4043	0.4077	0.4157	0.3939	0.2965	0.3652	0.2952	0.4117	0.3973	0.2564
iria-4	0.4739	0.4044	0.5818	0.4622	0.3608	0.4354	0.3754	0.4086	0.5642	0.3135
test_unibitri	0.2581	0.2511	0.2600	0.2447	0.1735	0.2282	0.1669	0.2616	0.2548	0.1468
BioASQ_Baseline	0.2636	0.2392	0.3301	0.2602	0.3502	0.4374	0.3590	0.2268	0.3147	0.1551
MTI First Line Index	0.5807	0.6322	0.5602	0.5717	0.5864	0.5041	0.4820	0.6298	0.5386	0.4183
Default MTI	0.5821	0.5849	0.6054	0.5730	0.5431	0.5497	0.5064	0.5815	0.5828	0.4180
spoon-spoon	0.4754	0.4833	0.4875	0.4634	0.4935	0.3201	0.3138	0.4800	0.4710	0.3161
fork-fork	0.4375	0.6380	0.3360	0.4155	0.6700	0.1711	0.1901	0.6495	0.3298	0.2795
it is a test submit	0.3031	0.2643	0.3808	0.2980	0.2634	0.3459	0.2804	0.2643	0.3554	0.1826
qaiiit system 1	0.3716	0.3027	0.4916	0.3599	0.2554	0.1944	0.1547	0.3025	0.4818	0.2302
USI 10 neighbours	0.5174	0.5416	0.5210	0.5055	0.4818	0.3852	0.3715	0.5320	0.5036	0.3529
USI 20 neighbours	0.5806	0.6243	0.5690	0.5742	0.5838	0.4946	0.4786	0.6282	0.5396	0.4196
Abstract framework	0.5217	0.5505	0.5292	0.5096	0.4819	0.3903	0.3759	0.5310	0.5127	0.3572
auth1	0.6012	0.5944	0.6182	0.5881	0.5913	0.4813	0.4649	0.6011	0.6014	0.4341
auth4	0.5938	0.6506	0.5542	0.5767	0.6747	0.3920	0.4030	0.6553	0.5429	0.4245
auth5	0.5911	0.6804	0.5317	0.5746	0.6976	0.3758	0.3921	0.6853	0.5197	0.4232
MeSH Now BF	0.5984	0.6067	0.6094	0.5858	0.5764	0.4795	0.4644	0.6048	0.5921	0.4310
MeSH Now HR	0.2087	0.1192	0.8636	0.2055	0.0914	0.7785	0.3181	0.1188	0.8577	0.1168
MeSHLabeler-2	0.6241	0.6565	0.6175	0.6150	0.5973	0.5089	0.4942	0.6596	0.5923	0.4615
MeSHLabeler-3	0.6247	0.6584	0.6173	0.6156	0.5983	0.5076	0.4935	0.6613	0.5919	0.4621
MeSHLabeler-4	0.6239	0.6551	0.6187	0.6146	0.5974	0.5098	0.4946	0.6581	0.5931	0.4612
USI baseline	0.5982	0.6522	0.5729	0.5892	0.6194	0.4393	0.4404	0.6608	0.5465	0.4350
iria-1	0.4695	0.4822	0.4850	0.4618	0.3964	0.3160	0.3044	0.4822	0.4576	0.3145

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
IIIT system 2	0.2798	0.3405	0.7630	0.4498	0.2033	0.5089
MeSH-UK0	0.4918	0.7794	0.6899	0.7064	0.5472	0.4776
MeSH-UK1	0.4938	0.7738	0.6932	0.7062	0.5420	0.4851
MeSH-UK2	0.4992	0.7603	0.7138	0.7124	0.5336	0.5004
MeSH-UK3	0.4918	0.7794	0.6899	0.7064	0.5472	0.4776
MeSH-UK4	0.4980	0.7660	0.7103	0.7129	0.5396	0.4934
Dexstr system	0.3730	0.6129	0.6266	0.5919	0.3819	0.3957
iria-2	0.4199	0.5805	0.7354	0.6274	0.3872	0.4909
iria-3	0.4213	0.5780	0.7433	0.6291	0.3864	0.4967
testLee15	0.3483	0.4902	0.7198	0.5623	0.3055	0.4382
pseudo n-grams	0.3563	0.5467	0.5999	0.5449	0.3644	0.3797
iria-4	0.4169	0.5741	0.7339	0.6228	0.3828	0.4905
test_unibitri	0.2737	0.4270	0.4590	0.4179	0.2794	0.2929
BioASQ_Baseline	0.3016	0.4945	0.5989	0.5131	0.3002	0.3387
MTI First Line Index	0.4797	0.7673	0.6748	0.6902	0.5382	0.4650
Default MTI	0.4850	0.7289	0.7258	0.7013	0.5077	0.4991
spoon-spoon	0.3965	0.6558	0.6025	0.5985	0.4227	0.4095
fork-fork	0.3369	0.7869	0.3823	0.4793	0.5267	0.2700
it is a test submit	0.3224	0.4998	0.6208	0.5304	0.3090	0.3663
qaiiit system 1	0.3583	0.5041	0.6478	0.5424	0.3254	0.4370
USI 10 neighbours	0.4398	0.6972	0.6597	0.6470	0.4792	0.4427
USI 20 neighbours	0.4787	0.7579	0.6856	0.6938	0.5306	0.4666
Abstract framework	0.4389	0.7047	0.6577	0.6460	0.4796	0.4446
auth1	0.4958	0.7319	0.7347	0.7105	0.5110	0.5121
auth4	0.4790	0.7856	0.6572	0.6880	0.5446	0.4577
auth5	0.4742	0.8075	0.6319	0.6803	0.5613	0.4394
MeSH Now BF	0.4958	0.7440	0.7292	0.7105	0.5190	0.5099
MeSH Now HR	0.2407	0.2328	0.9654	0.3651	0.1504	0.6694
MeSHLabeler-2	0.5085	0.7741	0.7187	0.7203	0.5485	0.5063
MeSHLabeler-3	0.5092	0.7751	0.7177	0.7200	0.5497	0.5068
MeSHLabeler-4	0.5084	0.7726	0.7193	0.7196	0.5473	0.5071
USI baseline	0.4929	0.7790	0.6910	0.7069	0.5489	0.4779
iria-1	0.4104	0.6553	0.6182	0.6064	0.4436	0.4157

+ Test batch 3, week 1

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2681	0.2513	0.3171	0.2652	0.3667	0.4207	0.3501	0.2388	0.3055	0.1582
MTI First Line Index	0.5782	0.6517	0.5397	0.5706	0.6044	0.4851	0.4721	0.6455	0.5237	0.4152
Default MTI	0.5836	0.6018	0.5877	0.5763	0.5608	0.5317	0.4984	0.5975	0.5704	0.4196
it is a test submit	0.3043	0.2721	0.3681	0.3009	0.2760	0.3428	0.2835	0.2721	0.3452	0.1836
USI 10 neighbours	0.5168	0.5596	0.5050	0.5069	0.5000	0.3767	0.3693	0.5453	0.4912	0.3534
USI 20 neighbours	0.5806	0.6449	0.5505	0.5764	0.6041	0.4776	0.4699	0.6472	0.5265	0.4199
Abstract framework	0.5226	0.5683	0.5140	0.5117	0.4987	0.3801	0.3719	0.5454	0.5016	0.3583
MeSHLabeler	0.6324	0.6886	0.6067	0.6255	0.6454	0.5011	0.4984	0.6875	0.5854	0.4701
MeSH Now BF	0.6036	0.6297	0.5959	0.5929	0.6024	0.4687	0.4632	0.6238	0.5846	0.4369
MeSH Now HR	0.2189	0.1260	0.8606	0.2162	0.0969	0.7680	0.3197	0.1255	0.8558	0.1232
MeSHLabeler-1	0.6339	0.6906	0.6070	0.6265	0.6460	0.5032	0.5013	0.6908	0.5857	0.4712
MeSHLabeler-2	0.6334	0.6873	0.6077	0.6259	0.6445	0.5023	0.4999	0.6880	0.5868	0.4704
MeSHLabeler-3	0.6333	0.6919	0.6025	0.6256	0.6540	0.4968	0.4968	0.6935	0.5827	0.4703
MeSHLabeler-4	0.6325	0.6866	0.6073	0.6251	0.6434	0.5043	0.5016	0.6868	0.5862	0.4697
USI baseline	0.5993	0.6770	0.5541	0.5921	0.6487	0.4232	0.4328	0.6822	0.5344	0.4361
iria-1	0.4731	0.5008	0.4715	0.4673	0.4123	0.3085	0.3015	0.5008	0.4484	0.3184
IIIT system 2	0.2709	0.2244	0.4508	0.2829	0.2515	0.4180	0.3127	0.1963	0.4373	0.1697
IIIT system 3	0.4983	0.4404	0.5857	0.4865	0.4424	0.4393	0.3923	0.4397	0.5749	0.3328
IIIT system 4	0.4029	0.4865	0.3681	0.3990	0.5072	0.3189	0.3035	0.4733	0.3507	0.2585
MeSH-UK0	0.6186	0.6397	0.6072	0.6072	0.6391	0.4839	0.4803	0.6485	0.5914	0.4512
MeSH-UK1	0.6164	0.6124	0.6283	0.6056	0.6108	0.5045	0.4916	0.6196	0.6132	0.4489
MeSH-UK2	0.6182	0.6226	0.6216	0.6071	0.6211	0.4984	0.4884	0.6304	0.6065	0.4507
MeSH-UK3	0.6205	0.6675	0.5898	0.6092	0.6624	0.4674	0.4710	0.6754	0.5738	0.4537
MeSH-UK4	0.6092	0.6403	0.5890	0.5971	0.6396	0.4574	0.4587	0.6499	0.5733	0.4409
Dexstr system	0.4206	0.4452	0.4187	0.4148	0.4244	0.3319	0.3108	0.4452	0.3986	0.2710
iria-2	0.4888	0.4281	0.5782	0.4792	0.4018	0.4358	0.3869	0.4322	0.5625	0.3271
iria-3	0.4880	0.4260	0.5793	0.4782	0.3967	0.4301	0.3806	0.4300	0.5642	0.3262
testLee15	0.3396	0.2685	0.4809	0.3332	0.2830	0.4879	0.3659	0.2697	0.4585	0.2058
pseudo n-grams	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
iria-4	0.4170	0.3724	0.5053	0.4117	0.2728	0.4057	0.3410	0.3615	0.4927	0.2699
IMI-KOI	0.2096	0.3784	0.1666	0.2141	0.3973	0.2490	0.2479	0.3363	0.1522	0.1264
Repertoire_0100NF	0.2696	0.3310	0.2963	0.2561	0.1397	0.0183	0.0070	0.2382	0.3106	0.1551
Repertoire_0101NF-Re	0.2074	0.6742	0.1271	0.2052	0.4091	0.0030	0.0032	0.6998	0.1218	0.1215
0100F	0.2848	0.3569	0.2861	0.2650	0.1652	0.0144	0.0060	0.2711	0.2999	0.1620
IMI-KOI_R	0.2177	0.3211	0.1854	0.2207	0.3557	0.2697	0.2626	0.3010	0.1705	0.1306

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3091	0.5204	0.5897	0.5289	0.3145	0.3322
MTI First Line Index	0.4833	0.7948	0.6602	0.6971	0.5565	0.4530
Default MTI	0.4945	0.7550	0.7147	0.7125	0.5283	0.4930
it is a test submit	0.3279	0.5198	0.6142	0.5426	0.3203	0.3601
USI 10 neighbours	0.4453	0.7200	0.6488	0.6553	0.4937	0.4367
USI 20 neighbours	0.4863	0.7848	0.6714	0.7020	0.5523	0.4594
Abstract framework	0.4454	0.7277	0.6459	0.6538	0.4958	0.4390
MeSHLabeler	0.5186	0.8075	0.7100	0.7325	0.5754	0.5000
MeSH Now BF	0.5070	0.7693	0.7224	0.7227	0.5424	0.5068
MeSH Now HR	0.2527	0.2444	0.9646	0.3811	0.1591	0.6703
MeSHLabeler-1	0.5198	0.8090	0.7102	0.7332	0.5770	0.5006
MeSHLabeler-2	0.5198	0.8067	0.7122	0.7334	0.5750	0.5014
MeSHLabeler-3	0.5187	0.8116	0.7054	0.7327	0.5778	0.4974
MeSHLabeler-4	0.5191	0.8070	0.7110	0.7331	0.5745	0.5006
USI baseline	0.4994	0.8051	0.6795	0.7150	0.5730	0.4678
iria-1	0.4161	0.6803	0.6076	0.6161	0.4608	0.4078
IIIT system 2	0.3096	0.4446	0.7327	0.5241	0.2508	0.4614
IIIT system 3	0.4339	0.6370	0.7106	0.6494	0.4143	0.4888
IIIT system 4	0.3737	0.7089	0.5736	0.6058	0.4301	0.3560
MeSH-UK0	0.5106	0.7796	0.7134	0.7253	0.5491	0.5025
MeSH-UK1	0.5125	0.7570	0.7383	0.7296	0.5311	0.5199
MeSH-UK2	0.5124	0.7657	0.7301	0.7291	0.5381	0.5140
MeSH-UK3	0.5080	0.7989	0.6936	0.7214	0.5636	0.4876
MeSH-UK4	0.5002	0.7812	0.6950	0.7149	0.5472	0.4863
Dexstr system	0.3723	0.6340	0.6107	0.6001	0.3832	0.3867
iria-2	0.4324	0.6040	0.7328	0.6443	0.4041	0.4927
iria-3	0.4308	0.6002	0.7359	0.6430	0.4014	0.4938
testLee15	0.3537	0.5044	0.7179	0.5743	0.3139	0.4340
pseudo n-grams	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
iria-4	0.3897	0.5458	0.7090	0.5940	0.3581	0.4631
IMI-KOI	0.2469	0.6440	0.3585	0.4278	0.3855	0.1972
Repertoire_0100NF	0.2780	0.5470	0.3955	0.3993	0.3541	0.2825
Repertoire_0101NF-Re	0.1825	0.8175	0.1100	0.1819	0.6039	0.1121
0100F	0.2775	0.5689	0.3675	0.3911	0.3744	0.2660
IMI-KOI_R	0.2598	0.5938	0.4054	0.4530	0.3575	0.2203

+ Test batch 3, week 2

System Name	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ_Baseline	0.2598	0.2422	0.3075	0.2555	0.3521	0.4120	0.3405	0.2307	0.2975	0.1517
MTI First Line Index	0.5690	0.6415	0.5344	0.5599	0.5792	0.4666	0.4501	0.6298	0.5189	0.4044
Default MTI	0.5841	0.6029	0.5888	0.5756	0.5480	0.5210	0.4848	0.5974	0.5714	0.4197
it is a test submit	0.3105	0.2792	0.3739	0.3077	0.2676	0.3344	0.2780	0.2792	0.3496	0.1884
qaiiit system 1	0.4105	0.3968	0.4322	0.3991	0.3665	0.1510	0.1386	0.3972	0.4248	0.2620
USI 10 neighbours	0.5202	0.5532	0.5177	0.5112	0.4983	0.3786	0.3664	0.5398	0.5019	0.3571
USI 20 neighbours	0.5801	0.6428	0.5508	0.5754	0.5913	0.4647	0.4534	0.6444	0.5274	0.4196
Abstract framework	0.5231	0.5605	0.5236	0.5136	0.4963	0.3803	0.3680	0.5383	0.5087	0.3598
auth1	0.6056	0.6162	0.6090	0.5965	0.6061	0.4712	0.4625	0.6204	0.5915	0.4400
auth2	0.6067	0.6501	0.5815	0.5976	0.6172	0.4632	0.4578	0.6561	0.5642	0.4419
auth3	0.6107	0.6552	0.5846	0.6005	0.6225	0.4574	0.4542	0.6591	0.5688	0.4444
auth4	0.6235	0.6762	0.5934	0.6135	0.6774	0.4410	0.4471	0.6769	0.5779	0.4588
auth5	0.6121	0.6519	0.5901	0.6022	0.6209	0.4614	0.4572	0.6559	0.5738	0.4461
MeSHLabeler	0.6384	0.6721	0.6310	0.6309	0.6283	0.5187	0.5057	0.6673	0.6118	0.4757
MeSH Now BF	0.6022	0.6232	0.6041	0.5934	0.5916	0.4686	0.4581	0.6165	0.5886	0.4369
MeSH Now HR	0.2224	0.1280	0.8668	0.2195	0.0966	0.7669	0.3181	0.1277	0.8601	0.1253
MeSHLabeler-1	0.6396	0.6767	0.6265	0.6322	0.6357	0.5140	0.5040	0.6750	0.6077	0.4774
MeSHLabeler-2	0.6394	0.6746	0.6296	0.6326	0.6368	0.5157	0.5058	0.6721	0.6098	0.4774
MeSHLabeler-3	0.6395	0.6771	0.6265	0.6324	0.6378	0.5143	0.5047	0.6750	0.6076	0.4774
MeSHLabeler-4	0.6399	0.6822	0.6231	0.6326	0.6286	0.5193	0.5102	0.6794	0.6047	0.4779
USI baseline	0.5993	0.6744	0.5599	0.5943	0.6417	0.4220	0.4284	0.6793	0.5362	0.4380
IIIT system 3	0.4942	0.4343	0.5859	0.4827	0.4344	0.4290	0.3772	0.4339	0.5738	0.3290
IIIT system 4	0.3916	0.3332	0.4879	0.3819	0.2942	0.1995	0.1629	0.3305	0.4805	0.2467
MeSH-UK0	0.6196	0.6643	0.5954	0.6104	0.6505	0.4633	0.4617	0.6711	0.5755	0.4555
MeSH-UK1	0.6179	0.6360	0.6128	0.6082	0.6271	0.4805	0.4718	0.6443	0.5936	0.4525
MeSH-UK2	0.6163	0.6172	0.6269	0.6067	0.6069	0.4953	0.4802	0.6246	0.6082	0.4505
MeSH-UK3	0.6091	0.6995	0.5568	0.6003	0.6828	0.4199	0.4293	0.7048	0.5363	0.4461
MeSH-UK4	0.6194	0.6690	0.5921	0.6103	0.6506	0.4633	0.4617	0.6754	0.5720	0.4554
Dexstr system	0.4084	0.4353	0.3997	0.4018	0.4495	0.3403	0.3171	0.4353	0.3847	0.2614
iria-2	0.4932	0.4415	0.5702	0.4843	0.4093	0.4158	0.3757	0.4461	0.5514	0.3317
iria-3	0.4871	0.4227	0.5845	0.4778	0.3885	0.4266	0.3765	0.4269	0.5672	0.3257
testLee15	0.3105	0.2792	0.3739	0.3077	0.2676	0.3344	0.2780	0.2792	0.3496	0.1884
pseudo n-grams	0.3972	0.4062	0.4029	0.3901	0.2926	0.3515	0.2804	0.4115	0.3839	0.2523
iria-4	0.4708	0.5634	0.4261	0.4637	0.4287	0.3229	0.3159	0.5544	0.4091	0.3166
test_unibitri	0.3687	0.3759	0.3994	0.3682	0.2374	0.3411	0.2666	0.3561	0.3821	0.2354
IMI-KOI	0.2387	0.3061	0.2156	0.2417	0.3669	0.3042	0.2835	0.2998	0.1983	0.1432
IMI-KOI_R	0.2190	0.2923	0.1926	0.2203	0.3256	0.2799	0.2669	0.2836	0.1784	0.1296
Dexstr 2	0.4260	0.4540	0.4202	0.4205	0.4519	0.3413	0.3161	0.4540	0.4012	0.2756
iria-mix	0.5744	0.5499	0.6108	0.5646	0.5161	0.5412	0.4919	0.5586	0.5912	0.4073

System Name	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ_Baseline	0.3040	0.5127	0.5840	0.5198	0.3104	0.3280
MTI First Line Index	0.4721	0.7779	0.6496	0.6791	0.5451	0.4463
Default MTI	0.4923	0.7453	0.7088	0.7020	0.5268	0.4922
it is a test submit	0.3338	0.5249	0.6152	0.5461	0.3276	0.3640
qaiiit system 1	0.3735	0.6059	0.5436	0.5468	0.4007	0.3755
USI 10 neighbours	0.4471	0.7089	0.6525	0.6495	0.4882	0.4432
USI 20 neighbours	0.4845	0.7761	0.6659	0.6930	0.5497	0.4585
Abstract framework	0.4454	0.7151	0.6484	0.6468	0.4892	0.4436
auth1	0.5021	0.7492	0.7214	0.7136	0.5285	0.5056
auth2	0.4997	0.7727	0.6950	0.7102	0.5486	0.4846
auth3	0.5017	0.7761	0.6939	0.7103	0.5500	0.4874
auth4	0.5101	0.7947	0.6939	0.7177	0.5658	0.4923
auth5	0.5036	0.7734	0.6994	0.7123	0.5481	0.4923
MeSHLabeler	0.5244	0.7872	0.7258	0.7320	0.5652	0.5188
MeSH Now BF	0.5042	0.7535	0.7221	0.7135	0.5338	0.5094
MeSH Now HR	0.2551	0.2474	0.9662	0.3848	0.1608	0.6711
MeSHLabeler-1	0.5247	0.7897	0.7224	0.7323	0.5675	0.5162
MeSHLabeler-2	0.5250	0.7889	0.7249	0.7332	0.5662	0.5181
MeSHLabeler-3	0.5247	0.7903	0.7221	0.7323	0.5676	0.5162
MeSHLabeler-4	0.5244	0.7948	0.7180	0.7321	0.5713	0.5132
USI baseline	0.4991	0.7931	0.6755	0.7070	0.5671	0.4713
IIIT system 3	0.4335	0.6315	0.7088	0.6448	0.4140	0.4892
IIIT system 4	0.3760	0.5422	0.6301	0.5584	0.3554	0.4302
MeSH-UK0	0.5081	0.7913	0.6946	0.7166	0.5626	0.4901
MeSH-UK1	0.5096	0.7702	0.7142	0.7197	0.5462	0.5045
MeSH-UK2	0.5104	0.7544	0.7304	0.7221	0.5331	0.5160
MeSH-UK3	0.4934	0.8169	0.6504	0.6983	0.5795	0.4561
MeSH-UK4	0.5077	0.7926	0.6940	0.7166	0.5642	0.4884
Dexstr system	0.3583	0.6265	0.5810	0.5798	0.3714	0.3707
iria-2	0.4344	0.6125	0.7172	0.6411	0.4140	0.4845
iria-3	0.4308	0.5944	0.7350	0.6382	0.3997	0.4955
testLee15	0.3338	0.5249	0.6152	0.5461	0.3276	0.3640
pseudo n-grams	0.3605	0.5499	0.5918	0.5480	0.3707	0.3764
iria-4	0.3995	0.7031	0.5541	0.5886	0.4792	0.3704
test_unibitri	0.3512	0.5335	0.5913	0.5344	0.3564	0.3781
IMI-KOI	0.2842	0.5891	0.4455	0.4848	0.3582	0.2516
IMI-KOI_R	0.2629	0.5603	0.4227	0.4575	0.3408	0.2294
Dexstr 2	0.3744	0.6362	0.6104	0.6001	0.3880	0.3869
iria-mix	0.4877	0.6964	0.7450	0.7005	0.4910	0.5112

+ Test batch 3, week 3

BioASQ Participants Area

Test Results for Task 3a

+ Test batch 1, week 1

Annotated articles:11756/21014

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:3215/4435

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:2832/3638

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:1538/2153

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:3945/5725

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:2299/3617

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:2792/4725

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:3136/4861

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:2202/2902

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:2829/4059

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:2901/3902

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:2726/4027

Flat Measures

Hierarchical Measures

+ Test batch 3, week 3

Annotated articles:2095/3162

Flat Measures