BioASQ Participants Area

Test Results for Task 6a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Dry Run 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
deep learning only	0.5614	0.5704	0.5609	0.5439	0.5710	0.3180	0.3336	0.5683	0.5547	0.3881

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
deep learning only	0.4616	0.7414	0.6591	0.6746	0.5056	0.4525

+ Dry Run 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6844	0.7010	0.6873	0.6751	0.6294	0.6275	0.6109	0.6898	0.6791	0.5348
Default MTI	0.6797	0.6454	0.7501	0.6762	0.5706	0.6779	0.6493	0.6334	0.7334	0.5298

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5559	0.8129	0.7777	0.7778	0.5882	0.5573
Default MTI	0.5545	0.7574	0.8298	0.7763	0.5449	0.5927

+ Test batch 1, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6134	0.6654	0.5868	0.6022	0.6204	0.5367	0.5159	0.6553	0.5766	0.4520
Default MTI	0.6231	0.6264	0.6424	0.6151	0.5881	0.5828	0.5414	0.6222	0.6240	0.4640
iria-1	0.4992	0.4331	0.5873	0.4846	0.4053	0.4603	0.4060	0.4402	0.5765	0.3340
iria-2	0.4961	0.4310	0.5830	0.4816	0.3992	0.4556	0.4008	0.4380	0.5721	0.3313
DeepMeSH1	0.6360	0.6652	0.6180	0.6228	0.6369	0.4867	0.4851	0.6698	0.6054	0.4697
DeepMeSH2	0.6565	0.6858	0.6426	0.6459	0.6588	0.5344	0.5274	0.6894	0.6266	0.4954
DeepMeSH3	0.6434	0.6758	0.6255	0.6316	0.6406	0.5040	0.4997	0.6793	0.6111	0.4789
DeepMeSH4	0.6573	0.6888	0.6414	0.6465	0.6592	0.5357	0.5281	0.6917	0.6261	0.4962
xgx	0.5348	0.6013	0.4860	0.5184	0.5415	0.3318	0.3462	0.5997	0.4826	0.3684
AUTH-Atypon 1	0.5593	0.6199	0.5109	0.5363	0.5918	0.3467	0.3534	0.6183	0.5106	0.3843

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4965	0.7902	0.6984	0.7202	0.5463	0.4818
Default MTI	0.5110	0.7567	0.7467	0.7329	0.5281	0.5223
iria-1	0.4304	0.5983	0.7289	0.6404	0.4050	0.4859
iria-2	0.4284	0.5976	0.7257	0.6386	0.4033	0.4833
DeepMeSH1	0.5087	0.7910	0.7159	0.7338	0.5507	0.4959
DeepMeSH2	0.5271	0.8042	0.7344	0.7503	0.5670	0.5159
DeepMeSH3	0.5153	0.7983	0.7215	0.7400	0.5583	0.5022
DeepMeSH4	0.5276	0.8064	0.7326	0.7500	0.5690	0.5159
xgx	0.4403	0.7577	0.6072	0.6519	0.5115	0.4095
AUTH-Atypon 1	0.4504	0.7883	0.6119	0.6632	0.5264	0.4197

+ Test batch 1, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6219	0.6734	0.6013	0.6157	0.6317	0.5533	0.5312	0.6642	0.5847	0.4641
Default MTI	0.6339	0.6422	0.6520	0.6293	0.6018	0.5957	0.5541	0.6366	0.6313	0.4767
iria-1	0.5223	0.4682	0.6015	0.5136	0.4445	0.4743	0.4252	0.4721	0.5845	0.3584
iria-2	0.5100	0.4585	0.5863	0.5015	0.4174	0.4592	0.4091	0.4614	0.5700	0.3477
DeepMeSH1	0.6456	0.6819	0.6292	0.6384	0.6458	0.4871	0.4865	0.6824	0.6126	0.4837
DeepMeSH2	0.6651	0.7026	0.6509	0.6597	0.6684	0.5380	0.5331	0.7017	0.6321	0.5079
DeepMeSH3	0.6515	0.6926	0.6333	0.6451	0.6536	0.5043	0.5017	0.6917	0.6158	0.4911
DeepMeSH4	0.6664	0.7050	0.6509	0.6607	0.6706	0.5414	0.5358	0.7039	0.6327	0.5091
xgx	0.5697	0.6755	0.5064	0.5611	0.6061	0.3504	0.3717	0.6685	0.4963	0.4062
xgx0	0.5683	0.6841	0.5004	0.5601	0.6303	0.3425	0.3663	0.6766	0.4898	0.4051

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5059	0.7913	0.7073	0.7276	0.5535	0.4918
Default MTI	0.5193	0.7639	0.7533	0.7410	0.5375	0.5290
iria-1	0.4459	0.6271	0.7308	0.6597	0.4267	0.4918
iria-2	0.4376	0.6169	0.7201	0.6489	0.4187	0.4835
DeepMeSH1	0.5176	0.8007	0.7177	0.7401	0.5639	0.5009
DeepMeSH2	0.5356	0.8135	0.7356	0.7562	0.5796	0.5208
DeepMeSH3	0.5226	0.8082	0.7211	0.7449	0.5708	0.5048
DeepMeSH4	0.5358	0.8153	0.7351	0.7567	0.5808	0.5206
xgx	0.4603	0.8124	0.6096	0.6762	0.5564	0.4142
xgx0	0.4584	0.8217	0.5969	0.6709	0.5627	0.4081

+ Test batch 1, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6220	0.6810	0.5938	0.6155	0.6506	0.5452	0.5293	0.6709	0.5797	0.4627
Default MTI	0.6361	0.6503	0.6454	0.6308	0.6196	0.5888	0.5530	0.6442	0.6281	0.4777
iria-1	0.5198	0.4671	0.5970	0.5118	0.4482	0.4688	0.4211	0.4702	0.5811	0.3566
iria-2	0.5257	0.4719	0.6034	0.5174	0.4537	0.4692	0.4228	0.4753	0.5880	0.3615
DeepMeSH1	0.6463	0.6888	0.6236	0.6388	0.6592	0.4757	0.4809	0.6893	0.6082	0.4843
DeepMeSH2	0.6662	0.7092	0.6455	0.6603	0.6843	0.5314	0.5314	0.7083	0.6288	0.5084
DeepMeSH3	0.6535	0.6971	0.6319	0.6470	0.6623	0.5024	0.5030	0.6965	0.6155	0.4932
DeepMeSH4	0.6678	0.7123	0.6461	0.6621	0.6843	0.5330	0.5330	0.7117	0.6290	0.5105
xgx	0.5678	0.6762	0.5045	0.5596	0.6040	0.3458	0.3706	0.6668	0.4944	0.4044
xgx1	0.5601	0.6666	0.4955	0.5518	0.6149	0.3324	0.3613	0.6635	0.4847	0.3976
xgx0	0.5734	0.6646	0.5202	0.5651	0.6061	0.3552	0.3778	0.6557	0.5094	0.4098
AUTH-Atypon 1	0.5739	0.6600	0.5217	0.5634	0.6035	0.3440	0.3565	0.6540	0.5113	0.4076

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5032	0.7972	0.7009	0.7275	0.5558	0.4850
Default MTI	0.5179	0.7693	0.7471	0.7412	0.5399	0.5238
iria-1	0.4435	0.6229	0.7260	0.6558	0.4247	0.4882
iria-2	0.4478	0.6280	0.7326	0.6617	0.4285	0.4931
DeepMeSH1	0.5170	0.8056	0.7164	0.7421	0.5647	0.4995
DeepMeSH2	0.5350	0.8176	0.7341	0.7577	0.5806	0.5188
DeepMeSH3	0.5239	0.8098	0.7235	0.7480	0.5705	0.5072
DeepMeSH4	0.5357	0.8185	0.7342	0.7583	0.5815	0.5193
xgx	0.4588	0.8111	0.6076	0.6737	0.5544	0.4132
xgx1	0.4540	0.8034	0.5988	0.6660	0.5500	0.4076
xgx0	0.4656	0.8048	0.6224	0.6800	0.5529	0.4255
AUTH-Atypon 1	0.4654	0.8078	0.6218	0.6812	0.5516	0.4255

+ Test batch 1, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6212	0.6681	0.5981	0.6114	0.6184	0.5483	0.5244	0.6598	0.5869	0.4589
Default MTI	0.6328	0.6381	0.6505	0.6258	0.5854	0.5926	0.5475	0.6316	0.6340	0.4723
iria-1	0.5240	0.4632	0.6091	0.5132	0.4409	0.4840	0.4290	0.4678	0.5956	0.3579
iria-3	0.4913	0.4334	0.5691	0.4799	0.4013	0.4191	0.3723	0.4387	0.5583	0.3284
iria-4	0.2692	0.2593	0.3094	0.2678	0.4109	0.3933	0.3334	0.2469	0.2960	0.1609
DeepMeSH1	0.6454	0.6820	0.6245	0.6353	0.6357	0.4871	0.4820	0.6845	0.6105	0.4815
DeepMeSH2	0.6647	0.7000	0.6468	0.6559	0.6603	0.5392	0.5303	0.7017	0.6313	0.5045
DeepMeSH3	0.6522	0.6884	0.6321	0.6426	0.6411	0.5117	0.5030	0.6913	0.6173	0.4894
DeepMeSH4	0.6664	0.7030	0.6478	0.6580	0.6630	0.5428	0.5337	0.7050	0.6318	0.5068
xgx3	0.6003	0.7268	0.5188	0.5838	0.6311	0.3829	0.4011	0.7244	0.5125	0.4312
AUTH-Atypon 1	0.5755	0.6506	0.5291	0.5623	0.5961	0.3638	0.3667	0.6464	0.5186	0.4074

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5028	0.7878	0.7017	0.7221	0.5495	0.4894
Default MTI	0.5170	0.7586	0.7501	0.7358	0.5334	0.5292
iria-1	0.4478	0.6228	0.7393	0.6609	0.4246	0.4983
iria-3	0.4276	0.5970	0.7073	0.6325	0.4063	0.4752
iria-4	0.2941	0.4864	0.5535	0.4972	0.2819	0.3386
DeepMeSH1	0.5159	0.8012	0.7138	0.7376	0.5616	0.5006
DeepMeSH2	0.5343	0.8121	0.7319	0.7530	0.5767	0.5211
DeepMeSH3	0.5230	0.8051	0.7210	0.7436	0.5673	0.5082
DeepMeSH4	0.5358	0.8131	0.7326	0.7538	0.5791	0.5220
xgx3	0.4719	0.8373	0.6126	0.6845	0.5747	0.4233
AUTH-Atypon 1	0.4685	0.7968	0.6290	0.6798	0.5471	0.4343

+ Test batch 1, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6277	0.6820	0.6024	0.6195	0.6368	0.5402	0.5228	0.6731	0.5881	0.4670
Default MTI	0.6383	0.6515	0.6498	0.6320	0.6064	0.5829	0.5476	0.6456	0.6311	0.4791
Search system 1	0.2311	0.1854	0.3376	0.2286	0.3927	0.4482	0.3640	0.1795	0.3246	0.1334
DeepMeSH1	0.6461	0.6896	0.6241	0.6377	0.6398	0.4672	0.4687	0.6919	0.6061	0.4837
DeepMeSH2	0.6534	0.6970	0.6323	0.6455	0.6466	0.4897	0.4884	0.6992	0.6131	0.4919
DeepMeSH3	0.6534	0.6970	0.6323	0.6455	0.6466	0.4897	0.4884	0.6992	0.6131	0.4919
xgx	0.6193	0.6546	0.6039	0.6069	0.5851	0.4514	0.4508	0.6477	0.5933	0.4515
xgx2	0.6364	0.6897	0.6081	0.6256	0.6385	0.4532	0.4588	0.6832	0.5955	0.4713
xgx3	0.6383	0.6946	0.6062	0.6266	0.6444	0.4442	0.4503	0.6881	0.5953	0.4723
xgx1	0.6342	0.6785	0.6132	0.6235	0.6156	0.4628	0.4633	0.6712	0.6011	0.4686
xgx0	0.6293	0.6583	0.6199	0.6180	0.5885	0.4705	0.4646	0.6512	0.6089	0.4626
TDM KE 1	0.2311	0.1854	0.3376	0.2286	0.3927	0.4482	0.3640	0.1795	0.3246	0.1334
TDM KE 2	0.2312	0.1853	0.3382	0.2287	0.3913	0.4499	0.3654	0.1794	0.3251	0.1335
TDM KE 3	0.1569	0.2192	0.1337	0.1528	0.3583	0.1871	0.1948	0.2169	0.1229	0.0878
TDM KE 4	0.2089	0.1582	0.3534	0.2093	0.2742	0.4764	0.3618	0.1503	0.3422	0.1202
TDM KE 5	0.2036	0.1528	0.3525	0.2043	0.2665	0.4773	0.3604	0.1451	0.3412	0.1169
AttentionMeSH	0.5817	0.5575	0.6185	0.5672	0.4277	0.4204	0.3851	0.5549	0.6112	0.4103

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5066	0.7954	0.7102	0.7307	0.5566	0.4912
Default MTI	0.5211	0.7688	0.7549	0.7434	0.5417	0.5296
Search system 1	0.2597	0.3870	0.5949	0.4527	0.2114	0.3739
DeepMeSH1	0.5151	0.8002	0.7152	0.7374	0.5638	0.4985
DeepMeSH2	0.5219	0.8047	0.7234	0.7440	0.5693	0.5070
DeepMeSH3	0.5219	0.8047	0.7234	0.7440	0.5693	0.5070
xgx	0.4999	0.7816	0.7049	0.7196	0.5457	0.4890
xgx2	0.5110	0.8096	0.6986	0.7292	0.5699	0.4895
xgx3	0.5099	0.8125	0.6980	0.7302	0.5691	0.4881
xgx1	0.5116	0.7983	0.7115	0.7317	0.5614	0.4968
xgx0	0.5084	0.7827	0.7171	0.7280	0.5487	0.5007
TDM KE 1	0.2597	0.3870	0.5949	0.4527	0.2114	0.3739
TDM KE 2	0.2595	0.3866	0.5956	0.4527	0.2110	0.3742
TDM KE 3	0.2279	0.4858	0.3303	0.3677	0.2961	0.2071
TDM KE 4	0.2408	0.3407	0.6346	0.4286	0.1824	0.3998
TDM KE 5	0.2364	0.3319	0.6360	0.4217	0.1772	0.4012
AttentionMeSH	0.4713	0.6989	0.7243	0.6914	0.4757	0.4970

+ Test batch 2, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6258	0.6668	0.6098	0.6161	0.6108	0.5549	0.5297	0.6577	0.5968	0.4632
Default MTI	0.6361	0.6354	0.6591	0.6284	0.5811	0.5995	0.5540	0.6297	0.6426	0.4750
iria-1	0.5251	0.4646	0.6127	0.5145	0.4435	0.4816	0.4297	0.4689	0.5967	0.3592
DeepMeSH1	0.6475	0.6838	0.6308	0.6386	0.6318	0.4842	0.4819	0.6863	0.6129	0.4843
DeepMeSH2	0.6675	0.7024	0.6543	0.6601	0.6556	0.5385	0.5318	0.7031	0.6353	0.5085
DeepMeSH3	0.6540	0.6921	0.6373	0.6458	0.6382	0.5049	0.5008	0.6939	0.6185	0.4921
DeepMeSH4	0.6682	0.7055	0.6537	0.6610	0.6556	0.5381	0.5320	0.7057	0.6344	0.5094
xgx	0.6275	0.6696	0.6051	0.6152	0.6275	0.4418	0.4456	0.6654	0.5938	0.4599
Semantic NoSQL KE 1	0.2326	0.1839	0.3450	0.2296	0.3885	0.4518	0.3658	0.1782	0.3345	0.1339
Semantic NoSQL KE 2	0.2328	0.1840	0.3459	0.2298	0.3878	0.4544	0.3681	0.1783	0.3353	0.1341
Semantic NoSQL KE 3	0.1710	0.2229	0.1509	0.1661	0.3558	0.2144	0.2175	0.2205	0.1397	0.0959
Semantic NoSQL KE 4	0.2094	0.1575	0.3604	0.2100	0.2706	0.4839	0.3659	0.1490	0.3519	0.1206
Semantic NoSQL KE 5	0.2043	0.1522	0.3601	0.2051	0.2647	0.4872	0.3658	0.1439	0.3518	0.1174
PLM + VP	0.3552	0.3892	0.3295	0.3441	0.4438	0.0746	0.0741	0.3882	0.3274	0.2191

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5056	0.7814	0.7161	0.7256	0.5483	0.4979
Default MTI	0.5187	0.7527	0.7613	0.7378	0.5309	0.5358
iria-1	0.4489	0.6189	0.7423	0.6584	0.4257	0.5019
DeepMeSH1	0.5168	0.7965	0.7196	0.7375	0.5611	0.5042
DeepMeSH2	0.5353	0.8073	0.7395	0.7533	0.5758	0.5253
DeepMeSH3	0.5229	0.8024	0.7266	0.7439	0.5671	0.5101
DeepMeSH4	0.5358	0.8098	0.7394	0.7545	0.5776	0.5247
xgx	0.5053	0.8010	0.6933	0.7216	0.5596	0.4868
Semantic NoSQL KE 1	0.2591	0.3804	0.6003	0.4496	0.2092	0.3797
Semantic NoSQL KE 2	0.2589	0.3800	0.6009	0.4496	0.2088	0.3800
Semantic NoSQL KE 3	0.2361	0.4845	0.3552	0.3830	0.2923	0.2223
Semantic NoSQL KE 4	0.2406	0.3350	0.6386	0.4246	0.1816	0.4048
Semantic NoSQL KE 5	0.2365	0.3260	0.6406	0.4176	0.1766	0.4070
PLM + VP	0.3115	0.5884	0.3899	0.4462	0.3985	0.2744

+ Test batch 2, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6296	0.6745	0.6053	0.6177	0.6312	0.5511	0.5288	0.6684	0.5949	0.4653
Default MTI	0.6454	0.6524	0.6550	0.6353	0.6054	0.5973	0.5573	0.6493	0.6416	0.4833
iria-1	0.5346	0.4756	0.6146	0.5225	0.4501	0.4849	0.4318	0.4812	0.6012	0.3669
DeepMeSH1	0.6527	0.6904	0.6295	0.6412	0.6458	0.4844	0.4832	0.6952	0.6151	0.4883
DeepMeSH2	0.6732	0.7085	0.6530	0.6627	0.6713	0.5383	0.5327	0.7125	0.6380	0.5126
DeepMeSH3	0.6593	0.6979	0.6373	0.6488	0.6476	0.5074	0.5033	0.7018	0.6218	0.4963
DeepMeSH4	0.6751	0.7128	0.6534	0.6649	0.6755	0.5415	0.5360	0.7167	0.6380	0.5150
xgx	0.6414	0.6804	0.6146	0.6279	0.6325	0.4616	0.4651	0.6823	0.6051	0.4735
xgx2	0.6211	0.6906	0.5718	0.6072	0.6430	0.3783	0.3924	0.6961	0.5606	0.4524
xgx3	0.5925	0.6844	0.5284	0.5777	0.6380	0.2894	0.3095	0.6929	0.5176	0.4229
xgx1	0.6343	0.6872	0.5967	0.6208	0.6443	0.4178	0.4279	0.6911	0.5862	0.4666
xgx0	0.6376	0.6804	0.6088	0.6244	0.6326	0.4418	0.4473	0.6831	0.5978	0.4700
AttentionMeSH	0.6195	0.6657	0.5821	0.6006	0.5881	0.3751	0.3845	0.6650	0.5799	0.4471
Semantic NoSQL KE 1	0.2286	0.1826	0.3355	0.2257	0.3944	0.4533	0.3637	0.1761	0.3258	0.1314
Semantic NoSQL KE 2	0.2286	0.1825	0.3360	0.2257	0.3925	0.4542	0.3644	0.1759	0.3262	0.1314
Semantic NoSQL KE 3	0.1681	0.2218	0.1475	0.1632	0.3631	0.2186	0.2205	0.2199	0.1361	0.0941
Semantic NoSQL KE 4	0.2069	0.1567	0.3542	0.2076	0.2743	0.4860	0.3630	0.1476	0.3456	0.1191
Semantic NoSQL KE 5	0.2018	0.1515	0.3537	0.2028	0.2672	0.4875	0.3624	0.1425	0.3452	0.1160
PLM + VP	0.0979	0.4425	0.0636	0.1036	0.4712	0.0687	0.0693	0.3536	0.0568	0.0580
AttentionMeSH2	0.5656	0.5492	0.5889	0.5498	0.3783	0.4207	0.3737	0.5467	0.5859	0.3952
AttentionMeSH3	0.5976	0.6184	0.5810	0.5794	0.4995	0.3909	0.3806	0.6177	0.5788	0.4247
AttentionMeSH4	0.6022	0.6334	0.5763	0.5832	0.5543	0.3756	0.3734	0.6324	0.5747	0.4288
AttentionMeSH5	0.6030	0.6391	0.5731	0.5836	0.5817	0.3644	0.3658	0.6379	0.5717	0.4294
MeSHmallow-1	0.1027	0.4829	0.0661	0.1083	0.4476	0.0690	0.0704	0.3788	0.0594	0.0606
MeSHmallow-2	0.0979	0.4425	0.0636	0.1036	0.4712	0.0687	0.0693	0.3536	0.0568	0.0580

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5074	0.7913	0.7106	0.7289	0.5551	0.4939
Default MTI	0.5248	0.7689	0.7551	0.7442	0.5457	0.5321
iria-1	0.4535	0.6304	0.7417	0.6659	0.4326	0.5021
DeepMeSH1	0.5202	0.8034	0.7171	0.7404	0.5682	0.5034
DeepMeSH2	0.5385	0.8147	0.7359	0.7568	0.5831	0.5238
DeepMeSH3	0.5268	0.8075	0.7247	0.7467	0.5731	0.5109
DeepMeSH4	0.5399	0.8171	0.7372	0.7586	0.5855	0.5243
xgx	0.5152	0.8054	0.7056	0.7341	0.5670	0.4964
xgx2	0.4949	0.8158	0.6645	0.7130	0.5713	0.4600
xgx3	0.4678	0.8171	0.6171	0.6823	0.5648	0.4224
xgx1	0.5072	0.8125	0.6871	0.7260	0.5695	0.4809
xgx0	0.5124	0.8053	0.7005	0.7308	0.5664	0.4919
AttentionMeSH	0.4866	0.7914	0.6631	0.6999	0.5481	0.4624
Semantic NoSQL KE 1	0.2585	0.3846	0.5947	0.4511	0.2097	0.3756
Semantic NoSQL KE 2	0.2584	0.3843	0.5952	0.4510	0.2095	0.3759
Semantic NoSQL KE 3	0.2341	0.4855	0.3502	0.3808	0.2924	0.2190
Semantic NoSQL KE 4	0.2396	0.3387	0.6354	0.4270	0.1814	0.4012
Semantic NoSQL KE 5	0.2354	0.3298	0.6370	0.4199	0.1763	0.4029
PLM + VP	0.1350	0.6791	0.1238	0.1948	0.3722	0.0880
AttentionMeSH2	0.4623	0.6921	0.6963	0.6749	0.4732	0.4788
AttentionMeSH3	0.4769	0.7506	0.6766	0.6909	0.5182	0.4673
AttentionMeSH4	0.4782	0.7629	0.6692	0.6915	0.5277	0.4628
AttentionMeSH5	0.4777	0.7676	0.6649	0.6907	0.5309	0.4598
MeSHmallow-1	0.1296	0.6934	0.1221	0.1929	0.3643	0.0843
MeSHmallow-2	0.1350	0.6791	0.1238	0.1948	0.3722	0.0880

+ Test batch 2, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6268	0.6787	0.6044	0.6191	0.6359	0.5595	0.5360	0.6678	0.5907	0.4661
Default MTI	0.6420	0.6534	0.6579	0.6374	0.6072	0.6043	0.5615	0.6447	0.6394	0.4843
iria-1	0.5351	0.4798	0.6127	0.5262	0.4629	0.4863	0.4350	0.4840	0.5983	0.3697
DeepMeSH1	0.6518	0.6970	0.6261	0.6437	0.6525	0.4889	0.4883	0.6983	0.6110	0.4903
DeepMeSH2	0.6719	0.7174	0.6492	0.6656	0.6755	0.5431	0.5376	0.7166	0.6325	0.5148
DeepMeSH3	0.6588	0.7063	0.6332	0.6518	0.6595	0.5129	0.5099	0.7062	0.6175	0.4988
DeepMeSH4	0.6727	0.7203	0.6484	0.6664	0.6782	0.5458	0.5415	0.7195	0.6315	0.5156
xgx	0.6380	0.6898	0.6046	0.6257	0.6462	0.4603	0.4656	0.6851	0.5970	0.4710
xgx1	0.6153	0.7027	0.5574	0.6022	0.6546	0.3713	0.3892	0.7003	0.5487	0.4477
xgx0	0.6344	0.6895	0.5988	0.6221	0.6458	0.4416	0.4496	0.6860	0.5899	0.4675
AttentionMeSH	0.6099	0.6493	0.5815	0.5955	0.5479	0.3841	0.3810	0.6468	0.5771	0.4405
Semantic NoSQL KE 1	0.2292	0.1822	0.3330	0.2261	0.4000	0.4589	0.3704	0.1775	0.3235	0.1315
Semantic NoSQL KE 2	0.2294	0.1823	0.3335	0.2263	0.3985	0.4604	0.3718	0.1775	0.3241	0.1316
Semantic NoSQL KE 3	0.1692	0.2200	0.1484	0.1640	0.3648	0.2182	0.2214	0.2195	0.1377	0.0946
Semantic NoSQL KE 4	0.2090	0.1574	0.3525	0.2094	0.2811	0.4917	0.3700	0.1501	0.3438	0.1200
Semantic NoSQL KE 5	0.2037	0.1522	0.3513	0.2044	0.2750	0.4934	0.3696	0.1449	0.3425	0.1168
PLM + VP	0.3451	0.3828	0.3121	0.3336	0.4609	0.0738	0.0733	0.3812	0.3151	0.2113
AttentionMeSH2	0.6134	0.6586	0.5807	0.5989	0.5819	0.3804	0.3798	0.6554	0.5764	0.4440
AttentionMeSH3	0.6138	0.6601	0.5803	0.5992	0.5900	0.3780	0.3779	0.6568	0.5760	0.4444
AttentionMeSH4	0.6139	0.6607	0.5800	0.5993	0.5938	0.3767	0.3767	0.6574	0.5758	0.4446
AttentionMeSH5	0.6139	0.6609	0.5798	0.5993	0.5956	0.3755	0.3758	0.6576	0.5756	0.4445
MeSHmallow-1	0.1011	0.4636	0.0634	0.1049	0.4786	0.0687	0.0694	0.3872	0.0582	0.0585
MeSHmallow-2	0.0307	0.2809	0.4618	0.2905	0.0117	0.1640	0.0275	0.0159	0.4711	0.1826
MeSHmallow-3	0.0293	0.2755	0.4664	0.2890	0.0109	0.1665	0.0263	0.0151	0.4758	0.1814

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5071	0.7948	0.7044	0.7266	0.5562	0.4922
Default MTI	0.5238	0.7707	0.7520	0.7431	0.5442	0.5317
iria-1	0.4546	0.6353	0.7369	0.6679	0.4352	0.4991
DeepMeSH1	0.5193	0.8081	0.7140	0.7408	0.5685	0.5001
DeepMeSH2	0.5372	0.8196	0.7322	0.7564	0.5841	0.5198
DeepMeSH3	0.5261	0.8133	0.7206	0.7473	0.5749	0.5070
DeepMeSH4	0.5377	0.8215	0.7314	0.7570	0.5857	0.5193
xgx	0.5109	0.8121	0.6945	0.7293	0.5691	0.4875
xgx1	0.4888	0.8240	0.6491	0.7057	0.5746	0.4477
xgx0	0.5084	0.8111	0.6901	0.7264	0.5689	0.4830
AttentionMeSH	0.4828	0.7728	0.6727	0.6995	0.5318	0.4657
Semantic NoSQL KE 1	0.2612	0.3826	0.5860	0.4479	0.2127	0.3738
Semantic NoSQL KE 2	0.2610	0.3823	0.5867	0.4479	0.2123	0.3740
Semantic NoSQL KE 3	0.2371	0.4861	0.3463	0.3781	0.2980	0.2211
Semantic NoSQL KE 4	0.2433	0.3399	0.6299	0.4277	0.1846	0.4010
Semantic NoSQL KE 5	0.2392	0.3313	0.6311	0.4209	0.1797	0.4021
PLM + VP	0.3001	0.5945	0.3701	0.4360	0.4024	0.2541
AttentionMeSH2	0.4841	0.7803	0.6703	0.7011	0.5374	0.4641
AttentionMeSH3	0.4842	0.7817	0.6697	0.7012	0.5384	0.4636
AttentionMeSH4	0.4842	0.7822	0.6693	0.7012	0.5388	0.4633
AttentionMeSH5	0.4842	0.7825	0.6690	0.7011	0.5391	0.4630
MeSHmallow-1	0.1427	0.7223	0.1194	0.1913	0.4300	0.0906
MeSHmallow-2	0.2900	0.4554	0.5580	0.4217	0.3088	0.4088
MeSHmallow-3	0.2901	0.4489	0.5643	0.4217	0.3047	0.4133

+ Test batch 2, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6098	0.6642	0.5876	0.6023	0.6220	0.5402	0.5189	0.6554	0.5702	0.4485
Default MTI	0.6262	0.6412	0.6391	0.6203	0.5922	0.5829	0.5433	0.6345	0.6181	0.4666
iria-1	0.5222	0.4674	0.6022	0.5121	0.4538	0.4796	0.4286	0.4719	0.5846	0.3567
DeepMeSH1	0.6376	0.6844	0.6145	0.6296	0.6454	0.4718	0.4712	0.6860	0.5956	0.4746
DeepMeSH2	0.6556	0.7018	0.6348	0.6488	0.6642	0.5217	0.5174	0.7027	0.6143	0.4959
DeepMeSH3	0.6443	0.6920	0.6204	0.6362	0.6489	0.4924	0.4894	0.6935	0.6016	0.4817
DeepMeSH4	0.6569	0.7041	0.6352	0.6503	0.6682	0.5236	0.5191	0.7055	0.6145	0.4976
xgx	0.6285	0.6630	0.6112	0.6164	0.6122	0.4577	0.4585	0.6602	0.5998	0.4612
xgx1	0.6192	0.6673	0.5912	0.6069	0.6174	0.4156	0.4230	0.6658	0.5787	0.4512
xgx0	0.6238	0.6618	0.6039	0.6117	0.6125	0.4421	0.4440	0.6603	0.5911	0.4561
AttentionMeSH	0.6069	0.6446	0.5811	0.5917	0.5702	0.3879	0.3868	0.6431	0.5746	0.4361
Semantic NoSQL KE 1	0.2276	0.1796	0.3416	0.2250	0.3894	0.4493	0.3613	0.1747	0.3267	0.1307
Semantic NoSQL KE 2	0.2276	0.1795	0.3420	0.2250	0.3873	0.4505	0.3621	0.1745	0.3271	0.1307
Semantic NoSQL KE 3	0.1666	0.2219	0.1456	0.1621	0.3535	0.2074	0.2108	0.2193	0.1343	0.0934
Semantic NoSQL KE 4	0.2061	0.1540	0.3587	0.2064	0.2703	0.4790	0.3600	0.1468	0.3459	0.1181
Semantic NoSQL KE 5	0.2012	0.1492	0.3577	0.2018	0.2632	0.4818	0.3598	0.1420	0.3451	0.1151
AttentionMeSH2	0.6078	0.6476	0.5807	0.5926	0.5808	0.3859	0.3856	0.6456	0.5742	0.4371
AttentionMeSH3	0.6081	0.6485	0.5805	0.5929	0.5852	0.3854	0.3852	0.6463	0.5741	0.4374
AttentionMeSH4	0.6082	0.6488	0.5805	0.5930	0.5872	0.3850	0.3848	0.6466	0.5741	0.4375
AttentionMeSH5	0.6083	0.6489	0.5804	0.5931	0.5885	0.3848	0.3847	0.6468	0.5740	0.4375
Access Inn MAIstro	0.2527	0.2304	0.3202	0.2525	0.3663	0.4028	0.3341	0.2172	0.3021	0.1496

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4924	0.7870	0.6916	0.7152	0.5438	0.4775
Default MTI	0.5086	0.7640	0.7380	0.7313	0.5326	0.5154
iria-1	0.4409	0.6253	0.7276	0.6563	0.4239	0.4858
DeepMeSH1	0.5078	0.8034	0.7032	0.7323	0.5600	0.4885
DeepMeSH2	0.5211	0.8133	0.7187	0.7454	0.5710	0.5037
DeepMeSH3	0.5124	0.8070	0.7081	0.7366	0.5647	0.4930
DeepMeSH4	0.5233	0.8147	0.7194	0.7465	0.5732	0.5057
xgx	0.5038	0.7946	0.7019	0.7261	0.5522	0.4893
xgx1	0.4948	0.8007	0.6827	0.7171	0.5538	0.4728
xgx0	0.5008	0.7943	0.6965	0.7226	0.5520	0.4841
AttentionMeSH	0.4777	0.7720	0.6706	0.6982	0.5267	0.4620
Semantic NoSQL KE 1	0.2583	0.3792	0.5971	0.4473	0.2093	0.3770
Semantic NoSQL KE 2	0.2581	0.3786	0.5977	0.4471	0.2089	0.3773
Semantic NoSQL KE 3	0.2346	0.4848	0.3449	0.3761	0.2960	0.2188
Semantic NoSQL KE 4	0.2410	0.3339	0.6386	0.4237	0.1815	0.4056
Semantic NoSQL KE 5	0.2373	0.3258	0.6402	0.4172	0.1769	0.4076
AttentionMeSH2	0.4780	0.7746	0.6699	0.6986	0.5286	0.4613
AttentionMeSH3	0.4781	0.7753	0.6696	0.6988	0.5291	0.4611
AttentionMeSH4	0.4782	0.7755	0.6695	0.6988	0.5293	0.4610
AttentionMeSH5	0.4782	0.7756	0.6695	0.6988	0.5294	0.4610
Access Inn MAIstro	0.2863	0.4553	0.5653	0.4825	0.2659	0.3472

+ Test batch 2, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6307	0.6830	0.6003	0.6192	0.6420	0.5647	0.5440	0.6761	0.5910	0.4687
Default MTI	0.6450	0.6538	0.6537	0.6360	0.6095	0.6067	0.5668	0.6491	0.6409	0.4850
iria-1	0.5297	0.4715	0.6102	0.5192	0.4618	0.4922	0.4411	0.4759	0.5974	0.3643
DeepMeSH1	0.6463	0.6889	0.6198	0.6362	0.6565	0.4861	0.4867	0.6930	0.6055	0.4833
DeepMeSH2	0.6702	0.7128	0.6453	0.6613	0.6824	0.5436	0.5401	0.7155	0.6302	0.5115
DeepMeSH3	0.6549	0.6995	0.6281	0.6454	0.6613	0.5083	0.5073	0.7027	0.6131	0.4930
DeepMeSH4	0.6719	0.7174	0.6449	0.6629	0.6842	0.5438	0.5419	0.7200	0.6299	0.5132
xgx	0.6335	0.6680	0.6120	0.6203	0.6217	0.4604	0.4622	0.6656	0.6045	0.4665
xgx0	0.6293	0.6684	0.6049	0.6163	0.6257	0.4418	0.4464	0.6665	0.5961	0.4623
AttentionMeSH	0.6295	0.6660	0.6011	0.6138	0.5907	0.4142	0.4126	0.6645	0.5980	0.4610
AttentionMeSH2	0.6305	0.6685	0.6010	0.6148	0.6010	0.4133	0.4122	0.6668	0.5979	0.4620
AttentionMeSH3	0.6307	0.6691	0.6009	0.6150	0.6038	0.4128	0.4117	0.6674	0.5978	0.4623
AttentionMeSH4	0.6308	0.6695	0.6008	0.6151	0.6054	0.4123	0.4113	0.6676	0.5978	0.4624
AttentionMeSH5	0.6309	0.6699	0.6008	0.6152	0.6070	0.4122	0.4112	0.6679	0.5977	0.4625

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5069	0.7970	0.7026	0.7275	0.5587	0.4893
Default MTI	0.5237	0.7706	0.7520	0.7438	0.5442	0.5308
iria-1	0.4504	0.6284	0.7364	0.6633	0.4297	0.4979
DeepMeSH1	0.5159	0.8040	0.7088	0.7366	0.5646	0.4976
DeepMeSH2	0.5361	0.8183	0.7293	0.7551	0.5831	0.5187
DeepMeSH3	0.5234	0.8108	0.7160	0.7437	0.5724	0.5049
DeepMeSH4	0.5362	0.8221	0.7278	0.7558	0.5849	0.5178
xgx	0.5109	0.7961	0.7040	0.7286	0.5584	0.4952
xgx0	0.5071	0.7976	0.6983	0.7258	0.5582	0.4891
AttentionMeSH	0.4970	0.7824	0.6909	0.7151	0.5426	0.4824
AttentionMeSH2	0.4974	0.7843	0.6904	0.7156	0.5439	0.4821
AttentionMeSH3	0.4975	0.7848	0.6901	0.7157	0.5444	0.4820
AttentionMeSH4	0.4976	0.7852	0.6901	0.7157	0.5446	0.4820
AttentionMeSH5	0.4976	0.7855	0.6900	0.7158	0.5447	0.4819

+ Test batch 3, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6235	0.6806	0.5946	0.6135	0.6447	0.5557	0.5378	0.6713	0.5821	0.4620
Default MTI	0.6391	0.6538	0.6486	0.6322	0.6163	0.5997	0.5631	0.6463	0.6320	0.4807
iria-1	0.5223	0.4657	0.6046	0.5129	0.4562	0.4835	0.4338	0.4697	0.5882	0.3578
xgx	0.6212	0.6540	0.6032	0.6083	0.6044	0.4779	0.4745	0.6506	0.5942	0.4528
xgx1	0.6211	0.6425	0.6133	0.6084	0.6057	0.4596	0.4571	0.6400	0.6032	0.4525
xgx0	0.6241	0.6607	0.6036	0.6111	0.6223	0.4636	0.4653	0.6569	0.5944	0.4554
AttentionMeSH	0.6291	0.6740	0.5993	0.6145	0.6167	0.4216	0.4254	0.6698	0.5931	0.4606
Semantic NoSQL KE 1	0.2363	0.1871	0.3465	0.2331	0.4035	0.4490	0.3654	0.1816	0.3383	0.1361
Semantic NoSQL KE 2	0.2363	0.1870	0.3470	0.2331	0.4017	0.4504	0.3665	0.1815	0.3388	0.1361
Semantic NoSQL KE 3	0.1710	0.2242	0.1497	0.1662	0.3551	0.2027	0.2083	0.2213	0.1393	0.0959
Semantic NoSQL KE 4	0.2126	0.1598	0.3624	0.2129	0.2761	0.4768	0.3624	0.1518	0.3546	0.1225
Semantic NoSQL KE 5	0.2074	0.1546	0.3613	0.2080	0.2693	0.4772	0.3612	0.1467	0.3536	0.1193
AttentionMeSH2	0.6296	0.6753	0.5992	0.6150	0.6229	0.4213	0.4253	0.6710	0.5930	0.4611
AttentionMeSH3	0.6297	0.6757	0.5992	0.6151	0.6246	0.4211	0.4252	0.6712	0.5930	0.4613
AttentionMeSH4	0.6298	0.6759	0.5992	0.6151	0.6257	0.4210	0.4251	0.6714	0.5930	0.4613
AttentionMeSH5	0.6298	0.6760	0.5992	0.6152	0.6268	0.4210	0.4251	0.6715	0.5930	0.4614
MeSHmallow-1	0.0985	0.4542	0.0639	0.1055	0.4741	0.0669	0.0674	0.3736	0.0567	0.0592
MeSHmallow-2	0.0985	0.4542	0.0639	0.1055	0.4741	0.0669	0.0674	0.3736	0.0567	0.0592
MeSHmallow-3	0.0985	0.4542	0.0639	0.1055	0.4741	0.0669	0.0674	0.3736	0.0567	0.0592
UMass Amherst T2T	0.1081	0.7339	0.0653	0.1180	0.7339	0.0001	0.0001	0.7339	0.0584	0.0653
3_runs_ckp4	0.1899	0.1942	0.1872	0.1841	0.1278	0.0011	0.0003	0.1924	0.1876	0.1053
3_runs	0.2006	0.2029	0.1978	0.1938	0.0617	0.0015	0.0005	0.2005	0.2007	0.1130
2_runs_ckp2	0.1978	0.1899	0.2045	0.1906	0.0739	0.0016	0.0005	0.1893	0.2072	0.1101

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5013	0.7937	0.7014	0.7245	0.5542	0.4847
Default MTI	0.5186	0.7685	0.7497	0.7409	0.5413	0.5258
iria-1	0.4460	0.6235	0.7364	0.6595	0.4253	0.4949
xgx	0.5019	0.7847	0.7057	0.7235	0.5456	0.4908
xgx1	0.5042	0.7793	0.7138	0.7256	0.5428	0.4973
xgx0	0.5026	0.7920	0.7026	0.7245	0.5503	0.4890
AttentionMeSH	0.4958	0.7879	0.6948	0.7181	0.5433	0.4820
Semantic NoSQL KE 1	0.2599	0.3812	0.5967	0.4495	0.2103	0.3772
Semantic NoSQL KE 2	0.2597	0.3808	0.5974	0.4494	0.2099	0.3776
Semantic NoSQL KE 3	0.2343	0.4827	0.3470	0.3771	0.2928	0.2193
Semantic NoSQL KE 4	0.2432	0.3362	0.6348	0.4249	0.1841	0.4046
Semantic NoSQL KE 5	0.2389	0.3276	0.6363	0.4181	0.1790	0.4056
AttentionMeSH2	0.4961	0.7891	0.6945	0.7185	0.5442	0.4819
AttentionMeSH3	0.4962	0.7894	0.6944	0.7185	0.5443	0.4819
AttentionMeSH4	0.4962	0.7894	0.6944	0.7185	0.5443	0.4818
AttentionMeSH5	0.4961	0.7895	0.6943	0.7185	0.5444	0.4818
MeSHmallow-1	0.1327	0.6805	0.1203	0.1917	0.3733	0.0854
MeSHmallow-2	0.1327	0.6805	0.1203	0.1917	0.3733	0.0854
MeSHmallow-3	0.1327	0.6805	0.1203	0.1917	0.3733	0.0854
UMass Amherst T2T	0.0786	0.7522	0.0735	0.1320	0.2904	0.0465
3_runs_ckp4	0.2114	0.2800	0.3138	0.2857	0.2160	0.2209
3_runs	0.2241	0.3707	0.3212	0.3320	0.2408	0.2245
2_runs_ckp2	0.2090	0.3376	0.2391	0.2697	0.2353	0.1999

+ Test batch 3, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6262	0.6794	0.5975	0.6172	0.6370	0.5543	0.5316	0.6713	0.5867	0.4638
Default MTI	0.6378	0.6481	0.6486	0.6310	0.6004	0.5954	0.5526	0.6419	0.6337	0.4781
iria-1	0.5333	0.4754	0.6151	0.5235	0.4552	0.4944	0.4409	0.4790	0.6015	0.3673
iria-2	0.5004	0.4459	0.5762	0.4907	0.4151	0.4328	0.3880	0.4498	0.5636	0.3376
DeepMeSH1	0.6471	0.6910	0.6211	0.6377	0.6519	0.4761	0.4758	0.6935	0.6065	0.4834
DeepMeSH2	0.6688	0.7128	0.6452	0.6611	0.6754	0.5309	0.5264	0.7136	0.6293	0.5096
DeepMeSH3	0.6600	0.7023	0.6356	0.6510	0.6627	0.5049	0.5023	0.7047	0.6206	0.4979
DeepMeSH4	0.6732	0.7193	0.6481	0.6657	0.6817	0.5349	0.5312	0.7201	0.6321	0.5148
xgx	0.6322	0.6659	0.6137	0.6205	0.6026	0.4685	0.4655	0.6606	0.6061	0.4652
xgx2	0.6457	0.6770	0.6256	0.6344	0.6282	0.4837	0.4825	0.6770	0.6172	0.4799
xgx3	0.6564	0.6968	0.6322	0.6448	0.6476	0.4947	0.4953	0.6926	0.6238	0.4916
xgx1	0.6359	0.6695	0.6176	0.6243	0.6059	0.4846	0.4803	0.6649	0.6092	0.4691
xgx0	0.6380	0.6789	0.6142	0.6257	0.6201	0.4722	0.4716	0.6729	0.6065	0.4710
AttentionMeSH	0.6415	0.6762	0.6138	0.6254	0.6082	0.4213	0.4210	0.6736	0.6123	0.4716
Semantic NoSQL KE 1	0.2346	0.1850	0.3457	0.2314	0.3944	0.4509	0.3645	0.1805	0.3351	0.1349
Semantic NoSQL KE 2	0.2347	0.1849	0.3463	0.2315	0.3927	0.4523	0.3655	0.1804	0.3357	0.1349
Semantic NoSQL KE 3	0.1690	0.2229	0.1475	0.1640	0.3556	0.2095	0.2129	0.2206	0.1370	0.0943
Semantic NoSQL KE 4	0.2132	0.1589	0.3659	0.2133	0.2751	0.4882	0.3674	0.1521	0.3558	0.1224
Semantic NoSQL KE 5	0.2075	0.1532	0.3642	0.2078	0.2677	0.4890	0.3656	0.1467	0.3543	0.1188
AttentionMeSH2	0.6420	0.6777	0.6137	0.6259	0.6143	0.4206	0.4205	0.6748	0.6122	0.4723
AttentionMeSH3	0.6420	0.6779	0.6137	0.6260	0.6156	0.4202	0.4201	0.6750	0.6121	0.4723
AttentionMeSH4	0.6421	0.6781	0.6136	0.6260	0.6165	0.4200	0.4198	0.6751	0.6121	0.4724
AttentionMeSH5	0.6421	0.6782	0.6136	0.6261	0.6172	0.4199	0.4197	0.6752	0.6121	0.4724
MeSHmallow-1	0.0957	0.4566	0.0606	0.1011	0.4695	0.0616	0.0636	0.3767	0.0548	0.0564
MeSHmallow-2	0.0957	0.4566	0.0606	0.1011	0.4695	0.0616	0.0636	0.3767	0.0548	0.0564
MeSHmallow-3	0.0957	0.4566	0.0606	0.1011	0.4695	0.0616	0.0636	0.3767	0.0548	0.0564
UMass Amherst T2T	0.4679	0.5087	0.4480	0.4584	0.3684	0.1983	0.1951	0.4908	0.4470	0.3131

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5057	0.7983	0.7020	0.7289	0.5577	0.4868
Default MTI	0.5195	0.7671	0.7473	0.7402	0.5408	0.5258
iria-1	0.4506	0.6282	0.7372	0.6636	0.4311	0.4959
iria-2	0.4292	0.6018	0.7072	0.6357	0.4099	0.4733
DeepMeSH1	0.5147	0.8051	0.7091	0.7374	0.5675	0.4935
DeepMeSH2	0.5361	0.8193	0.7290	0.7555	0.5855	0.5173
DeepMeSH3	0.5258	0.8115	0.7210	0.7474	0.5760	0.5060
DeepMeSH4	0.5389	0.8235	0.7306	0.7584	0.5899	0.5184
xgx	0.5085	0.7942	0.7055	0.7290	0.5543	0.4936
xgx2	0.5187	0.8036	0.7148	0.7401	0.5644	0.5022
xgx3	0.5255	0.8155	0.7165	0.7450	0.5764	0.5067
xgx1	0.5130	0.7968	0.7104	0.7328	0.5592	0.4982
xgx0	0.5101	0.8019	0.7020	0.7295	0.5620	0.4917
AttentionMeSH	0.5045	0.7881	0.6986	0.7218	0.5509	0.4893
Semantic NoSQL KE 1	0.2601	0.3803	0.5964	0.4499	0.2100	0.3768
Semantic NoSQL KE 2	0.2599	0.3798	0.5971	0.4497	0.2097	0.3771
Semantic NoSQL KE 3	0.2347	0.4813	0.3431	0.3749	0.2952	0.2178
Semantic NoSQL KE 4	0.2428	0.3349	0.6389	0.4264	0.1823	0.4056
Semantic NoSQL KE 5	0.2385	0.3256	0.6398	0.4189	0.1773	0.4068
AttentionMeSH2	0.5048	0.7893	0.6983	0.7221	0.5516	0.4892
AttentionMeSH3	0.5048	0.7895	0.6982	0.7222	0.5517	0.4892
AttentionMeSH4	0.5048	0.7896	0.6981	0.7221	0.5518	0.4891
AttentionMeSH5	0.5048	0.7896	0.6980	0.7221	0.5518	0.4890
MeSHmallow-1	0.1297	0.6868	0.1145	0.1845	0.3746	0.0828
MeSHmallow-2	0.1297	0.6868	0.1145	0.1845	0.3746	0.0828
MeSHmallow-3	0.1297	0.6868	0.1145	0.1845	0.3746	0.0828
UMass Amherst T2T	0.4029	0.6878	0.5561	0.5931	0.4618	0.3800

+ Test batch 3, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6247	0.6802	0.5998	0.6178	0.6391	0.5410	0.5217	0.6687	0.5862	0.4663
Default MTI	0.6373	0.6481	0.6524	0.6324	0.6042	0.5852	0.5472	0.6398	0.6348	0.4800
iria-1	0.5319	0.4739	0.6162	0.5228	0.4628	0.4970	0.4447	0.4774	0.6003	0.3669
iria-2	0.4993	0.4447	0.5765	0.4900	0.4209	0.4370	0.3932	0.4486	0.5627	0.3369
DeepMeSH1	0.6428	0.6857	0.6214	0.6345	0.6534	0.4718	0.4722	0.6860	0.6047	0.4798
DeepMeSH2	0.6627	0.7071	0.6429	0.6562	0.6777	0.5218	0.5192	0.7055	0.6249	0.5044
DeepMeSH3	0.6505	0.6964	0.6274	0.6425	0.6616	0.4936	0.4926	0.6962	0.6104	0.4885
DeepMeSH4	0.6652	0.7111	0.6442	0.6589	0.6824	0.5295	0.5269	0.7095	0.6261	0.5072
xgx	0.6583	0.6764	0.6517	0.6489	0.6543	0.5199	0.5146	0.6785	0.6392	0.4947
xgx2	0.6712	0.6938	0.6680	0.6627	0.6568	0.5584	0.5431	0.6891	0.6541	0.5110
xgx3	0.6567	0.6995	0.6340	0.6463	0.6699	0.4961	0.4988	0.6942	0.6230	0.4928
xgx1	0.6705	0.7161	0.6482	0.6619	0.6806	0.5364	0.5308	0.7109	0.6344	0.5106
xgx0	0.6697	0.7158	0.6471	0.6610	0.6791	0.5353	0.5300	0.7107	0.6332	0.5096
AttentionMeSH	0.6398	0.6716	0.6187	0.6256	0.6081	0.4195	0.4180	0.6684	0.6136	0.4720
Semantic NoSQL KE 1	0.2316	0.1831	0.3433	0.2295	0.3970	0.4411	0.3563	0.1778	0.3319	0.1337
Semantic NoSQL KE 2	0.2315	0.1829	0.3435	0.2294	0.3946	0.4419	0.3569	0.1776	0.3322	0.1336
Semantic NoSQL KE 3	0.1719	0.2245	0.1524	0.1679	0.3644	0.2103	0.2145	0.2214	0.1405	0.0969
Semantic NoSQL KE 4	0.2099	0.1575	0.3608	0.2111	0.2756	0.4748	0.3595	0.1499	0.3502	0.1210
Semantic NoSQL KE 5	0.2045	0.1521	0.3599	0.2060	0.2691	0.4749	0.3589	0.1446	0.3493	0.1177
AttentionMeSH2	0.6404	0.6732	0.6185	0.6262	0.6143	0.4187	0.4175	0.6698	0.6134	0.4727
AttentionMeSH3	0.6405	0.6737	0.6185	0.6264	0.6165	0.4185	0.4174	0.6702	0.6134	0.4729
AttentionMeSH4	0.6406	0.6739	0.6184	0.6264	0.6176	0.4183	0.4172	0.6704	0.6134	0.4729
AttentionMeSH5	0.6407	0.6741	0.6184	0.6265	0.6185	0.4181	0.4171	0.6705	0.6133	0.4730
MeSHmallow-1	0.0997	0.4446	0.0650	0.1061	0.4647	0.0666	0.0663	0.3623	0.0578	0.0595
MeSHmallow-2	0.0997	0.4446	0.0650	0.1061	0.4647	0.0666	0.0663	0.3623	0.0578	0.0595
MeSHmallow-3	0.0997	0.4446	0.0650	0.1061	0.4647	0.0666	0.0663	0.3623	0.0578	0.0595
UMass Amherst T2T	0.4805	0.5198	0.4640	0.4708	0.4084	0.2195	0.2181	0.5020	0.4608	0.3242

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5074	0.7999	0.7046	0.7298	0.5616	0.4880
Default MTI	0.5222	0.7705	0.7506	0.7431	0.5434	0.5291
iria-1	0.4534	0.6323	0.7426	0.6682	0.4329	0.5006
iria-2	0.4322	0.6057	0.7104	0.6390	0.4119	0.4784
DeepMeSH1	0.5123	0.8054	0.7077	0.7355	0.5630	0.4940
DeepMeSH2	0.5316	0.8193	0.7267	0.7527	0.5804	0.5148
DeepMeSH3	0.5190	0.8122	0.7144	0.7421	0.5703	0.5002
DeepMeSH4	0.5326	0.8208	0.7279	0.7542	0.5818	0.5153
xgx	0.5328	0.8023	0.7398	0.7539	0.5692	0.5238
xgx2	0.5418	0.8095	0.7520	0.7621	0.5768	0.5373
xgx3	0.5274	0.8201	0.7196	0.7476	0.5793	0.5097
xgx1	0.5382	0.8270	0.7322	0.7586	0.5889	0.5210
xgx0	0.5373	0.8264	0.7309	0.7575	0.5883	0.5205
AttentionMeSH	0.5054	0.7866	0.7028	0.7233	0.5501	0.4929
Semantic NoSQL KE 1	0.2598	0.3830	0.5995	0.4524	0.2099	0.3770
Semantic NoSQL KE 2	0.2596	0.3825	0.6001	0.4523	0.2095	0.3773
Semantic NoSQL KE 3	0.2360	0.4860	0.3541	0.3829	0.2932	0.2219
Semantic NoSQL KE 4	0.2425	0.3384	0.6411	0.4296	0.1828	0.4029
Semantic NoSQL KE 5	0.2379	0.3294	0.6426	0.4224	0.1775	0.4045
AttentionMeSH2	0.5057	0.7880	0.7025	0.7237	0.5509	0.4927
AttentionMeSH3	0.5058	0.7884	0.7024	0.7239	0.5512	0.4927
AttentionMeSH4	0.5058	0.7886	0.7023	0.7239	0.5513	0.4927
AttentionMeSH5	0.5058	0.7888	0.7023	0.7239	0.5515	0.4926
MeSHmallow-1	0.1349	0.6846	0.1225	0.1943	0.3760	0.0872
MeSHmallow-2	0.1349	0.6846	0.1225	0.1943	0.3760	0.0872
MeSHmallow-3	0.1349	0.6846	0.1225	0.1943	0.3760	0.0872
UMass Amherst T2T	0.4116	0.7001	0.5731	0.6076	0.4680	0.3913

+ Test batch 3, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6334	0.6913	0.6077	0.6264	0.6477	0.5634	0.5445	0.6803	0.5927	0.4754
Default MTI	0.6462	0.6556	0.6627	0.6406	0.6144	0.6097	0.5711	0.6479	0.6444	0.4883
iria-1	0.5284	0.4709	0.6106	0.5179	0.4404	0.4934	0.4409	0.4743	0.5963	0.3630
iria-2	0.4939	0.4398	0.5699	0.4835	0.3953	0.4322	0.3886	0.4436	0.5570	0.3318
DeepMeSH1	0.6557	0.7031	0.6310	0.6478	0.6590	0.5019	0.5016	0.7052	0.6126	0.4946
DeepMeSH2	0.6717	0.7180	0.6510	0.6660	0.6810	0.5387	0.5372	0.7181	0.6310	0.5156
DeepMeSH3	0.6773	0.7416	0.6424	0.6707	0.7072	0.5375	0.5412	0.7407	0.6238	0.5219
DeepMeSH4	0.6732	0.7211	0.6509	0.6674	0.6837	0.5407	0.5395	0.7216	0.6309	0.5169
DeepMeSH5	0.6482	0.6938	0.6235	0.6398	0.6525	0.4804	0.4816	0.6966	0.6060	0.4859
xgx	0.6791	0.6940	0.6778	0.6714	0.6538	0.5716	0.5594	0.6974	0.6617	0.5204
xgx2	0.6826	0.7144	0.6718	0.6750	0.6713	0.5665	0.5577	0.7106	0.6567	0.5253
xgx3	0.6805	0.6904	0.6916	0.6740	0.6372	0.6074	0.5827	0.6861	0.6751	0.5237
xgx1	0.6806	0.7067	0.6725	0.6719	0.6662	0.5511	0.5462	0.7038	0.6589	0.5217
xgx0	0.6831	0.7049	0.6810	0.6756	0.6626	0.5777	0.5638	0.7016	0.6655	0.5258
AttentionMeSH	0.6479	0.6862	0.6209	0.6340	0.6083	0.4279	0.4300	0.6834	0.6159	0.4819
AttentionMeSH2	0.6484	0.6876	0.6207	0.6345	0.6141	0.4272	0.4294	0.6846	0.6158	0.4825
AttentionMeSH3	0.6485	0.6880	0.6207	0.6346	0.6157	0.4269	0.4291	0.6849	0.6158	0.4826
AttentionMeSH4	0.6486	0.6882	0.6206	0.6347	0.6168	0.4266	0.4288	0.6851	0.6157	0.4827
AttentionMeSH5	0.6486	0.6883	0.6206	0.6347	0.6177	0.4266	0.4288	0.6852	0.6157	0.4827
Access Inn MAIstro	0.2593	0.3951	0.2098	0.2527	0.4185	0.3302	0.3027	0.3635	0.2015	0.1521
UMass Amherst T2T	0.4863	0.5320	0.4626	0.4759	0.4029	0.2266	0.2255	0.5166	0.4593	0.3291

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5152	0.8037	0.7106	0.7349	0.5677	0.4974
Default MTI	0.5286	0.7751	0.7594	0.7503	0.5478	0.5374
iria-1	0.4496	0.6243	0.7376	0.6604	0.4278	0.4994
iria-2	0.4286	0.5973	0.7079	0.6322	0.4073	0.4768
DeepMeSH1	0.5237	0.8122	0.7192	0.7454	0.5739	0.5051
DeepMeSH2	0.5403	0.8222	0.7351	0.7597	0.5879	0.5234
DeepMeSH3	0.5427	0.8388	0.7259	0.7613	0.6014	0.5179
DeepMeSH4	0.5398	0.8241	0.7348	0.7602	0.5884	0.5219
DeepMeSH5	0.5190	0.8066	0.7119	0.7386	0.5699	0.5002
xgx	0.5506	0.8083	0.7604	0.7690	0.5820	0.5451
xgx2	0.5516	0.8226	0.7539	0.7700	0.5915	0.5417
xgx3	0.5523	0.8039	0.7715	0.7712	0.5759	0.5563
xgx1	0.5516	0.8194	0.7540	0.7690	0.5898	0.5425
xgx0	0.5530	0.8150	0.7623	0.7714	0.5856	0.5486
AttentionMeSH	0.5136	0.7966	0.7046	0.7295	0.5594	0.4985
AttentionMeSH2	0.5139	0.7979	0.7043	0.7299	0.5604	0.4984
AttentionMeSH3	0.5139	0.7982	0.7042	0.7300	0.5606	0.4983
AttentionMeSH4	0.5139	0.7984	0.7041	0.7300	0.5607	0.4982
AttentionMeSH5	0.5140	0.7984	0.7041	0.7300	0.5608	0.4982
Access Inn MAIstro	0.2573	0.6179	0.4024	0.4591	0.3493	0.2257
UMass Amherst T2T	0.4175	0.7055	0.5721	0.6101	0.4765	0.3947

+ Test batch 3, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6332	0.6833	0.6121	0.6264	0.6377	0.5622	0.5408	0.6730	0.5977	0.4759
Default MTI	0.6474	0.6540	0.6648	0.6418	0.6086	0.6084	0.5667	0.6475	0.6473	0.4911
iria-1	0.5161	0.4609	0.5929	0.5058	0.4271	0.4658	0.4147	0.4654	0.5792	0.3525
iria-2	0.4822	0.4302	0.5521	0.4716	0.3849	0.4085	0.3666	0.4352	0.5405	0.3223
DeepMeSH1	0.6331	0.7562	0.5564	0.6216	0.7349	0.3881	0.4114	0.7564	0.5443	0.4690
DeepMeSH2	0.6441	0.7514	0.5768	0.6336	0.7326	0.4193	0.4394	0.7521	0.5632	0.4814
DeepMeSH3	0.6637	0.6767	0.6659	0.6544	0.6352	0.5455	0.5281	0.6761	0.6517	0.5020
DeepMeSH4	0.6288	0.8128	0.5226	0.6148	0.7943	0.3662	0.3964	0.8110	0.5134	0.4640
DeepMeSH5	0.6558	0.6678	0.6581	0.6458	0.6279	0.5235	0.5071	0.6673	0.6448	0.4921
xgx	0.6880	0.7193	0.6736	0.6789	0.6782	0.5804	0.5671	0.7172	0.6611	0.5310
xgx2	0.6836	0.6939	0.6833	0.6748	0.6485	0.5925	0.5730	0.6983	0.6695	0.5258
xgx3	0.6843	0.6981	0.6875	0.6765	0.6476	0.6110	0.5872	0.6955	0.6734	0.5279
xgx1	0.6875	0.7052	0.6850	0.6785	0.6628	0.5922	0.5728	0.7031	0.6726	0.5304
xgx0	0.6878	0.7045	0.6860	0.6787	0.6620	0.5936	0.5741	0.7025	0.6737	0.5306
AttentionMeSH	0.6635	0.6853	0.6488	0.6497	0.6178	0.4943	0.4827	0.6833	0.6447	0.4982
AttentionMeSH2	0.6635	0.6854	0.6488	0.6497	0.6185	0.4941	0.4826	0.6834	0.6447	0.4983
AttentionMeSH3	0.6635	0.6855	0.6488	0.6497	0.6187	0.4939	0.4823	0.6835	0.6447	0.4983
AttentionMeSH4	0.6635	0.6855	0.6487	0.6497	0.6189	0.4937	0.4821	0.6835	0.6446	0.4983
AttentionMeSH5	0.6635	0.6856	0.6487	0.6497	0.6193	0.4934	0.4819	0.6836	0.6446	0.4983
MeSHmallow-1	0.3161	0.3798	0.2661	0.3042	0.1333	0.0049	0.0037	0.3798	0.2707	0.1915
MeSHmallow-2	0.3161	0.3798	0.2661	0.3042	0.1333	0.0049	0.0037	0.3798	0.2707	0.1915
Access Inn MAIstro	0.2788	0.2488	0.3558	0.2775	0.3942	0.4641	0.3905	0.2351	0.3423	0.1669
MeSHmallow-3	0.3161	0.3798	0.2661	0.3042	0.1333	0.0049	0.0037	0.3798	0.2707	0.1915
UMass Amherst T2T	0.4988	0.5408	0.4789	0.4881	0.4179	0.2526	0.2481	0.5239	0.4759	0.3392

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5172	0.7964	0.7186	0.7373	0.5662	0.5014
Default MTI	0.5325	0.7703	0.7647	0.7514	0.5510	0.5415
iria-1	0.4443	0.6174	0.7290	0.6536	0.4251	0.4902
iria-2	0.4248	0.5888	0.6976	0.6238	0.4050	0.4700
DeepMeSH1	0.4945	0.8519	0.6419	0.7123	0.5969	0.4439
DeepMeSH2	0.5070	0.8480	0.6639	0.7256	0.5994	0.4615
DeepMeSH3	0.5366	0.7899	0.7555	0.7560	0.5643	0.5364
DeepMeSH4	0.4826	0.8907	0.6035	0.6981	0.6212	0.4157
DeepMeSH5	0.5289	0.7825	0.7473	0.7480	0.5567	0.5286
xgx	0.5572	0.8234	0.7589	0.7739	0.5972	0.5455
xgx2	0.5556	0.8058	0.7695	0.7736	0.5835	0.5518
xgx3	0.5563	0.8087	0.7726	0.7752	0.5829	0.5558
xgx1	0.5594	0.8127	0.7705	0.7757	0.5905	0.5552
xgx0	0.5596	0.8123	0.7714	0.7760	0.5901	0.5561
AttentionMeSH	0.5290	0.7902	0.7396	0.7472	0.5627	0.5235
AttentionMeSH2	0.5290	0.7902	0.7395	0.7472	0.5628	0.5235
AttentionMeSH3	0.5290	0.7902	0.7395	0.7472	0.5628	0.5234
AttentionMeSH4	0.5289	0.7903	0.7395	0.7472	0.5628	0.5234
AttentionMeSH5	0.5290	0.7905	0.7394	0.7472	0.5629	0.5234
MeSHmallow-1	0.2871	0.5633	0.3287	0.3967	0.4000	0.2369
MeSHmallow-2	0.2871	0.5633	0.3287	0.3967	0.4000	0.2369
Access Inn MAIstro	0.2964	0.4696	0.5921	0.5043	0.2722	0.3615
MeSHmallow-3	0.2871	0.5633	0.3287	0.3967	0.4000	0.2369
UMass Amherst T2T	0.4276	0.7094	0.5961	0.6262	0.4818	0.4087

Platinum sponsors of the BioASQ Challenge between 2012 and 2023

Google was a proud sponsor of the BioASQ Challenge from 2018 to 2022.
BioASQ was supported by the National Library of Medicine of the National Institutes of Health under award number R13LM012214. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
The research leading to these results has received funding from the European Commission's Seventh Framework Programme (FP7/2007-2013, ICT-2011.4.4(d), Intelligent Information Management, Targeted Competition Framework) under grant agreement n° 318652.

BioASQ Participants Area

Test Results for Task 6a

+ Dry Run 1

Annotated articles:27480/28563

Flat Measures

Hierarchical Measures

+ Dry Run 2

Annotated articles:198/255

Flat Measures

Hierarchical Measures

+ Test batch 1, week 1

Annotated articles:6639/7240

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:7499/7678

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:10319/10488

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:6073/6225

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:6486/6617

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:6118/6239

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:6803/7152

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:6575/7113

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:5412/5833

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:6606/7379

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:5768/6469

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:5501/6544

Flat Measures

Hierarchical Measures

+ Test batch 3, week 3

Annotated articles:5467/6743

Flat Measures

Hierarchical Measures

+ Test batch 3, week 4

Annotated articles:5615/8478

Flat Measures

Hierarchical Measures

+ Test batch 3, week 5

Annotated articles:4038/7478

Flat Measures

Hierarchical Measures