BioASQ Participants Area

Test Results for Task 9a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Test batch 1, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6551	0.7063	0.6392	0.6524	0.6656	0.5641	0.5488	0.6946	0.6198	0.5050
Default MTI	0.6536	0.6587	0.6750	0.6486	0.6243	0.5933	0.5586	0.6518	0.6554	0.4969
deepmesh_dmiip_fdu	0.6706	0.7475	0.6273	0.6665	0.7042	0.5182	0.5271	0.7480	0.6077	0.5192
deepmesh_dmiip_fdu_	0.6620	0.7376	0.6162	0.6559	0.6951	0.5013	0.5107	0.7400	0.5989	0.5042
attention_dmiip_fdu	0.1894	0.2087	0.1933	0.1933	0.0788	0.0826	0.0703	0.1961	0.1831	0.1393
NLM CNN	0.6517	0.6991	0.6264	0.6407	0.6438	0.4589	0.4659	0.6944	0.6141	0.4883
dmiip_fdu	0.1128	0.1146	0.1099	0.1059	0.0010	0.0008	0.0008	0.1160	0.1099	0.0597
NLM System 1	0.6762	0.7128	0.6614	0.6687	0.6696	0.5029	0.5063	0.7163	0.6403	0.5182
NLM System 3	0.6864	0.7319	0.6692	0.6824	0.6836	0.5242	0.5282	0.7316	0.6464	0.5375
bert_dna_2	0.1045	0.7112	0.0629	0.1138	0.7112	0.0001	0.0001	0.7112	0.0564	0.0629
pi_dna_2	0.1045	0.7112	0.0629	0.1138	0.7112	0.0001	0.0001	0.7112	0.0564	0.0629

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5323	0.8083	0.7343	0.7516	0.5771	0.5191
Default MTI	0.5278	0.7696	0.7677	0.7515	0.5432	0.5408
deepmesh_dmiip_fdu	0.5359	0.8375	0.7156	0.7560	0.5991	0.5059
deepmesh_dmiip_fdu_	0.5215	0.8304	0.7059	0.7470	0.5857	0.4912
attention_dmiip_fdu	0.2371	0.3219	0.2996	0.2966	0.2565	0.2367
NLM CNN	0.5178	0.8118	0.7118	0.7388	0.5693	0.5007
dmiip_fdu	0.1718	0.2351	0.2223	0.2143	0.1866	0.1751
NLM System 1	0.5403	0.8123	0.7484	0.7620	0.5800	0.5305
NLM System 3	0.5569	0.8261	0.7546	0.7725	0.5999	0.5436
bert_dna_2	0.0770	0.7321	0.0698	0.1258	0.2861	0.0455
pi_dna_2	0.0770	0.7321	0.0698	0.1258	0.2861	0.0455

+ Test batch 1, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6343	0.6640	0.6393	0.6310	0.6386	0.5797	0.5534	0.6533	0.6165	0.4798
Default MTI	0.6342	0.6246	0.6750	0.6294	0.6037	0.6114	0.5654	0.6179	0.6514	0.4759
deepmesh_dmiip_fdu	0.6854	0.7094	0.6898	0.6821	0.7003	0.5947	0.5870	0.7034	0.6682	0.5341
deepmesh_dmiip_fdu_	0.6827	0.7027	0.6892	0.6778	0.6951	0.5939	0.5852	0.6979	0.6681	0.5283
attention_dmiip_fdu	0.6834	0.7056	0.6881	0.6784	0.6977	0.5928	0.5855	0.7010	0.6667	0.5293
NLM CNN	0.6231	0.6596	0.6143	0.6137	0.6215	0.4371	0.4433	0.6510	0.5975	0.4588
dmiip_fdu	0.6858	0.7120	0.6880	0.6823	0.7027	0.5931	0.5868	0.7064	0.6664	0.5346
NLM System 1	0.6543	0.6795	0.6555	0.6481	0.6616	0.5001	0.5017	0.6794	0.6310	0.4947
NLM System 3	0.6589	0.6846	0.6600	0.6529	0.6650	0.5109	0.5119	0.6839	0.6356	0.5008
DistilBert	0.0001	0.0006	0.0001	0.0001	0.0006	0.0001	0.0000	0.0006	0.0000	0.0001

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5100	0.7832	0.7360	0.7397	0.5437	0.5085
Default MTI	0.5096	0.7491	0.7686	0.7406	0.5166	0.5326
deepmesh_dmiip_fdu	0.5505	0.8119	0.7739	0.7764	0.5798	0.5499
deepmesh_dmiip_fdu_	0.5453	0.8070	0.7735	0.7734	0.5731	0.5467
attention_dmiip_fdu	0.5457	0.8099	0.7715	0.7736	0.5750	0.5457
NLM CNN	0.4962	0.7895	0.7053	0.7241	0.5429	0.4861
dmiip_fdu	0.5506	0.8146	0.7715	0.7764	0.5815	0.5484
NLM System 1	0.5230	0.7916	0.7493	0.7519	0.5570	0.5200
NLM System 3	0.5271	0.7949	0.7524	0.7553	0.5611	0.5241
DistilBert	0.1188	0.7351	0.0478	0.0889	0.5421	0.0677

+ Test batch 1, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6463	0.6902	0.6394	0.6452	0.6406	0.5686	0.5469	0.6758	0.6194	0.4966
Default MTI	0.6461	0.6388	0.6824	0.6424	0.5964	0.6003	0.5581	0.6320	0.6609	0.4905
NLM System 2	0.6726	0.6750	0.6870	0.6640	0.6734	0.5136	0.5092	0.6756	0.6697	0.5121
deepmesh_dmiip_fdu	0.7032	0.7447	0.6907	0.7016	0.7103	0.5898	0.5860	0.7390	0.6707	0.5576
deepmesh_dmiip_fdu_	0.7008	0.7254	0.6976	0.6961	0.6971	0.5895	0.5824	0.7245	0.6786	0.5497
attention_dmiip_fdu	0.7039	0.7302	0.6995	0.6991	0.7037	0.5904	0.5848	0.7294	0.6800	0.5535
NLM CNN	0.6449	0.6836	0.6272	0.6346	0.6375	0.4544	0.4575	0.6793	0.6138	0.4814
dmiip_fdu	0.7049	0.7487	0.6902	0.7030	0.7156	0.5890	0.5868	0.7431	0.6704	0.5595
NLM System 1	0.6690	0.6942	0.6666	0.6626	0.6591	0.5080	0.5054	0.6962	0.6438	0.5111
NLM System 3	0.6763	0.6811	0.6905	0.6683	0.6729	0.5271	0.5206	0.6804	0.6723	0.5180
DistilBert	0.0004	0.0027	0.0002	0.0004	0.0024	0.0000	0.0001	0.0027	0.0002	0.0002
bert_dna	0.3321	0.3695	0.3079	0.3138	0.2100	0.0166	0.0161	0.3496	0.3163	0.1970
roberta	0.0940	0.2255	0.0565	0.0885	0.1268	0.0007	0.0004	0.2402	0.0584	0.0483

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5241	0.7983	0.7355	0.7487	0.5652	0.5146
Default MTI	0.5233	0.7573	0.7733	0.7491	0.5310	0.5439
NLM System 2	0.5363	0.7883	0.7682	0.7618	0.5589	0.5416
deepmesh_dmiip_fdu	0.5675	0.8345	0.7742	0.7894	0.6056	0.5570
deepmesh_dmiip_fdu_	0.5613	0.8210	0.7784	0.7848	0.5916	0.5576
attention_dmiip_fdu	0.5643	0.8251	0.7780	0.7864	0.5959	0.5597
NLM CNN	0.5097	0.8018	0.7117	0.7349	0.5577	0.4957
dmiip_fdu	0.5691	0.8380	0.7726	0.7900	0.6090	0.5571
NLM System 1	0.5334	0.7991	0.7543	0.7593	0.5682	0.5288
NLM System 3	0.5403	0.7920	0.7709	0.7649	0.5644	0.5448
DistilBert	0.0858	0.2831	0.0383	0.0650	0.3024	0.0511
bert_dna	0.3097	0.5768	0.3955	0.4416	0.3916	0.2794
roberta	0.1538	0.6757	0.0918	0.1573	0.4634	0.0949

+ Test batch 1, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6429	0.6912	0.6293	0.6394	0.6525	0.5669	0.5493	0.6795	0.6100	0.4914
Default MTI	0.6391	0.6383	0.6682	0.6333	0.6074	0.6026	0.5633	0.6312	0.6473	0.4807
NLM System 2	0.6606	0.6690	0.6730	0.6520	0.6722	0.5088	0.5059	0.6676	0.6537	0.4996
deepmesh_dmiip_fdu	0.6821	0.7129	0.6790	0.6768	0.6743	0.5923	0.5808	0.7038	0.6618	0.5295
deepmesh_dmiip_fdu_	0.6791	0.7045	0.6788	0.6719	0.6668	0.5923	0.5784	0.6971	0.6621	0.5231
attention_dmiip_fdu	0.6780	0.7029	0.6780	0.6709	0.6657	0.5875	0.5746	0.6955	0.6613	0.5213
NLM CNN	0.6303	0.6760	0.6111	0.6194	0.6278	0.4520	0.4554	0.6695	0.5954	0.4659
dmiip_fdu	0.6810	0.7113	0.6782	0.6758	0.6730	0.5875	0.5769	0.7022	0.6610	0.5277
NLM System 1	0.6595	0.6889	0.6559	0.6530	0.6663	0.5094	0.5090	0.6902	0.6315	0.5010
NLM System 3	0.6644	0.6730	0.6779	0.6560	0.6721	0.5261	0.5211	0.6707	0.6582	0.5054
bert_dna	0.4001	0.4294	0.3835	0.3809	0.3390	0.0652	0.0681	0.4172	0.3843	0.2476
pi_dna	0.4103	0.4179	0.4103	0.3924	0.3348	0.0817	0.0830	0.4095	0.4111	0.2566

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5187	0.8012	0.7260	0.7435	0.5628	0.5069
Default MTI	0.5113	0.7582	0.7612	0.7416	0.5240	0.5280
NLM System 2	0.5269	0.7911	0.7549	0.7548	0.5534	0.5301
deepmesh_dmiip_fdu	0.5478	0.8171	0.7644	0.7730	0.5805	0.5448
deepmesh_dmiip_fdu_	0.5420	0.8103	0.7649	0.7694	0.5722	0.5421
attention_dmiip_fdu	0.5397	0.8085	0.7639	0.7681	0.5689	0.5405
NLM CNN	0.4994	0.8033	0.6998	0.7269	0.5519	0.4836
dmiip_fdu	0.5455	0.8153	0.7634	0.7717	0.5772	0.5432
NLM System 1	0.5252	0.8014	0.7450	0.7547	0.5631	0.5183
NLM System 3	0.5315	0.7937	0.7595	0.7584	0.5578	0.5352
bert_dna	0.3587	0.6456	0.4935	0.5307	0.4293	0.3347
pi_dna	0.3699	0.6343	0.5244	0.5479	0.4236	0.3551

+ Test batch 1, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6485	0.6973	0.6372	0.6486	0.6517	0.5653	0.5457	0.6835	0.6169	0.5028
Default MTI	0.6426	0.6404	0.6759	0.6394	0.6017	0.5973	0.5537	0.6310	0.6546	0.4874
deepmesh_dmiip_fdu	0.6917	0.7080	0.6992	0.6863	0.6887	0.5998	0.5863	0.6994	0.6842	0.5387
deepmesh_dmiip_fdu_	0.6924	0.7056	0.7016	0.6868	0.6870	0.5981	0.5836	0.6982	0.6866	0.5390
attention_dmiip_fdu	0.6932	0.7083	0.7010	0.6876	0.6903	0.5969	0.5837	0.7007	0.6859	0.5402
NLM CNN	0.6382	0.6699	0.6290	0.6273	0.6204	0.4561	0.4548	0.6596	0.6182	0.4727
dmiip_fdu	0.6937	0.7095	0.7012	0.6884	0.6910	0.5972	0.5840	0.7016	0.6860	0.5414
NLM System 1	0.6729	0.6897	0.6773	0.6663	0.6636	0.5376	0.5300	0.6896	0.6569	0.5151
NLM System 3	0.6767	0.6942	0.6815	0.6706	0.6665	0.5491	0.5411	0.6931	0.6611	0.5208
bert_dna	0.5186	0.5468	0.5044	0.5034	0.3897	0.2496	0.2281	0.5416	0.4974	0.3522
pi_dna	0.5406	0.5687	0.5319	0.5283	0.4038	0.3065	0.2825	0.5601	0.5225	0.3750
AtyponSystem1	0.4660	0.5818	0.4184	0.4650	0.5038	0.3796	0.3716	0.5624	0.3978	0.3154

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5350	0.8059	0.7385	0.7544	0.5762	0.5232
Default MTI	0.5227	0.7598	0.7726	0.7493	0.5318	0.5422
deepmesh_dmiip_fdu	0.5594	0.8115	0.7833	0.7817	0.5844	0.5614
deepmesh_dmiip_fdu_	0.5602	0.8094	0.7862	0.7825	0.5831	0.5639
attention_dmiip_fdu	0.5603	0.8115	0.7845	0.7825	0.5845	0.5628
NLM CNN	0.5106	0.7961	0.7224	0.7377	0.5518	0.5024
dmiip_fdu	0.5615	0.8122	0.7846	0.7830	0.5859	0.5638
NLM System 1	0.5421	0.8004	0.7691	0.7685	0.5680	0.5437
NLM System 3	0.5472	0.8040	0.7728	0.7722	0.5733	0.5487
bert_dna	0.4320	0.7203	0.6192	0.6435	0.4802	0.4193
pi_dna	0.4479	0.7324	0.6454	0.6645	0.4905	0.4391
AtyponSystem1	0.3917	0.7528	0.6006	0.6440	0.4513	0.3722

+ Test batch 2, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6473	0.6964	0.6309	0.6435	0.6591	0.5636	0.5464	0.6863	0.6126	0.4975
Default MTI	0.6453	0.6446	0.6723	0.6397	0.6170	0.5987	0.5601	0.6388	0.6519	0.4881
iria-1	0.4633	0.4107	0.5380	0.4542	0.3680	0.3852	0.3439	0.4143	0.5254	0.3058
iria-mix	0.4633	0.4107	0.5380	0.4542	0.3679	0.3850	0.3438	0.4143	0.5254	0.3058
deepmesh_dmiip_fdu	0.6939	0.7235	0.6872	0.6870	0.7061	0.5902	0.5853	0.7207	0.6689	0.5410
deepmesh_dmiip_fdu_	0.6954	0.7267	0.6872	0.6892	0.7090	0.5904	0.5863	0.7241	0.6689	0.5440
attention_dmiip_fdu	0.6940	0.7229	0.6875	0.6873	0.7068	0.5876	0.5830	0.7206	0.6693	0.5410
NLM CNN	0.6346	0.6801	0.6145	0.6239	0.6289	0.4518	0.4559	0.6735	0.5999	0.4707
dmiip_fdu	0.6956	0.7262	0.6875	0.6894	0.7096	0.5878	0.5840	0.7240	0.6693	0.5439
NLM System 1	0.6718	0.6942	0.6734	0.6653	0.6656	0.5352	0.5320	0.6956	0.6497	0.5147
NLM System 3	0.6784	0.6984	0.6801	0.6709	0.6698	0.5494	0.5455	0.7003	0.6579	0.5224
bert_dna	0.5356	0.5946	0.5017	0.5211	0.4389	0.2978	0.2856	0.5894	0.4908	0.3701
pi_dna	0.5369	0.5793	0.5267	0.5266	0.4742	0.4042	0.3791	0.5635	0.5127	0.3728
DeepSys1	0.1972	0.3299	0.2130	0.2142	0.1941	0.2482	0.2016	0.1999	0.1946	0.1257

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5284	0.8044	0.7319	0.7491	0.5717	0.5161
Default MTI	0.5217	0.7612	0.7671	0.7474	0.5334	0.5383
iria-1	0.4067	0.5743	0.6841	0.6097	0.3857	0.4534
iria-mix	0.4067	0.5743	0.6841	0.6097	0.3857	0.4535
deepmesh_dmiip_fdu	0.5567	0.8213	0.7711	0.7790	0.5903	0.5527
deepmesh_dmiip_fdu_	0.5593	0.8232	0.7711	0.7801	0.5936	0.5544
attention_dmiip_fdu	0.5561	0.8208	0.7711	0.7790	0.5889	0.5525
NLM CNN	0.5046	0.8033	0.7036	0.7293	0.5557	0.4898
dmiip_fdu	0.5588	0.8227	0.7712	0.7802	0.5922	0.5542
NLM System 1	0.5399	0.8037	0.7632	0.7660	0.5701	0.5394
NLM System 3	0.5462	0.8066	0.7692	0.7705	0.5755	0.5465
bert_dna	0.4392	0.7554	0.6149	0.6541	0.5041	0.4155
pi_dna	0.4440	0.7402	0.6496	0.6679	0.4908	0.4356
DeepSys1	0.2202	0.4963	0.4098	0.3857	0.2794	0.2410

+ Test batch 2, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6381	0.6788	0.6267	0.6331	0.6495	0.5810	0.5601	0.6698	0.6093	0.4848
Default MTI	0.6363	0.6326	0.6668	0.6308	0.6089	0.6146	0.5715	0.6266	0.6464	0.4791
iria-1	0.4508	0.3932	0.5280	0.4392	0.3557	0.3831	0.3385	0.3984	0.5191	0.2944
iria-mix	0.4508	0.3932	0.5280	0.4392	0.3557	0.3831	0.3386	0.3984	0.5191	0.2944
deepmesh_dmiip_fdu	0.6912	0.7144	0.6847	0.6832	0.6964	0.6035	0.5962	0.7140	0.6698	0.5368
deepmesh_dmiip_fdu_	0.6914	0.7152	0.6845	0.6834	0.6968	0.6034	0.5963	0.7146	0.6697	0.5372
attention_dmiip_fdu	0.6914	0.7123	0.6863	0.6835	0.6966	0.6029	0.5955	0.7130	0.6711	0.5368
NLM CNN	0.6295	0.6702	0.6046	0.6153	0.6258	0.4593	0.4629	0.6661	0.5968	0.4621
dmiip_fdu	0.6916	0.7132	0.6861	0.6837	0.6969	0.6028	0.5956	0.7136	0.6710	0.5372
NLM System 1	0.6651	0.6758	0.6719	0.6558	0.6535	0.5475	0.5392	0.6797	0.6511	0.5047
NLM System 3	0.6708	0.6831	0.6763	0.6613	0.6592	0.5587	0.5503	0.6858	0.6565	0.5121
pi_dna_2	0.6067	0.6476	0.5839	0.5944	0.5395	0.4684	0.4478	0.6463	0.5717	0.4406
bert_dna	0.5843	0.6312	0.5562	0.5712	0.5206	0.4348	0.4163	0.6298	0.5450	0.4177
pi_dna	0.6042	0.6461	0.5810	0.5921	0.5301	0.4691	0.4472	0.6448	0.5684	0.4381
DeepSys1	0.2078	0.2834	0.3515	0.2622	0.1039	0.2775	0.1734	0.1525	0.3261	0.1594
pi_dna_3	0.6067	0.6261	0.6006	0.5940	0.5408	0.4682	0.4478	0.6240	0.5903	0.4395

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5165	0.7903	0.7321	0.7421	0.5540	0.5090
Default MTI	0.5128	0.7507	0.7676	0.7415	0.5210	0.5331
iria-1	0.3945	0.5542	0.6805	0.5958	0.3704	0.4462
iria-mix	0.3945	0.5542	0.6805	0.5959	0.3705	0.4462
deepmesh_dmiip_fdu	0.5529	0.8127	0.7738	0.7770	0.5825	0.5498
deepmesh_dmiip_fdu_	0.5532	0.8131	0.7736	0.7771	0.5829	0.5500
attention_dmiip_fdu	0.5524	0.8107	0.7748	0.7768	0.5804	0.5505
NLM CNN	0.4963	0.7926	0.7003	0.7234	0.5443	0.4822
dmiip_fdu	0.5527	0.8111	0.7747	0.7769	0.5809	0.5507
NLM System 1	0.5317	0.7861	0.7670	0.7593	0.5549	0.5362
NLM System 3	0.5377	0.7921	0.7702	0.7638	0.5621	0.5411
pi_dna_2	0.4848	0.7788	0.6939	0.7143	0.5285	0.4732
bert_dna	0.4692	0.7698	0.6696	0.6956	0.5175	0.4545
pi_dna	0.4826	0.7781	0.6918	0.7128	0.5263	0.4710
DeepSys1	0.2444	0.3955	0.5688	0.4016	0.2407	0.3546
pi_dna_3	0.4881	0.7652	0.7103	0.7178	0.5194	0.4861

+ Test batch 2, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6370	0.6874	0.6246	0.6366	0.6560	0.5749	0.5551	0.6766	0.6018	0.4902
Default MTI	0.6352	0.6395	0.6632	0.6325	0.6147	0.6067	0.5667	0.6318	0.6385	0.4805
iria-1	0.4637	0.4096	0.5420	0.4549	0.3947	0.3870	0.3486	0.4134	0.5277	0.3069
iria-mix	0.4636	0.4096	0.5420	0.4549	0.3946	0.3869	0.3485	0.4134	0.5277	0.3069
NLM System 4	0.6623	0.7210	0.6321	0.6558	0.6985	0.4992	0.5093	0.7162	0.6160	0.5056
NLM System 2	0.6689	0.7119	0.6509	0.6626	0.7028	0.5088	0.5144	0.7092	0.6329	0.5128
deepmesh_dmiip_fdu	0.6863	0.7215	0.6757	0.6816	0.7053	0.5951	0.5893	0.7188	0.6566	0.5344
deepmesh_dmiip_fdu_	0.6863	0.7215	0.6757	0.6816	0.7053	0.5951	0.5893	0.7188	0.6566	0.5344
attention_dmiip_fdu	0.6871	0.7210	0.6770	0.6822	0.7070	0.5937	0.5882	0.7192	0.6577	0.5350
NLM CNN	0.6240	0.6763	0.5962	0.6133	0.6282	0.4460	0.4514	0.6705	0.5835	0.4596
dmiip_fdu	0.6871	0.7210	0.6770	0.6822	0.7070	0.5937	0.5882	0.7192	0.6577	0.5350
NLM System 1	0.6625	0.6861	0.6637	0.6564	0.6614	0.5379	0.5322	0.6871	0.6396	0.5048
NLM System 3	0.6724	0.7133	0.6565	0.6660	0.7003	0.5222	0.5259	0.7103	0.6384	0.5174
pi_dna_2	0.6087	0.6580	0.5872	0.6010	0.5389	0.4812	0.4601	0.6526	0.5704	0.4464
bert_dna	0.5995	0.6525	0.5741	0.5910	0.5411	0.4509	0.4336	0.6475	0.5582	0.4364
pi_dna	0.6077	0.6566	0.5873	0.6004	0.5387	0.4825	0.4606	0.6507	0.5700	0.4457
DeepSys1	0.2117	0.2822	0.3529	0.2641	0.1056	0.2823	0.1746	0.1562	0.3286	0.1603
pi_dna_3	0.6081	0.6216	0.6169	0.6006	0.5363	0.4838	0.4613	0.6145	0.6018	0.4451

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5202	0.7972	0.7222	0.7400	0.5643	0.5070
Default MTI	0.5138	0.7590	0.7572	0.7404	0.5277	0.5286
iria-1	0.4028	0.5743	0.6799	0.6081	0.3838	0.4474
iria-mix	0.4028	0.5743	0.6799	0.6081	0.3837	0.4474
NLM System 4	0.5276	0.8258	0.7261	0.7557	0.5792	0.5077
NLM System 2	0.5323	0.8196	0.7392	0.7607	0.5758	0.5186
deepmesh_dmiip_fdu	0.5498	0.8232	0.7615	0.7756	0.5863	0.5406
deepmesh_dmiip_fdu_	0.5498	0.8232	0.7615	0.7756	0.5863	0.5406
attention_dmiip_fdu	0.5499	0.8222	0.7623	0.7757	0.5859	0.5412
NLM CNN	0.4924	0.8007	0.6900	0.7214	0.5479	0.4724
dmiip_fdu	0.5499	0.8222	0.7623	0.7757	0.5859	0.5412
NLM System 1	0.5301	0.7977	0.7551	0.7588	0.5619	0.5275
NLM System 3	0.5362	0.8207	0.7439	0.7637	0.5784	0.5237
pi_dna_2	0.4874	0.7843	0.6963	0.7187	0.5341	0.4737
bert_dna	0.4796	0.7840	0.6821	0.7101	0.5302	0.4629
pi_dna	0.4861	0.7828	0.6964	0.7182	0.5314	0.4730
DeepSys1	0.2454	0.3943	0.5637	0.4015	0.2406	0.3521
pi_dna_3	0.4931	0.7598	0.7248	0.7237	0.5192	0.4956

+ Test batch 2, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6461	0.7000	0.6279	0.6429	0.6598	0.5713	0.5545	0.6871	0.6098	0.4925
Default MTI	0.6492	0.6574	0.6697	0.6459	0.6227	0.6051	0.5690	0.6483	0.6501	0.4936
iria-1	0.4792	0.4282	0.5510	0.4705	0.4086	0.3885	0.3544	0.4328	0.5367	0.3203
iria-mix	0.4792	0.4282	0.5510	0.4705	0.4085	0.3886	0.3544	0.4328	0.5367	0.3203
NLM System 4	0.6782	0.7385	0.6471	0.6723	0.7072	0.5072	0.5176	0.7323	0.6315	0.5232
NLM System 2	0.6846	0.7312	0.6640	0.6789	0.7151	0.5179	0.5259	0.7271	0.6469	0.5303
deepmesh_dmiip_fdu	0.6989	0.7396	0.6849	0.6949	0.7145	0.5928	0.5903	0.7342	0.6669	0.5484
deepmesh_dmiip_fdu_	0.6988	0.7414	0.6836	0.6949	0.7158	0.5924	0.5905	0.7357	0.6654	0.5485
attention_dmiip_fdu	0.7005	0.7394	0.6873	0.6966	0.7153	0.5922	0.5898	0.7351	0.6690	0.5504
NLM CNN	0.6408	0.6921	0.6150	0.6312	0.6312	0.4550	0.4606	0.6853	0.6017	0.4778
dmiip_fdu	0.7004	0.7413	0.6860	0.6965	0.7166	0.5917	0.5899	0.7366	0.6676	0.5505
NLM System 1	0.6787	0.7098	0.6716	0.6734	0.6736	0.5331	0.5307	0.7112	0.6491	0.5231
NLM System 3	0.6873	0.7333	0.6673	0.6813	0.7166	0.5284	0.5355	0.7287	0.6503	0.5335
pi_dna_2	0.6249	0.6760	0.6021	0.6175	0.5477	0.4819	0.4641	0.6703	0.5853	0.4625
bert_dna	0.6142	0.6683	0.5892	0.6067	0.5363	0.4530	0.4339	0.6626	0.5723	0.4520
pi_dna	0.6239	0.6745	0.6011	0.6164	0.5469	0.4845	0.4654	0.6692	0.5844	0.4614
DeepSys1	0.2233	0.3031	0.3517	0.2763	0.1112	0.2753	0.1791	0.1691	0.3287	0.1689
pi_dna_3	0.6221	0.6433	0.6253	0.6148	0.5610	0.4692	0.4572	0.6352	0.6096	0.4590

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5236	0.8070	0.7268	0.7472	0.5691	0.5097
Default MTI	0.5270	0.7729	0.7640	0.7520	0.5423	0.5392
iria-1	0.4167	0.5935	0.6887	0.6231	0.3982	0.4601
iria-mix	0.4168	0.5935	0.6886	0.6231	0.3983	0.4601
NLM System 4	0.5468	0.8342	0.7413	0.7691	0.5952	0.5284
NLM System 2	0.5506	0.8299	0.7518	0.7733	0.5925	0.5372
deepmesh_dmiip_fdu	0.5654	0.8335	0.7709	0.7863	0.6033	0.5548
deepmesh_dmiip_fdu_	0.5652	0.8348	0.7699	0.7862	0.6044	0.5537
attention_dmiip_fdu	0.5667	0.8330	0.7728	0.7873	0.6034	0.5567
NLM CNN	0.5127	0.8100	0.7073	0.7364	0.5647	0.4946
dmiip_fdu	0.5665	0.8342	0.7718	0.7873	0.6045	0.5555
NLM System 1	0.5471	0.8131	0.7627	0.7715	0.5810	0.5407
NLM System 3	0.5535	0.8318	0.7540	0.7753	0.5953	0.5404
pi_dna_2	0.5048	0.7954	0.7088	0.7316	0.5490	0.4920
bert_dna	0.4953	0.7912	0.6983	0.7233	0.5414	0.4813
pi_dna	0.5037	0.7939	0.7087	0.7309	0.5471	0.4918
DeepSys1	0.2558	0.4145	0.5622	0.4152	0.2558	0.3535
pi_dna_3	0.5088	0.7745	0.7308	0.7339	0.5377	0.5092

+ Test batch 2, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6533	0.6922	0.6506	0.6497	0.6562	0.5776	0.5564	0.6837	0.6255	0.5009
Default MTI	0.6513	0.6500	0.6845	0.6468	0.6158	0.6098	0.5700	0.6436	0.6592	0.4945
NLM System 4	0.6796	0.7262	0.6602	0.6727	0.7082	0.5192	0.5271	0.7198	0.6436	0.5247
NLM System 2	0.6854	0.7179	0.6765	0.6783	0.7131	0.5270	0.5312	0.7148	0.6584	0.5302
deepmesh_dmiip_fdu	0.7005	0.7250	0.7004	0.6949	0.7001	0.6054	0.5978	0.7224	0.6798	0.5493
attention_dmiip_fdu	0.7010	0.7233	0.7023	0.6953	0.7011	0.6041	0.5968	0.7216	0.6815	0.5495
NLM CNN	0.6458	0.6847	0.6296	0.6355	0.6382	0.4657	0.4683	0.6784	0.6161	0.4826
dmiip_fdu	0.7008	0.7234	0.7019	0.6951	0.7011	0.6036	0.5964	0.7216	0.6812	0.5492
NLM System 1	0.6790	0.7010	0.6816	0.6726	0.6750	0.5507	0.5464	0.7029	0.6567	0.5227
NLM System 3	0.6919	0.7225	0.6843	0.6845	0.7132	0.5434	0.5455	0.7199	0.6661	0.5383
pi_dna_2	0.6311	0.6718	0.6176	0.6241	0.5515	0.4939	0.4748	0.6665	0.5993	0.4702
bert_dna	0.6242	0.6754	0.6024	0.6170	0.5465	0.4624	0.4478	0.6704	0.5839	0.4625
DeepSys1	0.5683	0.5271	0.6327	0.5567	0.4705	0.3570	0.3256	0.5239	0.6210	0.3995
DeepSys2	0.5844	0.6633	0.5357	0.5696	0.6072	0.2619	0.2747	0.6611	0.5237	0.4165

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5295	0.7994	0.7450	0.7524	0.5655	0.5259
Default MTI	0.5280	0.7656	0.7771	0.7536	0.5375	0.5477
NLM System 4	0.5492	0.8295	0.7518	0.7717	0.5937	0.5361
NLM System 2	0.5523	0.8236	0.7617	0.7748	0.5915	0.5435
deepmesh_dmiip_fdu	0.5656	0.8234	0.7829	0.7867	0.5956	0.5635
attention_dmiip_fdu	0.5666	0.8234	0.7843	0.7877	0.5954	0.5651
NLM CNN	0.5179	0.8084	0.7196	0.7426	0.5639	0.5048
dmiip_fdu	0.5664	0.8234	0.7839	0.7875	0.5954	0.5648
NLM System 1	0.5479	0.8086	0.7703	0.7723	0.5792	0.5461
NLM System 3	0.5577	0.8251	0.7682	0.7789	0.5955	0.5500
pi_dna_2	0.5100	0.7946	0.7213	0.7382	0.5494	0.5013
bert_dna	0.5024	0.7965	0.7081	0.7312	0.5488	0.4886
DeepSys1	0.4781	0.6966	0.7380	0.6981	0.4753	0.5100
DeepSys2	0.4680	0.8049	0.6289	0.6835	0.5508	0.4327

+ Test batch 3, week 1

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6555	0.7086	0.6324	0.6507	0.6656	0.5641	0.5488	0.6984	0.6175	0.5032
Default MTI	0.6538	0.6594	0.6703	0.6474	0.6217	0.5957	0.5597	0.6535	0.6541	0.4958
iria-1	0.4895	0.4351	0.5635	0.4799	0.4052	0.3877	0.3513	0.4395	0.5524	0.3284
iria-mix	0.4894	0.4350	0.5634	0.4799	0.4049	0.3874	0.3511	0.4394	0.5523	0.3284
NLM System 4	0.6844	0.7354	0.6584	0.6776	0.7083	0.5133	0.5230	0.7304	0.6437	0.5287
NLM System 2	0.6894	0.7267	0.6733	0.6825	0.7116	0.5207	0.5270	0.7244	0.6576	0.5338
deepmesh_dmiip_fdu	0.7048	0.7392	0.6905	0.6989	0.7160	0.5906	0.5900	0.7365	0.6757	0.5529
deepmesh_dmiip_fdu_	0.7045	0.7407	0.6892	0.6987	0.7169	0.5906	0.5902	0.7376	0.6743	0.5526
attention_dmiip_fdu	0.7060	0.7371	0.6935	0.7000	0.7166	0.5901	0.5895	0.7358	0.6786	0.5542
NLM CNN	0.6482	0.6923	0.6246	0.6375	0.6347	0.4602	0.4650	0.6873	0.6133	0.4840
dmiip_fdu	0.7058	0.7386	0.6921	0.6997	0.7176	0.5901	0.5897	0.7368	0.6772	0.5539
NLM System 1	0.6825	0.7119	0.6750	0.6762	0.6731	0.5350	0.5329	0.7136	0.6540	0.5256
NLM System 3	0.6933	0.7343	0.6751	0.6866	0.7149	0.5299	0.5358	0.7306	0.6596	0.5393
pi_dna_2	0.6371	0.6839	0.6127	0.6286	0.5523	0.4854	0.4686	0.6804	0.5990	0.4740
bert_dna	0.6433	0.6819	0.6257	0.6349	0.5579	0.4924	0.4736	0.6782	0.6117	0.4807
pi_dna	0.6443	0.6662	0.6442	0.6372	0.5252	0.5532	0.5143	0.6606	0.6287	0.4825
DeepSys2	0.5780	0.5416	0.6340	0.5678	0.4799	0.3613	0.3309	0.5369	0.6259	0.4096

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5317	0.8122	0.7309	0.7529	0.5780	0.5162
Default MTI	0.5286	0.7732	0.7645	0.7529	0.5446	0.5398
iria-1	0.4226	0.5930	0.6949	0.6260	0.4028	0.4665
iria-mix	0.4226	0.5930	0.6949	0.6259	0.4027	0.4665
NLM System 4	0.5512	0.8339	0.7475	0.7725	0.5987	0.5340
NLM System 2	0.5547	0.8295	0.7555	0.7754	0.5965	0.5419
deepmesh_dmiip_fdu	0.5688	0.8352	0.7726	0.7888	0.6066	0.5574
deepmesh_dmiip_fdu_	0.5684	0.8361	0.7718	0.7887	0.6070	0.5563
attention_dmiip_fdu	0.5700	0.8337	0.7740	0.7890	0.6062	0.5593
NLM CNN	0.5154	0.8106	0.7103	0.7393	0.5659	0.4983
dmiip_fdu	0.5695	0.8345	0.7732	0.7889	0.6066	0.5583
NLM System 1	0.5495	0.8161	0.7613	0.7719	0.5852	0.5423
NLM System 3	0.5590	0.8343	0.7586	0.7792	0.6022	0.5452
pi_dna_2	0.5120	0.8009	0.7140	0.7384	0.5569	0.4973
bert_dna	0.5180	0.7998	0.7249	0.7436	0.5567	0.5085
pi_dna	0.5203	0.7840	0.7453	0.7476	0.5451	0.5232
DeepSys2	0.4845	0.7064	0.7347	0.7035	0.4848	0.5100

+ Test batch 3, week 2

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6445	0.6933	0.6335	0.6442	0.6580	0.5779	0.5598	0.6827	0.6104	0.4945
Default MTI	0.6457	0.6522	0.6689	0.6427	0.6249	0.6097	0.5710	0.6452	0.6461	0.4899
iria-1	0.4778	0.4261	0.5555	0.4709	0.4061	0.3844	0.3489	0.4288	0.5396	0.3200
iria-mix	0.4778	0.4260	0.5555	0.4708	0.4061	0.3844	0.3489	0.4287	0.5396	0.3199
NLM System 4	0.6889	0.7349	0.6718	0.6849	0.7074	0.5449	0.5513	0.7271	0.6544	0.5373
NLM System 2	0.6938	0.7316	0.6824	0.6902	0.7078	0.5612	0.5638	0.7284	0.6623	0.5427
deepmesh_dmiip_fdu	0.6962	0.7262	0.6925	0.6936	0.7025	0.6037	0.5962	0.7202	0.6736	0.5460
deepmesh_dmiip_fdu_	0.6962	0.7262	0.6925	0.6936	0.7025	0.6037	0.5962	0.7202	0.6736	0.5460
attention_dmiip_fdu	0.6976	0.7262	0.6948	0.6951	0.7046	0.6015	0.5942	0.7210	0.6756	0.5477
NLM CNN	0.6434	0.6890	0.6251	0.6367	0.6331	0.4566	0.4619	0.6815	0.6093	0.4827
dmiip_fdu	0.6976	0.7262	0.6948	0.6951	0.7046	0.6015	0.5942	0.7210	0.6756	0.5477
NLM System 1	0.6769	0.7122	0.6708	0.6735	0.6723	0.5344	0.5340	0.7124	0.6448	0.5227
NLM System 3	0.6947	0.7316	0.6844	0.6912	0.7060	0.5711	0.5721	0.7278	0.6645	0.5440
pi_dna_2	0.6280	0.6800	0.6086	0.6235	0.5409	0.4802	0.4598	0.6730	0.5887	0.4682
bert_dna	0.6404	0.6913	0.6228	0.6366	0.5628	0.5109	0.4925	0.6837	0.6022	0.4823
pi_dna	0.6464	0.6878	0.6370	0.6431	0.5492	0.5538	0.5240	0.6797	0.6163	0.4889
pi_dna_3	0.6397	0.6883	0.6241	0.6362	0.5558	0.5139	0.4930	0.6804	0.6035	0.4818
DeepSys2	0.5674	0.5359	0.6262	0.5599	0.4754	0.3305	0.3049	0.5290	0.6119	0.4019

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5249	0.8032	0.7315	0.7483	0.5675	0.5129
Default MTI	0.5238	0.7689	0.7626	0.7488	0.5395	0.5358
iria-1	0.4168	0.5865	0.6873	0.6186	0.3987	0.4593
iria-mix	0.4167	0.5865	0.6872	0.6185	0.3986	0.4593
NLM System 4	0.5552	0.8308	0.7575	0.7765	0.5966	0.5432
NLM System 2	0.5585	0.8267	0.7672	0.7807	0.5955	0.5494
deepmesh_dmiip_fdu	0.5602	0.8240	0.7740	0.7837	0.5931	0.5541
deepmesh_dmiip_fdu_	0.5602	0.8240	0.7740	0.7837	0.5931	0.5541
attention_dmiip_fdu	0.5619	0.8243	0.7757	0.7849	0.5936	0.5563
NLM CNN	0.5138	0.8079	0.7107	0.7382	0.5633	0.4966
dmiip_fdu	0.5619	0.8243	0.7757	0.7849	0.5936	0.5563
NLM System 1	0.5432	0.8164	0.7569	0.7692	0.5810	0.5348
NLM System 3	0.5593	0.8268	0.7688	0.7816	0.5962	0.5505
pi_dna_2	0.5053	0.7983	0.7080	0.7323	0.5514	0.4915
bert_dna	0.5137	0.8031	0.7215	0.7425	0.5575	0.5008
pi_dna	0.5188	0.7970	0.7360	0.7481	0.5537	0.5133
pi_dna_3	0.5128	0.8000	0.7224	0.7415	0.5550	0.5015
DeepSys2	0.4769	0.7008	0.7233	0.6936	0.4804	0.5014

+ Test batch 3, week 3

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6541	0.6974	0.6484	0.6545	0.6459	0.5647	0.5444	0.6862	0.6249	0.5110
Default MTI	0.6440	0.6381	0.6808	0.6393	0.6006	0.5940	0.5541	0.6307	0.6579	0.4873
iria-1	0.4758	0.4131	0.5693	0.4666	0.3820	0.3953	0.3518	0.4178	0.5523	0.3165
iria-mix	0.4757	0.4130	0.5692	0.4665	0.3820	0.3952	0.3517	0.4178	0.5522	0.3165
NLM System 4	0.6885	0.7276	0.6824	0.6846	0.7051	0.5471	0.5465	0.7140	0.6648	0.5380
NLM System 2	0.6928	0.7169	0.6976	0.6885	0.7083	0.5652	0.5605	0.7072	0.6790	0.5417
deepmesh_dmiip_fdu	0.6965	0.7170	0.7028	0.6918	0.6962	0.5950	0.5845	0.7098	0.6836	0.5451
deepmesh_dmiip_fdu_	0.6965	0.7170	0.7028	0.6918	0.6962	0.5950	0.5845	0.7098	0.6836	0.5451
attention_dmiip_fdu	0.6980	0.7147	0.7071	0.6932	0.6973	0.5940	0.5831	0.7086	0.6876	0.5464
NLM CNN	0.6424	0.6740	0.6384	0.6347	0.6315	0.4653	0.4633	0.6642	0.6220	0.4809
dmiip_fdu	0.6980	0.7147	0.7071	0.6932	0.6973	0.5940	0.5831	0.7086	0.6876	0.5464
NLM System 1	0.6756	0.6887	0.6884	0.6702	0.6646	0.5464	0.5372	0.6888	0.6630	0.5202
NLM System 3	0.6961	0.7219	0.6981	0.6914	0.7064	0.5741	0.5687	0.7135	0.6795	0.5454
pi_dna_2	0.6291	0.6651	0.6209	0.6224	0.5469	0.4906	0.4718	0.6593	0.6015	0.4682
bert_dna	0.6421	0.6740	0.6363	0.6349	0.5666	0.5138	0.4944	0.6691	0.6172	0.4816
pi_dna	0.6503	0.6747	0.6531	0.6440	0.5622	0.5582	0.5305	0.6692	0.6325	0.4907
pi_dna_3	0.6403	0.6710	0.6352	0.6336	0.5612	0.5194	0.4987	0.6665	0.6161	0.4802

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5352	0.8053	0.7441	0.7573	0.5725	0.5262
Default MTI	0.5157	0.7589	0.7732	0.7489	0.5238	0.5360
iria-1	0.4111	0.5814	0.7048	0.6225	0.3872	0.4626
iria-mix	0.4111	0.5813	0.7048	0.6225	0.3871	0.4626
NLM System 4	0.5521	0.8278	0.7645	0.7781	0.5894	0.5452
NLM System 2	0.5556	0.8204	0.7772	0.7823	0.5858	0.5545
deepmesh_dmiip_fdu	0.5549	0.8170	0.7805	0.7827	0.5839	0.5538
deepmesh_dmiip_fdu_	0.5549	0.8170	0.7805	0.7827	0.5839	0.5538
attention_dmiip_fdu	0.5564	0.8155	0.7836	0.7839	0.5834	0.5569
NLM CNN	0.5090	0.8005	0.7238	0.7415	0.5511	0.5000
dmiip_fdu	0.5564	0.8155	0.7836	0.7839	0.5834	0.5569
NLM System 1	0.5406	0.8013	0.7716	0.7698	0.5662	0.5437
NLM System 3	0.5563	0.8223	0.7773	0.7831	0.5877	0.5538
pi_dna_2	0.5012	0.7888	0.7207	0.7352	0.5395	0.4940
bert_dna	0.5096	0.7917	0.7344	0.7440	0.5437	0.5048
pi_dna	0.5193	0.7907	0.7516	0.7536	0.5464	0.5209
pi_dna_3	0.5080	0.7895	0.7342	0.7434	0.5405	0.5047

+ Test batch 3, week 4

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6491	0.7101	0.6365	0.6542	0.6636	0.5888	0.5684	0.6936	0.6100	0.5110
Default MTI	0.6402	0.6520	0.6663	0.6401	0.6066	0.6181	0.5758	0.6401	0.6404	0.4879
iria-1	0.4818	0.4271	0.5651	0.4747	0.3967	0.4015	0.3608	0.4297	0.5482	0.3235
iria-mix	0.4818	0.4271	0.5652	0.4747	0.3966	0.4015	0.3607	0.4297	0.5482	0.3235
NLM System 4	0.6836	0.7288	0.6708	0.6809	0.6985	0.5540	0.5545	0.7177	0.6525	0.5335
NLM System 2	0.6894	0.7275	0.6808	0.6861	0.7088	0.5669	0.5657	0.7196	0.6617	0.5390
deepmesh_dmiip_fdu	0.6953	0.7265	0.6957	0.6942	0.6972	0.6115	0.5991	0.7176	0.6743	0.5493
deepmesh_dmiip_fdu_	0.6966	0.7290	0.6964	0.6960	0.6990	0.6120	0.6000	0.7199	0.6749	0.5522
attention_dmiip_fdu	0.6942	0.7228	0.6958	0.6929	0.6932	0.6013	0.5897	0.7158	0.6739	0.5468
NLM CNN	0.6381	0.6798	0.6274	0.6322	0.6226	0.4708	0.4693	0.6689	0.6100	0.4784
dmiip_fdu	0.6955	0.7253	0.6965	0.6948	0.6950	0.6018	0.5906	0.7180	0.6745	0.5497
NLM System 1	0.6801	0.7121	0.6778	0.6769	0.6834	0.5621	0.5576	0.7119	0.6509	0.5280
NLM System 3	0.6999	0.7467	0.6848	0.6970	0.7185	0.5875	0.5843	0.7382	0.6654	0.5534
pi_dna_2	0.6275	0.6722	0.6178	0.6242	0.5450	0.4989	0.4796	0.6623	0.5962	0.4703
bert_dna	0.6402	0.6844	0.6307	0.6372	0.5710	0.5279	0.5100	0.6753	0.6086	0.4841
pi_dna	0.6466	0.6822	0.6440	0.6437	0.5606	0.5644	0.5378	0.6735	0.6218	0.4908
pi_dna_3	0.6404	0.6834	0.6318	0.6375	0.5649	0.5325	0.5129	0.6738	0.6101	0.4843
DeepSys2	0.5651	0.5307	0.6304	0.5579	0.4701	0.3732	0.3403	0.5226	0.6151	0.4006

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5477	0.8141	0.7452	0.7625	0.5918	0.5329
Default MTI	0.5262	0.7662	0.7739	0.7532	0.5399	0.5408
iria-1	0.4168	0.5870	0.6989	0.6237	0.3954	0.4637
iria-mix	0.4168	0.5870	0.6988	0.6237	0.3954	0.4637
NLM System 4	0.5545	0.8248	0.7612	0.7760	0.5928	0.5445
NLM System 2	0.5563	0.8250	0.7654	0.7787	0.5935	0.5477
deepmesh_dmiip_fdu	0.5682	0.8232	0.7812	0.7869	0.6014	0.5621
deepmesh_dmiip_fdu_	0.5710	0.8249	0.7817	0.7881	0.6047	0.5642
attention_dmiip_fdu	0.5637	0.8200	0.7783	0.7839	0.5948	0.5590
NLM CNN	0.5117	0.8030	0.7146	0.7373	0.5597	0.4978
dmiip_fdu	0.5665	0.8217	0.7788	0.7852	0.5981	0.5611
NLM System 1	0.5500	0.8183	0.7644	0.7744	0.5864	0.5428
NLM System 3	0.5687	0.8356	0.7717	0.7871	0.6100	0.5567
pi_dna_2	0.5074	0.7917	0.7177	0.7347	0.5466	0.4987
bert_dna	0.5182	0.8006	0.7303	0.7463	0.5573	0.5098
pi_dna	0.5225	0.7931	0.7445	0.7511	0.5535	0.5205
pi_dna_3	0.5169	0.7982	0.7318	0.7461	0.5554	0.5088
DeepSys2	0.4745	0.6963	0.7312	0.6945	0.4750	0.5039

+ Test batch 3, week 5

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.6508	0.6950	0.6415	0.6495	0.6520	0.5917	0.5686	0.6799	0.6240	0.5027
Default MTI	0.6471	0.6542	0.6724	0.6451	0.6092	0.6245	0.5840	0.6414	0.6530	0.4951
iria-1	0.4729	0.4103	0.5598	0.4618	0.3855	0.4113	0.3648	0.4155	0.5486	0.3131
NLM System 4	0.6876	0.7221	0.6784	0.6827	0.6967	0.5609	0.5577	0.7114	0.6652	0.5361
NLM System 2	0.6932	0.7207	0.6891	0.6880	0.7095	0.5758	0.5723	0.7123	0.6750	0.5413
deepmesh_dmiip_fdu	0.7010	0.7217	0.7026	0.6967	0.7032	0.6190	0.6076	0.7158	0.6868	0.5527
deepmesh_dmiip_fdu_	0.7013	0.7150	0.7088	0.6965	0.7005	0.6153	0.6032	0.7108	0.6920	0.5516
NLM CNN	0.6424	0.6772	0.6332	0.6337	0.6280	0.4818	0.4796	0.6648	0.6214	0.4803
NLM System 1	0.6799	0.6916	0.6907	0.6733	0.6642	0.5714	0.5627	0.6923	0.6680	0.5234
NLM System 3	0.7075	0.7426	0.6940	0.7009	0.7174	0.5979	0.5925	0.7373	0.6799	0.5583
pi_dna_2	0.6298	0.6583	0.6248	0.6222	0.5538	0.5100	0.4915	0.6509	0.6099	0.4679
bert_dna	0.6390	0.6705	0.6345	0.6334	0.5711	0.5329	0.5168	0.6602	0.6191	0.4802
pi_dna	0.6498	0.6723	0.6542	0.6449	0.5644	0.5713	0.5474	0.6622	0.6378	0.4925
pi_dna_3	0.6385	0.6671	0.6374	0.6333	0.5651	0.5392	0.5209	0.6565	0.6214	0.4799
DeepSys2	0.5625	0.5251	0.6261	0.5527	0.4634	0.3969	0.3583	0.5171	0.6166	0.3973

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.5333	0.8049	0.7509	0.7607	0.5741	0.5218
Default MTI	0.5289	0.7726	0.7801	0.7600	0.5445	0.5402
iria-1	0.4110	0.5782	0.6998	0.6191	0.3872	0.4609
NLM System 4	0.5558	0.8294	0.7665	0.7818	0.5912	0.5468
NLM System 2	0.5585	0.8289	0.7711	0.7844	0.5908	0.5520
deepmesh_dmiip_fdu	0.5704	0.8277	0.7867	0.7926	0.6012	0.5645
deepmesh_dmiip_fdu_	0.5675	0.8227	0.7885	0.7908	0.5941	0.5655
NLM CNN	0.5121	0.8080	0.7189	0.7423	0.5564	0.4993
NLM System 1	0.5470	0.8087	0.7766	0.7769	0.5733	0.5480
NLM System 3	0.5704	0.8377	0.7768	0.7915	0.6065	0.5610
pi_dna_2	0.5060	0.7889	0.7253	0.7381	0.5405	0.4995
bert_dna	0.5150	0.7937	0.7358	0.7470	0.5491	0.5074
pi_dna	0.5235	0.7932	0.7536	0.7567	0.5488	0.5238
pi_dna_3	0.5139	0.7902	0.7385	0.7468	0.5466	0.5085
DeepSys2	0.4728	0.6976	0.7281	0.6935	0.4724	0.5013

Platinum sponsors of the BioASQ Challenge between 2012 and 2023

Google was a proud sponsor of the BioASQ Challenge from 2018 to 2022.
BioASQ was supported by the National Library of Medicine of the National Institutes of Health under award number R13LM012214. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
The research leading to these results has received funding from the European Commission's Seventh Framework Programme (FP7/2007-2013, ICT-2011.4.4(d), Intelligent Information Management, Targeted Competition Framework) under grant agreement n° 318652.

BioASQ Participants Area

Test Results for Task 9a

+ Test batch 1, week 1

Annotated articles:7808/7967

Flat Measures

Hierarchical Measures

+ Test batch 1, week 2

Annotated articles:9987/10053

Flat Measures

Hierarchical Measures

+ Test batch 1, week 3

Annotated articles:4854/4870

Flat Measures

Hierarchical Measures

+ Test batch 1, week 4

Annotated articles:5735/5758

Flat Measures

Hierarchical Measures

+ Test batch 1, week 5

Annotated articles:5666/5770

Flat Measures

Hierarchical Measures

+ Test batch 2, week 1

Annotated articles:6374/6376

Flat Measures

Hierarchical Measures

+ Test batch 2, week 2

Annotated articles:6403/9101

Flat Measures

Hierarchical Measures

+ Test batch 2, week 3

Annotated articles:6590/7013

Flat Measures

Hierarchical Measures

+ Test batch 2, week 4

Annotated articles:5914/6070

Flat Measures

Hierarchical Measures

+ Test batch 2, week 5

Annotated articles:5904/6151

Flat Measures

Hierarchical Measures

+ Test batch 3, week 1

Annotated articles:5730/5890

Flat Measures

Hierarchical Measures

+ Test batch 3, week 2

Annotated articles:9910/10818

Flat Measures

Hierarchical Measures

+ Test batch 3, week 3

Annotated articles:3493/4022

Flat Measures

Hierarchical Measures

+ Test batch 3, week 4

Annotated articles:4005/5373

Flat Measures

Hierarchical Measures

+ Test batch 3, week 5

Annotated articles:2351/5325

Flat Measures

Hierarchical Measures