BioASQ Participants Area

Test Results for Task 4a

The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.

+ Dry Run

Annotated articles:1389/2877

Flat Measures

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
BioASQ GrantID	0.0002	0.0003	0.0002	0.0002	0.0004	0.0000	0.0001	0.0004	0.0001	0.0001
MTI First Line Index	0.5510	0.6286	0.5178	0.5478	0.5599	0.4758	0.4595	0.6184	0.4969	0.3926
Default MTI	0.5660	0.5965	0.5671	0.5637	0.5322	0.5214	0.4920	0.5910	0.5431	0.4071
auth1	0.0871	0.6767	0.0523	0.0956	0.6767	0.0002	0.0002	0.6767	0.0466	0.0523

Hierarchical Measures

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
BioASQ GrantID	0.0981	0.1608	0.0879	0.1014	0.1554	0.0827
MTI First Line Index	0.4504	0.7651	0.6452	0.6796	0.5129	0.4262
Default MTI	0.4641	0.7350	0.6922	0.6948	0.4934	0.4630
auth1	0.0678	0.7031	0.0595	0.1084	0.2825	0.0393

+ Test batch 1, week 1

Annotated articles:1853/3740

Flat Measures

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.5702	0.6432	0.5341	0.5609	0.5862	0.4765	0.4666	0.6365	0.5163	0.4067
Default MTI	0.5795	0.6129	0.5740	0.5710	0.5580	0.5161	0.4933	0.6067	0.5547	0.4161
auth1	0.5940	0.6059	0.5965	0.5826	0.6086	0.4562	0.4480	0.6094	0.5793	0.4254
MeSHLabeler	0.6282	0.6682	0.6139	0.6201	0.6260	0.5210	0.5139	0.6661	0.5944	0.4654
MeSHLabeler-1	0.6190	0.6612	0.6033	0.6114	0.6092	0.5120	0.5044	0.6605	0.5824	0.4559

Hierarchical Measures

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R
MTI First Line Index	0.4519	0.7721	0.6508	0.6828	0.5144	0.4301
Default MTI	0.4634	0.7461	0.6899	0.6949	0.4987	0.4621
auth1	0.4678	0.7388	0.7013	0.6995	0.4982	0.4684
MeSHLabeler	0.4926	0.7842	0.7102	0.7250	0.5367	0.4820
MeSHLabeler-1	0.4825	0.7804	0.6980	0.7168	0.5301	0.4694

+ Test batch 1, week 2

Annotated articles:1578/2872

Flat Measures

System	MiF	EBP	EBR	EBF	MaP	MaR	MaF	MiP	MiR	Acc.
MTI First Line Index	0.5852	0.6527	0.5480	0.5734	0.5769	0.4750	0.4593	0.6469	0.5343	0.4204
Default MTI	0.5928	0.6228	0.5853	0.5825	0.5504	0.5149	0.4874	0.6184	0.5692	0.4288
auth1	0.6050	0.6143	0.6017	0.5912	0.5906	0.4465	0.4368	0.6222	0.5887	0.4361
auth2	0.6060	0.6550	0.5672	0.5907	0.6226	0.4215	0.4190	0.6660	0.5559	0.4367
MeSHLabeler	0.6424	0.6829	0.6228	0.6333	0.6252	0.5157	0.5069	0.6843	0.6054	0.4807
MeSHLabeler-1	0.6332	0.6738	0.6129	0.6239	0.6055	0.5102	0.4982	0.6765	0.5951	0.4710
BioASQ Filtering	0.5958	0.6124	0.5977	0.5839	0.5594	0.4753	0.4565	0.6087	0.5836	0.4288

Hierarchical Measures

System	LCA-F	HiP	HiR	HiF	LCA-P	LCA-R