BioASQ Participants Area

Task 10b: Test Results of Phase B

The test results are presented in separate tables for each type of annotation. The "System Description" of each system is used.
The evaluation measures that are used in Task B are presented here .
Warning: For ideal answers, good ROUGE results do not always imply good manual scores.

Test batch 1

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
bio-answerfinder	1.0000	1.0000	1.0000	1.0000	0.3824	0.4118	0.3897	0.5552	0.6107	0.5639
bio-answerfinder-2	1.0000	1.0000	1.0000	1.0000	0.3824	0.4118	0.3897	0.5552	0.6107	0.5639
Knowledge graph QA	0.6087	0.7097	0.4000	0.5548	0.2647	0.4412	0.3480	0.4481	0.3964	0.3540
orpheus_kg	0.9565	0.9714	0.9091	0.9403	0.1471	0.1471	0.1471	0.4233	0.4321	0.4240
LaRSA	0.9565	0.9697	0.9231	0.9464	0.2941	0.5588	0.4118	0.5714	0.4821	0.4959
UDEL-LAB1	0.9565	0.9714	0.9091	0.9403	0.3529	0.5882	0.4397	0.6974	0.8226	0.7346
UDEL-LAB2	0.9565	0.9714	0.9091	0.9403	0.3824	0.5588	0.4534	0.7201	0.8405	0.7469
UDEL-LAB3	0.9565	0.9714	0.9091	0.9403	0.2941	0.4412	0.3578	0.6893	0.7429	0.6731
UDEL-LAB4	0.9565	0.9714	0.9091	0.9403	0.3235	0.4412	0.3775	0.6762	0.8464	0.7229
UDEL-LAB5	0.9565	0.9714	0.9091	0.9403	0.3824	0.5882	0.4706	0.4714	0.8310	0.5810
MQ-1	0.7391	0.8500	-	0.4250	-	-	-	-	-	-
MQ-2	0.7391	0.8500	-	0.4250	-	-	-	-	-	-
AUEB-System1	0.7391	0.8500	-	0.4250	0.2941	0.3529	0.3235	0.1143	0.0786	0.0905
AUEB-System2	0.7391	0.8500	-	0.4250	0.2941	0.3824	0.3333	0.0536	0.0429	0.0476
AUEB-System3	0.7391	0.8500	-	0.4250	0.2941	0.3824	0.3221	0.0571	0.0786	0.0633
AUEB-System4	0.7391	0.8500	-	0.4250	0.2941	0.4118	0.3392	0.0607	0.0786	0.0667
simple baseline solr	0.6522	0.7895	-	0.3947	0.2941	0.2941	0.2941	-	-	-
KU-AAA637-system1	0.9130	0.9412	0.8333	0.8873	0.3529	0.5294	0.4167	0.5629	0.5774	0.5310
Ir_sys1	0.9565	0.9697	0.9231	0.9464	0.4118	0.5000	0.4559	0.5714	0.4702	0.4887
Ir_sys2	0.9130	0.9412	0.8333	0.8873	0.3529	0.5000	0.4176	0.5595	0.4738	0.4948
Ir_sys3	0.6957	0.8000	0.3636	0.5818	0.4118	0.5294	0.4559	0.6224	0.4881	0.5238
lalala	0.9565	0.9697	0.9231	0.9464	0.3824	0.5000	0.4363	0.6046	0.7286	0.6459
bioinfo-0	0.4783	0.4545	0.5000	0.4773	-	-	-	-	-	-
Ir_sys4	0.9565	0.9697	0.9231	0.9464	0.2353	0.4118	0.3039	0.7143	0.5619	0.6048
bioinfo-1	0.6957	0.7407	0.6316	0.6862	-	-	-	-	-	-
BioASQ_Baseline	0.4783	0.4545	0.5000	0.4773	0.0588	0.2647	0.1186	0.3016	0.3750	0.2650

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
bio-answerfinder	0.5057	0.3894	0.5063	0.3851	4.40	4.42	4.31	4.62
bio-answerfinder-2	0.4303	0.3913	0.4339	0.3894	4.57	4.29	4.36	4.92
Knowledge graph QA	0.2491	0.1732	0.2526	0.1715	2.07	2.31	2.19	2.34
orpheus_kg	0.0807	0.0970	0.0808	0.0961	4.42	4.18	4.56	4.92
LaRSA	-	-	-	-	-	-	-	-
UDEL-LAB1	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB2	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB3	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB4	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB5	-	-	-	-	0.38	0.38	0.38	0.38
MQ-1	0.5665	0.3511	0.5705	0.3418	4.14	4.51	4.01	4.29
MQ-2	0.5526	0.3406	0.5561	0.3316	4.13	4.49	4.02	4.27
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
simple baseline solr	0.0694	0.1103	0.0508	0.0829	4.07	3.44	4.41	4.94
KU-AAA637-system1	-	-	-	-	-	-	-	-
Ir_sys1	0.5791	0.3512	0.5833	0.3414	4.13	4.56	4.07	4.28
Ir_sys2	0.5569	0.3383	0.5614	0.3299	4.16	4.57	4.03	4.28
Ir_sys3	0.0877	0.1284	0.0652	0.0954	4.04	3.80	4.37	4.91
lalala	0.5748	0.3464	0.5773	0.3367	4.17	4.47	4.10	4.23
bioinfo-0	-	-	-	-	-	-	-	-
Ir_sys4	0.2651	0.2396	0.2620	0.2329	4.06	3.41	3.69	4.70
bioinfo-1	-	-	-	-	-	-	-	-
BioASQ_Baseline	-	-	-	-	-	-	-	-

Test batch 2

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
simple baseline solr	0.6111	0.7200	0.3636	0.5418	0.4118	0.4118	0.4118	0.2000	0.0611	0.0902
AUEB-System1	0.6667	0.8000	-	0.4000	0.3824	0.5000	0.4363	0.3967	0.2324	0.2663
AUEB-System2	0.6667	0.8000	-	0.4000	0.3235	0.5000	0.3995	0.1400	0.1574	0.1299
AUEB-System3	0.6667	0.8000	-	0.4000	0.4118	0.4706	0.4363	0.2533	0.2407	0.2191
AUEB-System4	0.6667	0.8000	-	0.4000	0.2647	0.4706	0.3578	0.2867	0.2204	0.2172
NCU-IISR/AS-GIS-1	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
NCU-IISR/AS-GIS-2	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
NCU-IISR/AS-GIS-3	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
bio-answerfinder	0.9444	0.9600	0.9091	0.9345	0.4118	0.4706	0.4412	0.5254	0.4157	0.4485
bio-answerfinder-2	0.9444	0.9600	0.9091	0.9345	0.4118	0.4706	0.4412	0.5254	0.4157	0.4485
orpheus_kg	1.0000	1.0000	1.0000	1.0000	0.3529	0.3529	0.3529	0.3833	0.3546	0.3657
LaRSA	0.3889	0.1538	0.5217	0.3378	0.4412	0.5882	0.5098	0.6143	0.4281	0.4490
MQ-1	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
MQ-2	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
UDEL-LAB1	1.0000	1.0000	1.0000	1.0000	0.5882	0.6471	0.6103	0.6707	0.6530	0.6393
UDEL-LAB2	1.0000	1.0000	1.0000	1.0000	0.5882	0.6471	0.6176	0.6914	0.7193	0.6787
UDEL-LAB3	1.0000	1.0000	1.0000	1.0000	0.5882	0.6765	0.6235	0.7042	0.7400	0.7051
UDEL-LAB4	1.0000	1.0000	1.0000	1.0000	0.5882	0.7059	0.6206	0.6859	0.7530	0.7011
KU-AAA637-system1	0.9444	0.9600	0.9091	0.9345	0.5294	0.6471	0.5745	0.4975	0.4170	0.3949
KU-AAA637-system2	0.9444	0.9600	0.9091	0.9345	0.5294	0.6176	0.5686	0.5324	0.4337	0.4210
KU-AAA637-system3	0.9444	0.9600	0.9091	0.9345	0.5294	0.6471	0.5745	0.5324	0.4337	0.4210
KU-AAA637-system4	0.9444	0.9600	0.9091	0.9345	0.5588	0.6471	0.5907	0.5324	0.4337	0.4210
KU-AAA637-system5	0.9444	0.9600	0.9091	0.9345	0.5588	0.6176	0.5833	0.5324	0.4337	0.4210
bioinfo-0	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
bioinfo-1	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
bioinfo-2	0.6667	0.7273	0.5714	0.6494	-	-	-	-	-	-
bioinfo-3	0.4444	0.5000	0.3750	0.4375	-	-	-	-	-	-
bioinfo-4	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
NCU-1	0.7778	0.8571	0.5000	0.6786	-	-	-	-	-	-
simple truncation	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
Ir_sys1	1.0000	1.0000	1.0000	1.0000	0.5588	0.6176	0.5833	0.5785	0.4004	0.4247
Ir_sys2	0.8333	0.8889	0.6667	0.7778	0.5588	0.6176	0.5882	0.5893	0.4763	0.4956
Ir_sys3	0.8333	0.8889	0.6667	0.7778	0.5882	0.6471	0.6103	0.6226	0.4281	0.4504
Ir_sys4	0.7222	0.7826	0.6154	0.6990	0.5882	0.6176	0.5980	0.6149	0.4504	0.4722
lalala	0.7222	0.7826	0.6154	0.6990	0.5882	0.6471	0.6039	0.4955	0.6067	0.5177
BioASQ_Baseline	0.4444	0.3750	0.5000	0.4375	0.0882	0.2647	0.1382	0.2892	0.4019	0.3053

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
simple baseline solr	0.4464	0.2606	0.4547	0.2515	3.63	4.06	3.46	3.87
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
NCU-IISR/AS-GIS-1	0.4704	0.4151	0.4772	0.4040	4.61	4.49	4.48	4.94
NCU-IISR/AS-GIS-2	0.4840	0.4283	0.4907	0.4177	4.53	4.48	4.46	4.90
NCU-IISR/AS-GIS-3	0.4824	0.4276	0.4897	0.4173	4.52	4.48	4.44	4.89
bio-answerfinder	0.5531	0.3683	0.5600	0.3609	4.27	4.59	4.19	4.46
bio-answerfinder-2	0.4749	0.4031	0.4805	0.3963	4.51	4.44	4.37	4.89
orpheus_kg	0.1411	0.1714	0.1332	0.1636	4.47	4.29	4.44	4.92
LaRSA	-	-	-	-	-	-	-	-
MQ-1	0.6053	0.3387	0.6184	0.3256	4.29	4.70	4.08	4.34
MQ-2	0.6079	0.3387	0.6207	0.3258	4.26	4.70	4.08	4.37
UDEL-LAB1	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB2	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB3	-	-	-	-	0.38	0.38	0.38	0.38
UDEL-LAB4	-	-	-	-	0.38	0.38	0.38	0.38
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
NCU-1	0.4881	0.4187	0.4943	0.4073	4.57	4.52	4.48	4.93
simple truncation	0.7085	0.3414	0.7128	0.3251	4.04	4.90	3.82	4.08
Ir_sys1	0.6085	0.3421	0.6169	0.3272	4.28	4.77	4.09	4.33
Ir_sys2	0.6128	0.3441	0.6282	0.3317	4.24	4.70	4.10	4.33
Ir_sys3	0.6026	0.3388	0.6153	0.3254	4.23	4.76	4.10	4.31
Ir_sys4	0.1114	0.1654	0.0864	0.1318	3.86	3.39	3.92	4.73
lalala	0.3131	0.2698	0.3210	0.2699	4.12	3.46	3.68	4.44
BioASQ_Baseline	-	-	-	-	-	-	-	-

Test batch 3

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
AUEB-System1	0.7600	0.8636	-	0.4318	0.3750	0.5313	0.4349	0.1818	0.1584	0.1292
AUEB-System2	0.7600	0.8636	-	0.4318	0.4688	0.4688	0.4688	0.1409	0.1662	0.1421
AUEB-System3	0.7600	0.8636	-	0.4318	0.3438	0.5313	0.4323	0.1152	0.1351	0.1065
AUEB-System4	0.7600	0.8636	-	0.4318	0.5000	0.5313	0.5156	0.1273	0.0766	0.0669
AUEB-System5	0.7600	0.8636	-	0.4318	0.3750	0.5625	0.4505	0.1227	0.1403	0.1128
bio-answerfinder	0.8800	0.9189	0.7692	0.8441	0.3750	0.4063	0.3906	0.6273	0.4472	0.4843
bio-answerfinder-2	0.8800	0.9189	0.7692	0.8441	0.3750	0.4063	0.3906	0.6273	0.4472	0.4843
LaRSA	0.9200	0.9500	0.8000	0.8750	0.5313	0.6875	0.5990	0.4923	0.4128	0.4052
MQ-1	0.7600	0.8636	-	0.4318	-	-	-	-	-	-
MQ-2	0.7600	0.8636	-	0.4318	-	-	-	-	-	-
NCU-1	0.8800	0.9268	0.6667	0.7967	0.4063	0.4063	0.4063	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-1	0.8800	0.9268	0.6667	0.7967	0.4063	0.4063	0.4063	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-2	0.8800	0.9268	0.6667	0.7967	0.4063	0.4063	0.4063	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-3	0.8800	0.9268	0.6667	0.7967	0.4063	0.4063	0.4063	0.3030	0.1608	0.1971
BioASQ-2022_UNCC	0.8800	0.9231	0.7273	0.8252	0.5000	0.6875	0.5714	-	-	-
UDEL-LAB1	0.9600	0.9730	0.9231	0.9480	0.5625	0.6563	0.6042	0.5442	0.6742	0.5655
UDEL-LAB2	0.9600	0.9730	0.9231	0.9480	0.5313	0.6563	0.5885	0.5174	0.6591	0.5558
UDEL-LAB4	0.9600	0.9730	0.9231	0.9480	0.5313	0.6250	0.5729	0.5263	0.5985	0.5188
UDEL-LAB3	0.9600	0.9730	0.9231	0.9480	0.5000	0.6250	0.5469	0.5293	0.6439	0.5255
UDEL-LAB5	0.9600	0.9730	0.9231	0.9480	0.5625	0.6563	0.5948	0.5447	0.5682	0.5094
KU-AAA637-system1	0.8800	0.9189	0.7692	0.8441	0.5000	0.6875	0.5729	0.5203	0.4794	0.4201
KU-AAA637-system2	0.9200	0.9444	0.8571	0.9008	0.5313	0.6875	0.5911	0.5152	0.4794	0.4192
KU-AAA637-system3	0.9200	0.9444	0.8571	0.9008	0.5313	0.6875	0.5885	0.5152	0.4794	0.4192
KU-AAA637-system4	0.8800	0.9189	0.7692	0.8441	0.5313	0.6875	0.5964	0.5195	0.4794	0.4212
KU-AAA637-system5	0.8800	0.9189	0.7692	0.8441	0.4375	0.6875	0.5323	0.5126	0.4613	0.4083
Fleming-3	0.8000	0.8837	0.2857	0.5847	-	-	-	-	-	-
bioinfo-0	0.7200	0.7879	0.5882	0.6881	-	-	-	-	-	-
bioinfo-1	0.8000	0.8649	0.6154	0.7401	-	-	-	-	-	-
bioinfo-2	0.6800	0.7500	0.5556	0.6528	-	-	-	-	-	-
bioinfo-3	0.7600	0.8421	0.5000	0.6711	-	-	-	-	-	-
bioinfo-4	0.7200	0.7879	0.5882	0.6881	-	-	-	-	-	-
new kgqa for yesno	0.7200	0.8293	0.2222	0.5257	0.3438	0.5625	0.4245	0.1955	0.2333	0.2030
extractive	0.7600	0.8636	-	0.4318	-	-	-	-	-	-
Ir_sys1	0.9200	0.9444	0.8571	0.9008	0.4688	0.6875	0.5573	0.5325	0.4552	0.4567
Ir_sys2	0.9200	0.9444	0.8571	0.9008	0.4375	0.6563	0.5151	0.5189	0.3697	0.4000
Ir_sys3	0.8400	0.9000	0.6000	0.7500	0.5000	0.6563	0.5625	0.5038	0.3652	0.3758
Ir_sys4	0.8400	0.9000	0.6000	0.7500	0.5313	0.6250	0.5703	0.5195	0.3392	0.3740
lalala	0.9200	0.9444	0.8571	0.9008	0.5000	0.6250	0.5573	0.4418	0.6076	0.4769
BioASQ_Baseline	0.2400	0.0952	0.3448	0.2200	0.1563	0.4063	0.2526	0.1338	0.3478	0.1752

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
bio-answerfinder	0.4721	0.3219	0.4879	0.3190	4.30	4.64	4.26	4.58
bio-answerfinder-2	0.3752	0.3471	0.3925	0.3474	4.61	4.61	4.43	4.97
LaRSA	0.4935	0.3222	0.4977	0.3086	4.54	4.40	3.90	4.73
MQ-1	0.5446	0.3253	0.5508	0.3119	4.14	4.74	4.07	4.26
MQ-2	0.5328	0.3179	0.5392	0.3046	4.17	4.76	4.10	4.23
NCU-1	0.4072	0.3696	0.4106	0.3613	4.58	4.48	4.49	4.94
NCU-IISR/AS-GIS-1	0.3787	0.3508	0.3786	0.3395	4.62	4.40	4.46	4.99
NCU-IISR/AS-GIS-2	0.4054	0.3690	0.4083	0.3611	4.58	4.46	4.49	4.97
NCU-IISR/AS-GIS-3	0.3910	0.3553	0.3959	0.3480	4.62	4.42	4.44	4.97
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
UDEL-LAB1	-	-	-	-	0.36	0.36	0.36	0.36
UDEL-LAB2	-	-	-	-	0.36	0.36	0.36	0.36
UDEL-LAB4	-	-	-	-	0.36	0.36	0.36	0.36
UDEL-LAB3	-	-	-	-	0.36	0.36	0.36	0.36
UDEL-LAB5	-	-	-	-	0.36	0.36	0.36	0.36
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
Fleming-3	-	-	-	-	1.28	0.96	1.19	1.39
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
new kgqa for yesno	0.1960	0.1455	0.2060	0.1430	1.89	2.19	1.94	2.11
extractive	0.1304	0.0979	0.1274	0.0939	0.80	1.09	1.01	1.08
Ir_sys1	0.5603	0.3408	0.5585	0.3238	4.11	4.72	4.06	4.28
Ir_sys2	0.5677	0.3403	0.5778	0.3258	4.13	4.80	4.08	4.29
Ir_sys3	0.2892	0.2512	0.3020	0.2548	4.09	3.81	3.63	4.64
Ir_sys4	0.5241	0.3178	0.5334	0.3060	4.11	4.74	4.08	4.24
lalala	0.5677	0.3403	0.5778	0.3258	4.13	4.80	4.08	4.29
BioASQ_Baseline	-	-	-	-	-	-	-	-

Test batch 4

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
AUEB-System1	0.7083	0.8293	-	0.4146	0.2903	0.4516	0.3495	0.2917	0.1285	0.1571
AUEB-System2	0.7083	0.8293	-	0.4146	0.3226	0.4516	0.3710	0.1000	0.0917	0.0933
AUEB-System3	0.7083	0.8293	-	0.4146	0.2903	0.4516	0.3602	0.1694	0.1424	0.1446
AUEB-System4	0.7083	0.8293	-	0.4146	0.2903	0.4839	0.3710	0.2250	0.2118	0.2032
AUEB-System5	0.7083	0.8293	-	0.4146	0.2903	0.5484	0.3855	0.2222	0.1563	0.1741
extractive	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
Fleming-3	0.7500	0.8500	0.2500	0.5500	-	-	-	-	-	-
bio-answerfinder	0.9583	0.9714	0.9231	0.9473	0.3548	0.4194	0.3871	0.3727	0.2701	0.2733
bio-answerfinder-2	0.9583	0.9714	0.9231	0.9473	0.3548	0.4194	0.3871	0.3727	0.2701	0.2733
bioinfo-0	0.7917	0.8387	0.7059	0.7723	-	-	-	-	-	-
bioinfo-1	0.7917	0.8718	0.4444	0.6581	-	-	-	-	-	-
bioinfo-2	0.8750	0.9032	0.8235	0.8634	-	-	-	-	-	-
bioinfo-3	0.8333	0.8947	0.6000	0.7474	-	-	-	-	-	-
bioinfo-4	0.7917	0.8387	0.7059	0.7723	-	-	-	-	-	-
LaRSA	0.9583	0.9714	0.9231	0.9473	0.4516	0.6129	0.5129	0.4736	0.3104	0.3048
BioASQ-2022_UNCC	0.9167	0.9412	0.8571	0.8992	0.5161	0.6129	0.5645	-	-	-
KU-AAA637-system1	0.9583	0.9714	0.9231	0.9473	0.4839	0.6452	0.5495	0.4491	0.3326	0.3068
KU-AAA637-system2	0.9583	0.9714	0.9231	0.9473	0.4839	0.6129	0.5430	0.4750	0.3535	0.3268
KU-AAA637-system3	0.9583	0.9714	0.9231	0.9473	0.4839	0.6452	0.5495	0.4491	0.3118	0.2846
KU-AAA637-system4	0.9583	0.9714	0.9231	0.9473	0.5161	0.6452	0.5656	0.4819	0.3660	0.3537
KU-AAA637-system5	0.9583	0.9714	0.9231	0.9473	0.4516	0.5806	0.5108	0.5083	0.4146	0.3669
MQ-1	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
MQ-2	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
UDEL-LAB1	0.9583	0.9697	0.9333	0.9515	0.4839	0.6129	0.5387	0.5799	0.5017	0.4950
UDEL-LAB2	0.9583	0.9697	0.9333	0.9515	0.4839	0.6129	0.5484	0.5834	0.5844	0.5386
UDEL-LAB3	1.0000	1.0000	1.0000	1.0000	0.5161	0.6129	0.5484	0.5584	0.4438	0.4501
UDEL-LAB4	1.0000	1.0000	1.0000	1.0000	0.5484	0.6129	0.5613	0.6162	0.4753	0.4752
NCU-1	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
UDEL-LAB5	1.0000	1.0000	1.0000	1.0000	0.5161	0.5806	0.5484	0.6132	0.4426	0.4434
NCU-IISR/AS-GIS-1	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
NCU-IISR/AS-GIS-2	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
simple truncation	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
Ir_sys1	1.0000	1.0000	1.0000	1.0000	0.4839	0.6452	0.5495	0.4444	0.2410	0.2747
NCU-IISR-AS-GIS-4	0.8333	0.8947	0.6000	0.7474	0.3226	0.3226	0.3226	0.3778	0.3122	0.3224
NCU-IISR-AS-GIS-5	0.8333	0.8947	0.6000	0.7474	0.3226	0.3226	0.3226	0.3778	0.3053	0.3200
Ir_sys2	1.0000	1.0000	1.0000	1.0000	0.4516	0.5161	0.4839	0.3889	0.2847	0.2718
Ir_sys3	0.9583	0.9714	0.9231	0.9473	0.5161	0.6774	0.5806	0.4583	0.3243	0.3314
Ir_sys4	0.9583	0.9714	0.9231	0.9473	0.4516	0.5806	0.5161	0.5083	0.2826	0.2841
lalala	1.0000	1.0000	1.0000	1.0000	0.5806	0.6452	0.5995	0.4089	0.4507	0.3835
BioASQ_Baseline	0.2917	0.1905	0.3704	0.2804	0.1613	0.3226	0.2177	0.2163	0.4035	0.2582

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
extractive	0.1809	0.1414	0.1811	0.1386	1.14	1.24	1.12	1.23
Fleming-3	-	-	-	-	1.10	0.90	1.11	1.33
bio-answerfinder	0.5208	0.3812	0.5183	0.3703	4.37	4.44	4.37	4.42
bio-answerfinder-2	0.4459	0.4249	0.4411	0.4142	4.63	4.36	4.61	4.98
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
LaRSA	0.5568	0.4127	0.5584	0.4013	4.41	4.68	4.27	4.59
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
MQ-1	0.5818	0.3613	0.5970	0.3520	4.09	4.69	4.00	4.31
MQ-2	0.5825	0.3636	0.5958	0.3536	4.08	4.69	4.03	4.30
UDEL-LAB1	-	-	-	-	0.34	0.34	0.34	0.34
UDEL-LAB2	-	-	-	-	0.34	0.34	0.34	0.34
UDEL-LAB3	-	-	-	-	0.34	0.34	0.34	0.34
UDEL-LAB4	-	-	-	-	0.34	0.34	0.34	0.34
NCU-1	0.4351	0.4301	0.4367	0.4213	4.58	4.44	4.69	5.00
UDEL-LAB5	-	-	-	-	0.34	0.34	0.34	0.34
NCU-IISR/AS-GIS-1	0.2842	0.2747	0.2868	0.2706	4.43	4.04	4.13	4.96
NCU-IISR/AS-GIS-2	0.4375	0.4419	0.4349	0.4355	4.62	4.49	4.70	5.00
simple truncation	-	-	-	-	-	-	-	-
Ir_sys1	0.5648	0.3540	0.5762	0.3449	4.21	4.70	4.01	4.34
NCU-IISR-AS-GIS-4	0.4351	0.4301	0.4367	0.4213	4.58	4.44	4.69	5.00
NCU-IISR-AS-GIS-5	0.4351	0.4301	0.4367	0.4213	4.58	4.44	4.69	5.00
Ir_sys2	0.1315	0.1439	0.1406	0.1501	4.62	3.63	4.32	4.86
Ir_sys3	0.2489	0.2124	0.2506	0.2125	4.22	3.46	3.77	4.67
Ir_sys4	0.5459	0.3464	0.5630	0.3390	4.12	4.63	3.97	4.31
lalala	0.5531	0.3467	0.5658	0.3388	4.14	4.60	3.99	4.29
BioASQ_Baseline	-	-	-	-	-	-	-	-

Test batch 5

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
simple truncation	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
bio-answerfinder	0.7857	0.8235	0.7273	0.7754	0.3448	0.4138	0.3736	0.3438	0.3290	0.3248
bio-answerfinder-2	0.7857	0.8235	0.7273	0.7754	0.3448	0.4138	0.3736	0.3438	0.3290	0.3248
AUEB-System1	0.5357	0.6977	-	0.3488	-	-	-	0.0278	0.0278	0.0278
AUEB-System2	0.5357	0.6977	-	0.3488	0.3103	0.3448	0.3218	0.1222	0.1012	0.0891
AUEB-System3	0.5357	0.6977	-	0.3488	-	-	-	0.0278	0.0278	0.0278
AUEB-System4	0.5357	0.6977	-	0.3488	0.2414	0.3793	0.3000	0.1037	0.0734	0.0714
AUEB-System5	0.5357	0.6977	-	0.3488	0.3103	0.4138	0.3534	0.0870	0.1012	0.0847
bioinfo-0	0.7857	0.7692	0.8000	0.7846	-	-	-	-	-	-
bioinfo-1	0.7500	0.7586	0.7407	0.7497	-	-	-	-	-	-
bioinfo-2	0.8214	0.8148	0.8276	0.8212	-	-	-	-	-	-
bioinfo-3	0.8214	0.8276	0.8148	0.8212	-	-	-	-	-	-
bioinfo-4	0.7857	0.7692	0.8000	0.7846	-	-	-	-	-	-
LaRSA	0.7500	0.8000	0.6667	0.7333	0.4138	0.5517	0.4626	0.4486	0.4188	0.4191
Fleming-4	0.6429	0.7500	0.3750	0.5625	-	-	-	-	-	-
UDEL-LAB1	0.8929	0.9091	0.8696	0.8893	0.3448	0.5172	0.4149	0.6082	0.5921	0.5794
UDEL-LAB2	0.8929	0.9091	0.8696	0.8893	0.3103	0.5172	0.3833	0.5642	0.6424	0.5860
UDEL-LAB3	0.8929	0.9091	0.8696	0.8893	0.4483	0.5862	0.5000	0.5790	0.6043	0.5793
UDEL-LAB4	0.8929	0.9091	0.8696	0.8893	0.3103	0.5862	0.4190	0.6120	0.6427	0.6123
UDEL-LAB5	0.8929	0.9091	0.8696	0.8893	0.3793	0.5172	0.4190	0.5881	0.6515	0.6076
BioASQ-2022_UNCC	0.8929	0.9032	0.8800	0.8916	0.1034	0.6207	0.2253	-	-	-
BioASQ-2022_UNCC1	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4868	-	-	-
MQ-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
MQ-2	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
BioASQ-2022_UNCC2	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4868	-	-	-
BioASQ-2022_UNCC3	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4868	-	-	-
KU-AAA637-system1	0.8929	0.9091	0.8696	0.8893	0.3448	0.4828	0.3994	0.5386	0.5095	0.4885
KU-AAA637-system2	0.9286	0.9375	0.9167	0.9271	0.3448	0.4828	0.4052	0.5858	0.5373	0.5285
KU-AAA637-system3	0.8929	0.9091	0.8696	0.8893	0.3448	0.5172	0.4080	0.5283	0.5049	0.5054
KU-AAA637-system4	0.8571	0.8824	0.8182	0.8503	0.3793	0.5172	0.4264	0.5361	0.5373	0.4999
KU-AAA637-system5	0.8929	0.9091	0.8696	0.8893	0.3448	0.5172	0.4109	0.5312	0.5595	0.5085
NCU-IISR/AS-GIS-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
NCU-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
NCU-IISR/AS-GIS-2	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
NCU-IISR-AS-GIS-4	0.8929	0.9091	0.8696	0.8893	0.4828	0.5862	0.5259	0.6500	0.5114	0.5517
NCU-IISR-AS-GIS-5	0.8929	0.9091	0.8696	0.8893	0.4828	0.5862	0.5259	0.6500	0.5114	0.5517
Ir_sys1	0.9286	0.9333	0.9231	0.9282	0.3793	0.5517	0.4540	0.6799	0.4716	0.5214
Ir_sys2	0.9286	0.9333	0.9231	0.9282	0.3448	0.5862	0.4494	0.6204	0.4660	0.5099
Ir_sys3	0.8929	0.9032	0.8800	0.8916	0.4828	0.5862	0.5098	0.5852	0.4114	0.4613
Ir_sys4	0.7857	0.8235	0.7273	0.7754	0.3793	0.4483	0.4138	0.6058	0.4354	0.4692
lalala	0.9286	0.9333	0.9231	0.9282	0.3793	0.5517	0.4494	0.5209	0.5840	0.5340
BioASQ_Baseline	0.4643	0.2857	0.5714	0.4286	0.0345	0.1034	0.0632	0.2125	0.4801	0.2572

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
simple truncation	0.4725	0.3202	0.4722	0.3091	4.23	4.71	3.92	4.53
bio-answerfinder	0.4367	0.3440	0.4369	0.3377	4.42	4.62	4.29	4.69
bio-answerfinder-2	0.3924	0.3537	0.3926	0.3485	4.47	4.57	4.38	4.91
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
LaRSA	0.5098	0.3646	0.5156	0.3572	4.56	4.78	4.14	4.60
Fleming-4	-	-	-	-	1.46	0.92	1.16	1.56
UDEL-LAB1	-	-	-	-	0.32	0.32	0.32	0.32
UDEL-LAB2	-	-	-	-	0.32	0.32	0.32	0.32
UDEL-LAB3	-	-	-	-	0.32	0.32	0.32	0.32
UDEL-LAB4	-	-	-	-	0.32	0.32	0.32	0.32
UDEL-LAB5	-	-	-	-	0.32	0.32	0.32	0.32
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC1	-	-	-	-	-	-	-	-
MQ-1	0.5446	0.3558	0.5511	0.3478	4.34	4.80	4.07	4.42
MQ-2	0.5416	0.3579	0.5460	0.3489	4.34	4.76	4.06	4.46
BioASQ-2022_UNCC2	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC3	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
NCU-IISR/AS-GIS-1	0.4154	0.3904	0.4133	0.3836	4.61	4.56	4.54	4.98
NCU-1	0.4050	0.3838	0.4030	0.3800	4.64	4.54	4.56	4.99
NCU-IISR/AS-GIS-2	0.4367	0.4118	0.4336	0.4049	4.64	4.61	4.60	4.99
NCU-IISR-AS-GIS-4	-	-	-	-	-	-	-	-
NCU-IISR-AS-GIS-5	-	-	-	-	-	-	-	-
Ir_sys1	0.5523	0.3555	0.5502	0.3426	4.34	4.76	4.01	4.36
Ir_sys2	0.6242	0.3262	0.6169	0.3101	4.10	4.68	3.91	3.91
Ir_sys3	0.2425	0.2019	0.2499	0.2069	4.23	3.57	3.40	4.56
Ir_sys4	0.4891	0.3198	0.4971	0.3134	4.40	4.73	4.08	4.53
lalala	0.5307	0.3381	0.5307	0.3260	4.29	4.71	3.96	4.38
BioASQ_Baseline	-	-	-	-	-	-	-	-

Test batch 6

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
Fleming-2	1.0000	1.0000	1.0000	1.0000	-	-	-	-	-	-
NCU-IISR-AS-GIS-5	0.8333	0.8571	0.8000	0.8286	0.1667	0.3333	0.2222	0.6933	0.3108	0.3458
AUEB-System1	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
AUEB-System2	0.5000	0.6667	-	0.3333	0.3333	0.5000	0.3889	0.3022	0.1953	0.2156
AUEB-System4	0.5000	0.6667	-	0.3333	0.1667	0.5000	0.3056	0.3467	0.1491	0.1912
AUEB-System5	0.5000	0.6667	-	0.3333	0.3333	0.5000	0.3889	0.2822	0.2285	0.2275
AUEB-System3	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
LaRSA	0.6667	0.7500	0.5000	0.6250	0.3333	0.3333	0.3333	0.6444	0.3985	0.4271
bio-answerfinder	0.6667	0.7500	0.5000	0.6250	0.1667	0.5000	0.3333	0.7031	0.3893	0.4405
bio-answerfinder-2	0.6667	0.7500	0.5000	0.6250	0.1667	0.5000	0.3333	0.7031	0.3893	0.4405
bioinfo-0	0.5000	0.5714	0.4000	0.4857	-	-	-	-	-	-
bioinfo-1	0.5000	0.5714	0.4000	0.4857	-	-	-	-	-	-
bioinfo-2	1.0000	1.0000	1.0000	1.0000	-	-	-	-	-	-
bioinfo-3	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
bioinfo-4	1.0000	1.0000	1.0000	1.0000	-	-	-	-	-	-
BioASQ-2022_UNCC	0.6667	0.6667	0.6667	0.6667	0.3333	0.5000	0.4167	-	-	-
MQ-1	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
MQ-2	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
NCU-IISR/AS-GIS-3	0.8333	0.8571	0.8000	0.8286	0.1667	0.3333	0.2222	0.6933	0.3108	0.3458
NCU-IISR/AS-GIS-2	0.8333	0.8571	0.8000	0.8286	0.1667	0.3333	0.2222	0.6937	0.3108	0.3487
NCU-IISR/AS-GIS-1	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
NCU-1	0.5000	0.6667	-	0.3333	-	-	-	-	-	-
KU-AAA637-system1	0.8333	0.8571	0.8000	0.8286	0.3333	0.5000	0.3750	0.5225	0.3361	0.3426
KU-AAA637-system2	0.8333	0.8571	0.8000	0.8286	0.3333	0.5000	0.4167	0.5744	0.3799	0.3917
KU-AAA637-system3	1.0000	1.0000	1.0000	1.0000	0.3333	0.3333	0.3333	0.5137	0.3572	0.3628
KU-AAA637-system4	0.8333	0.8571	0.8000	0.8286	0.3333	0.5000	0.3889	0.5507	0.3528	0.3607
KU-AAA637-system5	0.8333	0.8571	0.8000	0.8286	0.3333	0.5000	0.4167	0.4687	0.3755	0.3690
Ir_sys1	0.8333	0.8571	0.8000	0.8286	0.3333	0.6667	0.4306	0.6811	0.3851	0.4133
Ir_sys3	0.8333	0.8571	0.8000	0.8286	0.1667	0.5000	0.2917	0.5852	0.3380	0.3740
Ir_sys4	0.8333	0.8571	0.8000	0.8286	0.3333	0.5000	0.4167	0.7000	0.2971	0.3458
lalala	0.8333	0.8571	0.8000	0.8286	0.3333	0.6667	0.4222	0.5367	0.5018	0.4306
Ir_sys2	0.8333	0.8571	0.8000	0.8286	0.3333	0.6667	0.4722	0.5889	0.3356	0.3548
BioASQ_Baseline	0.5000	-	0.6667	0.3333	0.3333	0.5000	0.3667	0.2600	0.2992	0.2470

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
Fleming-2	-	-	-	-	0.70	0.38	0.70	0.70
NCU-IISR-AS-GIS-5	-	-	-	-	-	-	-	-
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
LaRSA	0.4182	0.3686	0.4369	0.3765	3.76	3.81	4.00	4.00
bio-answerfinder	0.3181	0.3103	0.3332	0.3221	4.16	3.51	4.14	4.38
bio-answerfinder-2	0.2625	0.2848	0.2757	0.2972	4.30	3.30	4.19	4.68
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
MQ-1	0.4338	0.3540	0.4558	0.3630	3.65	3.84	3.78	3.89
MQ-2	0.4382	0.3616	0.4595	0.3707	3.65	3.84	3.76	3.81
NCU-IISR/AS-GIS-3	0.2322	0.2672	0.2420	0.2772	4.14	3.05	4.08	4.76
NCU-IISR/AS-GIS-2	0.2322	0.2672	0.2420	0.2772	4.14	3.05	4.08	4.76
NCU-IISR/AS-GIS-1	0.2296	0.2605	0.2363	0.2690	4.24	3.05	4.11	4.84
NCU-1	0.2161	0.2472	0.2223	0.2554	4.16	2.97	3.97	4.73
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
Ir_sys1	0.4586	0.3754	0.4877	0.3874	3.73	3.89	3.89	3.86
Ir_sys3	0.1256	0.1286	0.1320	0.1350	3.27	1.89	2.46	3.41
Ir_sys4	0.4163	0.3430	0.4425	0.3534	3.65	3.70	3.81	3.78
lalala	0.4458	0.3641	0.4695	0.3717	3.59	3.78	3.70	3.78
Ir_sys2	0.4023	0.2834	0.4251	0.2891	2.86	3.57	3.38	3.35
BioASQ_Baseline	-	-	-	-	-	-	-	-