BioASQ Participants Area

Task 10b: Test Results of Phase B

The test results are presented in separate tables for each type of annotation. The "System Description" of each system is used.
The evaluation measures that are used in Task B are presented here .
Warning: For ideal answers, good ROUGE results do not always imply good manual scores.

Test batch 1

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
bio-answerfinder	1.0000	1.0000	1.0000	1.0000	0.3529	0.3824	0.3603	0.5552	0.6107	0.5639
bio-answerfinder-2	1.0000	1.0000	1.0000	1.0000	0.3529	0.3824	0.3603	0.5552	0.6107	0.5639
Knowledge graph QA	0.6087	0.7097	0.4000	0.5548	0.2647	0.4412	0.3480	0.4481	0.3964	0.3540
LaRSA	0.9565	0.9697	0.9231	0.9464	0.2941	0.5588	0.4118	0.5714	0.4821	0.4959
UDEL-LAB1	0.9565	0.9714	0.9091	0.9403	0.3529	0.5588	0.4299	0.6974	0.8226	0.7346
UDEL-LAB2	0.9565	0.9714	0.9091	0.9403	0.3824	0.5588	0.4534	0.7201	0.8405	0.7469
UDEL-LAB3	0.9565	0.9714	0.9091	0.9403	0.2941	0.4412	0.3578	0.6893	0.7429	0.6731
UDEL-LAB4	0.9565	0.9714	0.9091	0.9403	0.3235	0.4412	0.3775	0.6762	0.8464	0.7229
UDEL-LAB5	0.9565	0.9714	0.9091	0.9403	0.3824	0.5588	0.4608	0.4714	0.8310	0.5810
MQ-1	0.7391	0.8500	-	0.4250	-	-	-	-	-	-
MQ-2	0.7391	0.8500	-	0.4250	-	-	-	-	-	-
KU-AAA637-system1	0.9130	0.9412	0.8333	0.8873	0.3529	0.5294	0.4167	0.5629	0.5774	0.5310
Ir_sys1	0.9565	0.9697	0.9231	0.9464	0.4118	0.5000	0.4559	0.5714	0.4702	0.4887
Ir_sys2	0.9130	0.9412	0.8333	0.8873	0.3529	0.5000	0.4176	0.5595	0.4738	0.4948
Ir_sys3	0.6957	0.8000	0.3636	0.5818	0.4118	0.5000	0.4485	0.6224	0.4881	0.5238
lalala	0.9565	0.9697	0.9231	0.9464	0.3824	0.5000	0.4363	0.6046	0.7286	0.6459
bioinfo-0	0.4783	0.4545	0.5000	0.4773	-	-	-	-	-	-
Ir_sys4	0.9565	0.9697	0.9231	0.9464	0.2353	0.4118	0.3039	0.7143	0.5619	0.6048
bioinfo-1	0.6957	0.7407	0.6316	0.6862	-	-	-	-	-	-

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
bio-answerfinder	0.4785	0.3723	0.4818	0.3706	-	-	-	-
bio-answerfinder-2	0.4022	0.3706	0.4094	0.3715	-	-	-	-
Knowledge graph QA	0.2380	0.1717	0.2434	0.1717	-	-	-	-
LaRSA	-	-	-	-	-	-	-	-
UDEL-LAB1	-	-	-	-	-	-	-	-
UDEL-LAB2	-	-	-	-	-	-	-	-
UDEL-LAB3	-	-	-	-	-	-	-	-
UDEL-LAB4	-	-	-	-	-	-	-	-
UDEL-LAB5	-	-	-	-	-	-	-	-
MQ-1	0.5611	0.3596	0.5640	0.3490	-	-	-	-
MQ-2	0.5433	0.3429	0.5481	0.3339	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
Ir_sys1	0.5611	0.3500	0.5642	0.3399	-	-	-	-
Ir_sys2	0.5467	0.3427	0.5507	0.3337	-	-	-	-
Ir_sys3	0.0923	0.1314	0.0704	0.0999	-	-	-	-
lalala	0.5601	0.3526	0.5620	0.3423	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
Ir_sys4	0.2532	0.2285	0.2509	0.2223	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-

Test batch 2

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
simple baseline solr	0.6111	0.7200	0.3636	0.5418	0.3824	0.3824	0.3824	0.2000	0.0611	0.0902
AUEB-System1	0.6667	0.8000	-	0.4000	0.3529	0.4706	0.4069	0.3967	0.2324	0.2663
AUEB-System2	0.6667	0.8000	-	0.4000	0.2941	0.4706	0.3701	0.1400	0.1574	0.1299
AUEB-System3	0.6667	0.8000	-	0.4000	0.3824	0.4412	0.4069	0.2533	0.2407	0.2191
AUEB-System4	0.6667	0.8000	-	0.4000	0.2353	0.4412	0.3284	0.2867	0.2204	0.2172
NCU-IISR/AS-GIS-1	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
NCU-IISR/AS-GIS-2	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
NCU-IISR/AS-GIS-3	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
bio-answerfinder	0.9444	0.9600	0.9091	0.9345	0.4118	0.4706	0.4412	0.5254	0.4157	0.4485
bio-answerfinder-2	0.9444	0.9600	0.9091	0.9345	0.4118	0.4706	0.4412	0.5254	0.4157	0.4485
orpheus_kg	1.0000	1.0000	1.0000	1.0000	0.3529	0.3529	0.3529	0.3833	0.3546	0.3657
LaRSA	0.3889	0.1538	0.5217	0.3378	0.4412	0.5882	0.5098	0.6143	0.4281	0.4490
MQ-1	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
MQ-2	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
UDEL-LAB1	1.0000	1.0000	1.0000	1.0000	0.5588	0.6176	0.5809	0.6707	0.6530	0.6393
UDEL-LAB2	1.0000	1.0000	1.0000	1.0000	0.5588	0.6176	0.5882	0.6914	0.7193	0.6787
UDEL-LAB3	1.0000	1.0000	1.0000	1.0000	0.5588	0.6765	0.6000	0.7042	0.7400	0.7051
UDEL-LAB4	1.0000	1.0000	1.0000	1.0000	0.5588	0.6765	0.5912	0.6859	0.7530	0.7011
KU-AAA637-system1	0.9444	0.9600	0.9091	0.9345	0.5000	0.6176	0.5451	0.4975	0.4170	0.3949
KU-AAA637-system2	0.9444	0.9600	0.9091	0.9345	0.5000	0.5882	0.5392	0.5324	0.4337	0.4210
KU-AAA637-system3	0.9444	0.9600	0.9091	0.9345	0.5000	0.6176	0.5451	0.5324	0.4337	0.4210
KU-AAA637-system4	0.9444	0.9600	0.9091	0.9345	0.5294	0.6176	0.5613	0.5324	0.4337	0.4210
KU-AAA637-system5	0.9444	0.9600	0.9091	0.9345	0.5294	0.5882	0.5539	0.5324	0.4337	0.4210
bioinfo-0	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
bioinfo-1	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
bioinfo-2	0.6667	0.7273	0.5714	0.6494	-	-	-	-	-	-
bioinfo-3	0.4444	0.5000	0.3750	0.4375	-	-	-	-	-	-
bioinfo-4	0.5000	0.5263	0.4706	0.4985	-	-	-	-	-	-
NCU-1	0.7778	0.8571	0.5000	0.6786	-	-	-	-	-	-
simple truncation	0.6667	0.8000	-	0.4000	-	-	-	-	-	-
Ir_sys1	1.0000	1.0000	1.0000	1.0000	0.5294	0.5882	0.5539	0.5785	0.4004	0.4247
Ir_sys2	0.8333	0.8889	0.6667	0.7778	0.5294	0.5882	0.5588	0.5893	0.4763	0.4956
Ir_sys3	0.8333	0.8889	0.6667	0.7778	0.5588	0.6176	0.5809	0.6226	0.4281	0.4504
Ir_sys4	0.7222	0.7826	0.6154	0.6990	0.5588	0.5882	0.5686	0.6149	0.4504	0.4722
lalala	0.7222	0.7826	0.6154	0.6990	0.5588	0.6176	0.5745	0.4955	0.6067	0.5177

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
simple baseline solr	0.4537	0.2755	0.4605	0.2660	-	-	-	-
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
NCU-IISR/AS-GIS-1	0.4534	0.4058	0.4619	0.3951	-	-	-	-
NCU-IISR/AS-GIS-2	0.4629	0.4164	0.4725	0.4072	-	-	-	-
NCU-IISR/AS-GIS-3	0.4728	0.4262	0.4823	0.4168	-	-	-	-
bio-answerfinder	0.5552	0.3807	0.5618	0.3719	-	-	-	-
bio-answerfinder-2	0.4796	0.4161	0.4846	0.4074	-	-	-	-
orpheus_kg	0.1256	0.1544	0.1208	0.1492	-	-	-	-
LaRSA	-	-	-	-	-	-	-	-
MQ-1	0.6057	0.3614	0.6178	0.3484	-	-	-	-
MQ-2	0.6064	0.3608	0.6182	0.3480	-	-	-	-
UDEL-LAB1	-	-	-	-	-	-	-	-
UDEL-LAB2	-	-	-	-	-	-	-	-
UDEL-LAB3	-	-	-	-	-	-	-	-
UDEL-LAB4	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
NCU-1	0.4677	0.4093	0.4765	0.3991	-	-	-	-
simple truncation	0.7117	0.3628	0.7163	0.3462	-	-	-	-
Ir_sys1	0.6169	0.3672	0.6264	0.3522	-	-	-	-
Ir_sys2	0.6152	0.3647	0.6299	0.3521	-	-	-	-
Ir_sys3	0.6077	0.3599	0.6213	0.3463	-	-	-	-
Ir_sys4	0.1040	0.1573	0.0795	0.1235	-	-	-	-
lalala	0.2931	0.2570	0.3017	0.2581	-	-	-	-

Test batch 3

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
AUEB-System1	0.7200	0.8372	-	0.4186	0.3125	0.4688	0.3724	0.1818	0.1584	0.1292
AUEB-System2	0.7200	0.8372	-	0.4186	0.4063	0.4063	0.4063	0.1409	0.1662	0.1421
AUEB-System3	0.7200	0.8372	-	0.4186	0.3125	0.5000	0.4010	0.1152	0.1351	0.1065
AUEB-System4	0.7200	0.8372	-	0.4186	0.4688	0.4688	0.4688	0.1273	0.0766	0.0669
AUEB-System5	0.7200	0.8372	-	0.4186	0.3125	0.5000	0.3880	0.1227	0.1403	0.1128
bio-answerfinder	0.9200	0.9444	0.8571	0.9008	0.3438	0.3750	0.3594	0.6273	0.4472	0.4843
bio-answerfinder-2	0.9200	0.9444	0.8571	0.9008	0.3438	0.3750	0.3594	0.6273	0.4472	0.4843
LaRSA	0.8800	0.9231	0.7273	0.8252	0.5000	0.6563	0.5677	0.4923	0.4128	0.4052
MQ-1	0.7200	0.8372	-	0.4186	-	-	-	-	-	-
MQ-2	0.7200	0.8372	-	0.4186	-	-	-	-	-	-
NCU-1	0.8400	0.9000	0.6000	0.7500	0.3438	0.3438	0.3438	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-1	0.8400	0.9000	0.6000	0.7500	0.3438	0.3438	0.3438	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-2	0.8400	0.9000	0.6000	0.7500	0.3438	0.3438	0.3438	0.3030	0.1608	0.1971
NCU-IISR/AS-GIS-3	0.8400	0.9000	0.6000	0.7500	0.3438	0.3438	0.3438	0.3030	0.1608	0.1971
BioASQ-2022_UNCC	0.8400	0.8947	0.6667	0.7807	0.4375	0.6563	0.5167	-	-	-
UDEL-LAB1	1.0000	1.0000	1.0000	1.0000	0.5313	0.6250	0.5729	0.5442	0.6742	0.5655
UDEL-LAB2	1.0000	1.0000	1.0000	1.0000	0.4688	0.6250	0.5417	0.5174	0.6591	0.5558
UDEL-LAB4	1.0000	1.0000	1.0000	1.0000	0.5000	0.5938	0.5417	0.5263	0.5985	0.5188
UDEL-LAB3	1.0000	1.0000	1.0000	1.0000	0.4688	0.5938	0.5141	0.5293	0.6439	0.5255
UDEL-LAB5	1.0000	1.0000	1.0000	1.0000	0.5313	0.6563	0.5792	0.5447	0.5682	0.5094
KU-AAA637-system1	0.9200	0.9444	0.8571	0.9008	0.4688	0.6875	0.5505	0.5203	0.4794	0.4201
KU-AAA637-system2	0.9600	0.9714	0.9333	0.9524	0.5000	0.6875	0.5661	0.5152	0.4794	0.4192
KU-AAA637-system3	0.9600	0.9714	0.9333	0.9524	0.5000	0.6875	0.5651	0.5152	0.4794	0.4192
KU-AAA637-system4	0.9200	0.9444	0.8571	0.9008	0.5000	0.6875	0.5755	0.5195	0.4794	0.4212
KU-AAA637-system5	0.9200	0.9444	0.8571	0.9008	0.4063	0.6875	0.5115	0.5126	0.4613	0.4083
ELECTROLBERT-2	0.7600	0.8571	0.2500	0.5536	-	-	-	-	-	-
bioinfo-0	0.6800	0.7500	0.5556	0.6528	-	-	-	-	-	-
bioinfo-1	0.7600	0.8333	0.5714	0.7024	-	-	-	-	-	-
bioinfo-2	0.7200	0.7742	0.6316	0.7029	-	-	-	-	-	-
bioinfo-3	0.7200	0.8108	0.4615	0.6362	-	-	-	-	-	-
bioinfo-4	0.6800	0.7500	0.5556	0.6528	-	-	-	-	-	-
new kgqa for yesno	0.6800	0.8000	0.2000	0.5000	0.3125	0.5000	0.3828	0.1955	0.2333	0.2030
extractive	0.7200	0.8372	-	0.4186	-	-	-	-	-	-
Ir_sys1	0.9600	0.9714	0.9333	0.9524	0.4375	0.6563	0.5260	0.5325	0.4552	0.4567
Ir_sys2	0.9600	0.9714	0.9333	0.9524	0.4063	0.6250	0.4839	0.5189	0.3697	0.4000
Ir_sys3	0.8000	0.8718	0.5455	0.7086	0.4688	0.6250	0.5313	0.5038	0.3652	0.3758
Ir_sys4	0.8000	0.8718	0.5455	0.7086	0.5000	0.5938	0.5339	0.5195	0.3392	0.3740
lalala	0.9600	0.9714	0.9333	0.9524	0.4688	0.5938	0.5219	0.4418	0.6076	0.4769

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
bio-answerfinder	0.4915	0.3369	0.5086	0.3335	-	-	-	-
bio-answerfinder-2	0.3902	0.3595	0.4073	0.3586	-	-	-	-
LaRSA	0.4616	0.2877	0.4663	0.2735	-	-	-	-
MQ-1	0.5759	0.3492	0.5818	0.3344	-	-	-	-
MQ-2	0.5696	0.3456	0.5771	0.3316	-	-	-	-
NCU-1	0.4193	0.3747	0.4248	0.3678	-	-	-	-
NCU-IISR/AS-GIS-1	0.3947	0.3597	0.3953	0.3481	-	-	-	-
NCU-IISR/AS-GIS-2	0.4193	0.3761	0.4236	0.3689	-	-	-	-
NCU-IISR/AS-GIS-3	0.4055	0.3629	0.4116	0.3561	-	-	-	-
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
UDEL-LAB1	-	-	-	-	-	-	-	-
UDEL-LAB2	-	-	-	-	-	-	-	-
UDEL-LAB4	-	-	-	-	-	-	-	-
UDEL-LAB3	-	-	-	-	-	-	-	-
UDEL-LAB5	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
ELECTROLBERT-2	-	-	-	-	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
new kgqa for yesno	0.1933	0.1430	0.2025	0.1396	-	-	-	-
extractive	0.1413	0.1110	0.1374	0.1058	-	-	-	-
Ir_sys1	0.5825	0.3573	0.5802	0.3391	-	-	-	-
Ir_sys2	0.5851	0.3543	0.5948	0.3387	-	-	-	-
Ir_sys3	0.2868	0.2499	0.3000	0.2532	-	-	-	-
Ir_sys4	0.5509	0.3374	0.5592	0.3238	-	-	-	-
lalala	0.5851	0.3543	0.5948	0.3387	-	-	-	-

Test batch 4

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
AUEB-System1	0.7083	0.8293	-	0.4146	0.2903	0.4516	0.3495	0.2917	0.1285	0.1571
AUEB-System2	0.7083	0.8293	-	0.4146	0.3226	0.4516	0.3710	0.1000	0.0917	0.0933
AUEB-System3	0.7083	0.8293	-	0.4146	0.2903	0.4516	0.3602	0.1694	0.1424	0.1446
AUEB-System4	0.7083	0.8293	-	0.4146	0.2903	0.4839	0.3710	0.2250	0.2118	0.2032
AUEB-System5	0.7083	0.8293	-	0.4146	0.2903	0.5484	0.3855	0.2222	0.1563	0.1741
extractive	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
ELECTROLBERT-2	0.7500	0.8500	0.2500	0.5500	-	-	-	-	-	-
bio-answerfinder	0.9583	0.9714	0.9231	0.9473	0.3548	0.4194	0.3871	0.3727	0.2701	0.2733
bio-answerfinder-2	0.9583	0.9714	0.9231	0.9473	0.3548	0.4194	0.3871	0.3727	0.2701	0.2733
bioinfo-0	0.7917	0.8387	0.7059	0.7723	-	-	-	-	-	-
bioinfo-1	0.7917	0.8718	0.4444	0.6581	-	-	-	-	-	-
bioinfo-2	0.8750	0.9032	0.8235	0.8634	-	-	-	-	-	-
bioinfo-3	0.8333	0.8947	0.6000	0.7474	-	-	-	-	-	-
bioinfo-4	0.7917	0.8387	0.7059	0.7723	-	-	-	-	-	-
LaRSA	0.9583	0.9714	0.9231	0.9473	0.4516	0.6129	0.5129	0.4736	0.3104	0.3048
BioASQ-2022_UNCC	0.9167	0.9412	0.8571	0.8992	0.5161	0.6129	0.5645	-	-	-
KU-AAA637-system1	0.9583	0.9714	0.9231	0.9473	0.4839	0.6452	0.5495	0.4491	0.3326	0.3068
KU-AAA637-system2	0.9583	0.9714	0.9231	0.9473	0.4839	0.6129	0.5430	0.4750	0.3535	0.3268
KU-AAA637-system3	0.9583	0.9714	0.9231	0.9473	0.4839	0.6452	0.5495	0.4491	0.3118	0.2846
KU-AAA637-system4	0.9583	0.9714	0.9231	0.9473	0.5161	0.6452	0.5656	0.4819	0.3660	0.3537
KU-AAA637-system5	0.9583	0.9714	0.9231	0.9473	0.4516	0.5806	0.5108	0.5083	0.4146	0.3669
MQ-1	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
MQ-2	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
UDEL-LAB1	0.9583	0.9697	0.9333	0.9515	0.4839	0.6129	0.5387	0.5799	0.5017	0.4950
UDEL-LAB2	0.9583	0.9697	0.9333	0.9515	0.4839	0.6129	0.5484	0.5834	0.5844	0.5386
UDEL-LAB3	1.0000	1.0000	1.0000	1.0000	0.5161	0.6129	0.5484	0.5584	0.4438	0.4501
UDEL-LAB4	1.0000	1.0000	1.0000	1.0000	0.5484	0.6129	0.5613	0.6162	0.4753	0.4752
NCU-1	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
UDEL-LAB5	1.0000	1.0000	1.0000	1.0000	0.5161	0.5806	0.5484	0.6132	0.4426	0.4434
NCU-IISR/AS-GIS-1	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
NCU-IISR/AS-GIS-2	0.8333	0.8947	0.6000	0.7474	0.3548	0.3548	0.3548	0.4500	0.2801	0.3190
simple truncation	0.7083	0.8293	-	0.4146	-	-	-	-	-	-
Ir_sys1	1.0000	1.0000	1.0000	1.0000	0.4839	0.6452	0.5495	0.4444	0.2410	0.2747
NCU-IISR-AS-GIS-4	0.8333	0.8947	0.6000	0.7474	0.3226	0.3226	0.3226	0.3778	0.3122	0.3224
NCU-IISR-AS-GIS-5	0.8333	0.8947	0.6000	0.7474	0.3226	0.3226	0.3226	0.3778	0.3053	0.3200
Ir_sys2	1.0000	1.0000	1.0000	1.0000	0.4516	0.5161	0.4839	0.3889	0.2847	0.2718
Ir_sys3	0.9583	0.9714	0.9231	0.9473	0.5161	0.6774	0.5806	0.4583	0.3243	0.3314
Ir_sys4	0.9583	0.9714	0.9231	0.9473	0.4516	0.5806	0.5161	0.5083	0.2826	0.2841
lalala	1.0000	1.0000	1.0000	1.0000	0.5806	0.6452	0.5995	0.4089	0.4507	0.3835

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
extractive	0.1901	0.1500	0.1907	0.1472	-	-	-	-
ELECTROLBERT-2	-	-	-	-	-	-	-	-
bio-answerfinder	0.5086	0.3726	0.5078	0.3628	-	-	-	-
bio-answerfinder-2	0.4260	0.4080	0.4222	0.3978	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
LaRSA	0.5458	0.4132	0.5482	0.4022	-	-	-	-
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
MQ-1	0.5696	0.3610	0.5858	0.3525	-	-	-	-
MQ-2	0.5752	0.3655	0.5884	0.3556	-	-	-	-
UDEL-LAB1	-	-	-	-	-	-	-	-
UDEL-LAB2	-	-	-	-	-	-	-	-
UDEL-LAB3	-	-	-	-	-	-	-	-
UDEL-LAB4	-	-	-	-	-	-	-	-
NCU-1	0.4040	0.4026	0.4064	0.3941	-	-	-	-
UDEL-LAB5	-	-	-	-	-	-	-	-
NCU-IISR/AS-GIS-1	0.2784	0.2731	0.2797	0.2673	-	-	-	-
NCU-IISR/AS-GIS-2	0.4179	0.4229	0.4157	0.4165	-	-	-	-
simple truncation	-	-	-	-	-	-	-	-
Ir_sys1	0.5507	0.3531	0.5631	0.3450	-	-	-	-
NCU-IISR-AS-GIS-4	0.4040	0.4026	0.4064	0.3941	-	-	-	-
NCU-IISR-AS-GIS-5	0.4040	0.4026	0.4064	0.3941	-	-	-	-
Ir_sys2	0.1316	0.1441	0.1413	0.1505	-	-	-	-
Ir_sys3	0.2506	0.2163	0.2502	0.2142	-	-	-	-
Ir_sys4	0.5382	0.3488	0.5555	0.3418	-	-	-	-
lalala	0.5415	0.3471	0.5540	0.3393	-	-	-	-

Test batch 5

Exact Answers

	Yes/No				Factoid			List
System	Accuracy	F1 Yes	F1 No	Macro F1	Strict Acc.	Lenient Acc.	MRR	Mean Prec.	Recall	F-Measure
extractive	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
simple truncation	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
bio-answerfinder	0.7857	0.8235	0.7273	0.7754	0.3103	0.3448	0.3276	0.3253	0.3101	0.3072
bio-answerfinder-2	0.7857	0.8235	0.7273	0.7754	0.3103	0.3448	0.3276	0.3253	0.3101	0.3072
AUEB-System1	0.5357	0.6977	-	0.3488	-	-	-	0.0278	0.0278	0.0278
AUEB-System2	0.5357	0.6977	-	0.3488	0.3103	0.3448	0.3218	0.1222	0.1012	0.0891
AUEB-System3	0.5357	0.6977	-	0.3488	-	-	-	0.0278	0.0278	0.0278
AUEB-System4	0.5357	0.6977	-	0.3488	0.2414	0.3448	0.2931	0.1037	0.0734	0.0714
AUEB-System5	0.5357	0.6977	-	0.3488	0.3103	0.3793	0.3448	0.0870	0.1012	0.0847
bioinfo-0	0.7857	0.7692	0.8000	0.7846	-	-	-	-	-	-
bioinfo-1	0.7500	0.7586	0.7407	0.7497	-	-	-	-	-	-
bioinfo-2	0.8214	0.8148	0.8276	0.8212	-	-	-	-	-	-
bioinfo-3	0.8214	0.8276	0.8148	0.8212	-	-	-	-	-	-
bioinfo-4	0.7857	0.7692	0.8000	0.7846	-	-	-	-	-	-
LaRSA	0.7500	0.8000	0.6667	0.7333	0.4138	0.5517	0.4540	0.4486	0.4188	0.4191
ELECTROLBERT-3	0.6429	0.7500	0.3750	0.5625	-	-	-	-	-	-
UDEL-LAB1	0.8929	0.9091	0.8696	0.8893	0.3103	0.5172	0.3977	0.6003	0.5795	0.5707
UDEL-LAB2	0.8929	0.9091	0.8696	0.8893	0.2759	0.5172	0.3661	0.5587	0.6297	0.5793
UDEL-LAB3	0.8929	0.9091	0.8696	0.8893	0.4138	0.5862	0.4828	0.5651	0.5929	0.5669
UDEL-LAB4	0.8929	0.9091	0.8696	0.8893	0.2759	0.5862	0.4017	0.6009	0.6313	0.6016
UDEL-LAB5	0.8929	0.9091	0.8696	0.8893	0.3793	0.5172	0.4190	0.5742	0.6407	0.5956
BioASQ-2022_UNCC	0.8929	0.9032	0.8800	0.8916	0.0345	0.6207	0.1805	-	-	-
BioASQ-2022_UNCC1	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4764	-	-	-
MQ-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
MQ-2	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
BioASQ-2022_UNCC2	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4764	-	-	-
BioASQ-2022_UNCC3	0.8929	0.9032	0.8800	0.8916	0.4138	0.6207	0.4764	-	-	-
KU-AAA637-system1	0.8929	0.9091	0.8696	0.8893	0.3103	0.4828	0.3718	0.5275	0.4969	0.4774
KU-AAA637-system2	0.9286	0.9375	0.9167	0.9271	0.3103	0.4828	0.3776	0.5747	0.5246	0.5174
KU-AAA637-system3	0.8929	0.9091	0.8696	0.8893	0.3103	0.5172	0.3805	0.5190	0.4922	0.4953
KU-AAA637-system4	0.8571	0.8824	0.8182	0.8503	0.3448	0.4828	0.3920	0.5269	0.5246	0.4902
KU-AAA637-system5	0.8929	0.9091	0.8696	0.8893	0.3103	0.4828	0.3764	0.5201	0.5469	0.4974
NCU-IISR/AS-GIS-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
NCU-1	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
NCU-IISR/AS-GIS-2	0.5357	0.6977	-	0.3488	-	-	-	-	-	-
Ir_sys1	0.9286	0.9333	0.9231	0.9282	0.3448	0.5172	0.4195	0.6799	0.4722	0.5224
Ir_sys2	0.9286	0.9333	0.9231	0.9282	0.3103	0.5517	0.4149	0.6204	0.4660	0.5099
Ir_sys3	0.8929	0.9032	0.8800	0.8916	0.4828	0.5862	0.5098	0.5852	0.4120	0.4620
Ir_sys4	0.7857	0.8235	0.7273	0.7754	0.3448	0.4483	0.3776	0.6058	0.4367	0.4706
lalala	0.9286	0.9333	0.9231	0.9282	0.3448	0.5517	0.4218	0.5140	0.5714	0.5256

Ideal Answers

	Automatic scores (Rouge - R)				Manual scores
System	R-2 (Rec)	R-2 (F1)	R-SU4 (Rec)	R-SU4 (F1)	Readability	Recall	Precision	Repetition
extractive	-	-	-	-	-	-	-	-
simple truncation	0.4429	0.3028	0.4441	0.2927	-	-	-	-
bio-answerfinder	0.4296	0.3361	0.4270	0.3270	-	-	-	-
bio-answerfinder-2	0.3815	0.3435	0.3793	0.3357	-	-	-	-
AUEB-System1	-	-	-	-	-	-	-	-
AUEB-System2	-	-	-	-	-	-	-	-
AUEB-System3	-	-	-	-	-	-	-	-
AUEB-System4	-	-	-	-	-	-	-	-
AUEB-System5	-	-	-	-	-	-	-	-
bioinfo-0	-	-	-	-	-	-	-	-
bioinfo-1	-	-	-	-	-	-	-	-
bioinfo-2	-	-	-	-	-	-	-	-
bioinfo-3	-	-	-	-	-	-	-	-
bioinfo-4	-	-	-	-	-	-	-	-
LaRSA	0.4926	0.3500	0.4975	0.3423	-	-	-	-
ELECTROLBERT-3	-	-	-	-	-	-	-	-
UDEL-LAB1	-	-	-	-	-	-	-	-
UDEL-LAB2	-	-	-	-	-	-	-	-
UDEL-LAB3	-	-	-	-	-	-	-	-
UDEL-LAB4	-	-	-	-	-	-	-	-
UDEL-LAB5	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC1	-	-	-	-	-	-	-	-
MQ-1	0.5385	0.3513	0.5417	0.3415	-	-	-	-
MQ-2	0.5352	0.3539	0.5365	0.3431	-	-	-	-
BioASQ-2022_UNCC2	-	-	-	-	-	-	-	-
BioASQ-2022_UNCC3	-	-	-	-	-	-	-	-
KU-AAA637-system1	-	-	-	-	-	-	-	-
KU-AAA637-system2	-	-	-	-	-	-	-	-
KU-AAA637-system3	-	-	-	-	-	-	-	-
KU-AAA637-system4	-	-	-	-	-	-	-	-
KU-AAA637-system5	-	-	-	-	-	-	-	-
NCU-IISR/AS-GIS-1	0.3962	0.3707	0.3928	0.3626	-	-	-	-
NCU-1	0.3829	0.3606	0.3803	0.3559	-	-	-	-
NCU-IISR/AS-GIS-2	0.4270	0.4020	0.4208	0.3916	-	-	-	-
Ir_sys1	0.5508	0.3540	0.5452	0.3389	-	-	-	-
Ir_sys2	0.6071	0.3212	0.5984	0.3050	-	-	-	-
Ir_sys3	0.2263	0.1830	0.2337	0.1892	-	-	-	-
Ir_sys4	0.4841	0.3149	0.4899	0.3071	-	-	-	-
lalala	0.5305	0.3390	0.5263	0.3244	-	-	-	-