BioASQ Participants Area
Task 10b: Test Results of Phase B
The test results are presented in separate tables for each type of annotation. The "System Description" of each system is used.
The evaluation measures that are used in Task B are presented
here .
Warning: For ideal answers, good ROUGE results do not always imply good manual scores.
Test batch 1
Exact Answers
|
Yes/No |
Factoid |
List |
System |
Accuracy |
F1 Yes |
F1 No |
Macro F1 |
Strict Acc. |
Lenient Acc. |
MRR |
Mean Prec. |
Recall |
F-Measure |
bio-answerfinder |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.3824 |
0.4118 |
0.3897 |
0.5552 |
0.6107 |
0.5639 |
bio-answerfinder-2 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.3824 |
0.4118 |
0.3897 |
0.5552 |
0.6107 |
0.5639 |
Knowledge graph QA |
0.6087 |
0.7097 |
0.4000 |
0.5548 |
0.2647 |
0.4412 |
0.3480 |
0.4481 |
0.3964 |
0.3540 |
orpheus_kg |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.1471 |
0.1471 |
0.1471 |
0.4233 |
0.4321 |
0.4240 |
LaRSA |
0.9565 |
0.9697 |
0.9231 |
0.9464 |
0.2941 |
0.5588 |
0.4118 |
0.5714 |
0.4821 |
0.4959 |
UDEL-LAB1 |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.3529 |
0.5882 |
0.4397 |
0.6974 |
0.8226 |
0.7346 |
UDEL-LAB2 |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.3824 |
0.5588 |
0.4534 |
0.7201 |
0.8405 |
0.7469 |
UDEL-LAB3 |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.2941 |
0.4412 |
0.3578 |
0.6893 |
0.7429 |
0.6731 |
UDEL-LAB4 |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.3235 |
0.4412 |
0.3775 |
0.6762 |
0.8464 |
0.7229 |
UDEL-LAB5 |
0.9565 |
0.9714 |
0.9091 |
0.9403 |
0.3824 |
0.5882 |
0.4706 |
0.4714 |
0.8310 |
0.5810 |
MQ-1 |
0.7391 |
0.8500 |
- |
0.4250 |
- | - | - |
- | - | - |
MQ-2 |
0.7391 |
0.8500 |
- |
0.4250 |
- | - | - |
- | - | - |
AUEB-System1 |
0.7391 |
0.8500 |
- |
0.4250 |
0.2941 |
0.3529 |
0.3235 |
0.1143 |
0.0786 |
0.0905 |
AUEB-System2 |
0.7391 |
0.8500 |
- |
0.4250 |
0.2941 |
0.3824 |
0.3333 |
0.0536 |
0.0429 |
0.0476 |
AUEB-System3 |
0.7391 |
0.8500 |
- |
0.4250 |
0.2941 |
0.3824 |
0.3221 |
0.0571 |
0.0786 |
0.0633 |
AUEB-System4 |
0.7391 |
0.8500 |
- |
0.4250 |
0.2941 |
0.4118 |
0.3392 |
0.0607 |
0.0786 |
0.0667 |
simple baseline solr |
0.6522 |
0.7895 |
- |
0.3947 |
0.2941 |
0.2941 |
0.2941 |
- | - | - |
KU-AAA637-system1 |
0.9130 |
0.9412 |
0.8333 |
0.8873 |
0.3529 |
0.5294 |
0.4167 |
0.5629 |
0.5774 |
0.5310 |
Ir_sys1 |
0.9565 |
0.9697 |
0.9231 |
0.9464 |
0.4118 |
0.5000 |
0.4559 |
0.5714 |
0.4702 |
0.4887 |
Ir_sys2 |
0.9130 |
0.9412 |
0.8333 |
0.8873 |
0.3529 |
0.5000 |
0.4176 |
0.5595 |
0.4738 |
0.4948 |
Ir_sys3 |
0.6957 |
0.8000 |
0.3636 |
0.5818 |
0.4118 |
0.5294 |
0.4559 |
0.6224 |
0.4881 |
0.5238 |
lalala |
0.9565 |
0.9697 |
0.9231 |
0.9464 |
0.3824 |
0.5000 |
0.4363 |
0.6046 |
0.7286 |
0.6459 |
bioinfo-0 |
0.4783 |
0.4545 |
0.5000 |
0.4773 |
- | - | - |
- | - | - |
Ir_sys4 |
0.9565 |
0.9697 |
0.9231 |
0.9464 |
0.2353 |
0.4118 |
0.3039 |
0.7143 |
0.5619 |
0.6048 |
bioinfo-1 |
0.6957 |
0.7407 |
0.6316 |
0.6862 |
- | - | - |
- | - | - |
BioASQ_Baseline |
0.4783 |
0.4545 |
0.5000 |
0.4773 |
0.0588 |
0.2647 |
0.1186 |
0.3016 |
0.3750 |
0.2650 |
Ideal Answers
|
Automatic scores (Rouge - R) |
Manual scores |
System |
R-2 (Rec) |
R-2 (F1) |
R-SU4 (Rec) |
R-SU4 (F1) |
Readability |
Recall |
Precision |
Repetition |
bio-answerfinder |
0.5057 |
0.3894 |
0.5063 |
0.3851 |
4.40 |
4.42 |
4.31 |
4.62 |
bio-answerfinder-2 |
0.4303 |
0.3913 |
0.4339 |
0.3894 |
4.57 |
4.29 |
4.36 |
4.92 |
Knowledge graph QA |
0.2491 |
0.1732 |
0.2526 |
0.1715 |
2.07 |
2.31 |
2.19 |
2.34 |
orpheus_kg |
0.0807 |
0.0970 |
0.0808 |
0.0961 |
4.42 |
4.18 |
4.56 |
4.92 |
LaRSA |
- |
- |
- |
- |
- |
- |
- |
- |
UDEL-LAB1 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB2 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB3 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB4 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB5 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
MQ-1 |
0.5665 |
0.3511 |
0.5705 |
0.3418 |
4.14 |
4.51 |
4.01 |
4.29 |
MQ-2 |
0.5526 |
0.3406 |
0.5561 |
0.3316 |
4.13 |
4.49 |
4.02 |
4.27 |
AUEB-System1 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
- |
- |
- |
simple baseline solr |
0.0694 |
0.1103 |
0.0508 |
0.0829 |
4.07 |
3.44 |
4.41 |
4.94 |
KU-AAA637-system1 |
- |
- |
- |
- |
- |
- |
- |
- |
Ir_sys1 |
0.5791 |
0.3512 |
0.5833 |
0.3414 |
4.13 |
4.56 |
4.07 |
4.28 |
Ir_sys2 |
0.5569 |
0.3383 |
0.5614 |
0.3299 |
4.16 |
4.57 |
4.03 |
4.28 |
Ir_sys3 |
0.0877 |
0.1284 |
0.0652 |
0.0954 |
4.04 |
3.80 |
4.37 |
4.91 |
lalala |
0.5748 |
0.3464 |
0.5773 |
0.3367 |
4.17 |
4.47 |
4.10 |
4.23 |
bioinfo-0 |
- |
- |
- |
- |
- |
- |
- |
- |
Ir_sys4 |
0.2651 |
0.2396 |
0.2620 |
0.2329 |
4.06 |
3.41 |
3.69 |
4.70 |
bioinfo-1 |
- |
- |
- |
- |
- |
- |
- |
- |
BioASQ_Baseline |
- |
- |
- |
- |
- |
- |
- |
- |
Test batch 2
Exact Answers
|
Yes/No |
Factoid |
List |
System |
Accuracy |
F1 Yes |
F1 No |
Macro F1 |
Strict Acc. |
Lenient Acc. |
MRR |
Mean Prec. |
Recall |
F-Measure |
simple baseline solr |
0.6111 |
0.7200 |
0.3636 |
0.5418 |
0.4118 |
0.4118 |
0.4118 |
0.2000 |
0.0611 |
0.0902 |
AUEB-System1 |
0.6667 |
0.8000 |
- |
0.4000 |
0.3824 |
0.5000 |
0.4363 |
0.3967 |
0.2324 |
0.2663 |
AUEB-System2 |
0.6667 |
0.8000 |
- |
0.4000 |
0.3235 |
0.5000 |
0.3995 |
0.1400 |
0.1574 |
0.1299 |
AUEB-System3 |
0.6667 |
0.8000 |
- |
0.4000 |
0.4118 |
0.4706 |
0.4363 |
0.2533 |
0.2407 |
0.2191 |
AUEB-System4 |
0.6667 |
0.8000 |
- |
0.4000 |
0.2647 |
0.4706 |
0.3578 |
0.2867 |
0.2204 |
0.2172 |
NCU-IISR/AS-GIS-1 |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
NCU-IISR/AS-GIS-2 |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
NCU-IISR/AS-GIS-3 |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
bio-answerfinder |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.4118 |
0.4706 |
0.4412 |
0.5254 |
0.4157 |
0.4485 |
bio-answerfinder-2 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.4118 |
0.4706 |
0.4412 |
0.5254 |
0.4157 |
0.4485 |
orpheus_kg |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.3529 |
0.3529 |
0.3529 |
0.3833 |
0.3546 |
0.3657 |
LaRSA |
0.3889 |
0.1538 |
0.5217 |
0.3378 |
0.4412 |
0.5882 |
0.5098 |
0.6143 |
0.4281 |
0.4490 |
MQ-1 |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
MQ-2 |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
UDEL-LAB1 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5882 |
0.6471 |
0.6103 |
0.6707 |
0.6530 |
0.6393 |
UDEL-LAB2 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5882 |
0.6471 |
0.6176 |
0.6914 |
0.7193 |
0.6787 |
UDEL-LAB3 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5882 |
0.6765 |
0.6235 |
0.7042 |
0.7400 |
0.7051 |
UDEL-LAB4 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5882 |
0.7059 |
0.6206 |
0.6859 |
0.7530 |
0.7011 |
KU-AAA637-system1 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.5294 |
0.6471 |
0.5745 |
0.4975 |
0.4170 |
0.3949 |
KU-AAA637-system2 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.5294 |
0.6176 |
0.5686 |
0.5324 |
0.4337 |
0.4210 |
KU-AAA637-system3 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.5294 |
0.6471 |
0.5745 |
0.5324 |
0.4337 |
0.4210 |
KU-AAA637-system4 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.5588 |
0.6471 |
0.5907 |
0.5324 |
0.4337 |
0.4210 |
KU-AAA637-system5 |
0.9444 |
0.9600 |
0.9091 |
0.9345 |
0.5588 |
0.6176 |
0.5833 |
0.5324 |
0.4337 |
0.4210 |
bioinfo-0 |
0.5000 |
0.5263 |
0.4706 |
0.4985 |
- | - | - |
- | - | - |
bioinfo-1 |
0.5000 |
0.5263 |
0.4706 |
0.4985 |
- | - | - |
- | - | - |
bioinfo-2 |
0.6667 |
0.7273 |
0.5714 |
0.6494 |
- | - | - |
- | - | - |
bioinfo-3 |
0.4444 |
0.5000 |
0.3750 |
0.4375 |
- | - | - |
- | - | - |
bioinfo-4 |
0.5000 |
0.5263 |
0.4706 |
0.4985 |
- | - | - |
- | - | - |
NCU-1 |
0.7778 |
0.8571 |
0.5000 |
0.6786 |
- | - | - |
- | - | - |
simple truncation |
0.6667 |
0.8000 |
- |
0.4000 |
- | - | - |
- | - | - |
Ir_sys1 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5588 |
0.6176 |
0.5833 |
0.5785 |
0.4004 |
0.4247 |
Ir_sys2 |
0.8333 |
0.8889 |
0.6667 |
0.7778 |
0.5588 |
0.6176 |
0.5882 |
0.5893 |
0.4763 |
0.4956 |
Ir_sys3 |
0.8333 |
0.8889 |
0.6667 |
0.7778 |
0.5882 |
0.6471 |
0.6103 |
0.6226 |
0.4281 |
0.4504 |
Ir_sys4 |
0.7222 |
0.7826 |
0.6154 |
0.6990 |
0.5882 |
0.6176 |
0.5980 |
0.6149 |
0.4504 |
0.4722 |
lalala |
0.7222 |
0.7826 |
0.6154 |
0.6990 |
0.5882 |
0.6471 |
0.6039 |
0.4955 |
0.6067 |
0.5177 |
BioASQ_Baseline |
0.4444 |
0.3750 |
0.5000 |
0.4375 |
0.0882 |
0.2647 |
0.1382 |
0.2892 |
0.4019 |
0.3053 |
Ideal Answers
|
Automatic scores (Rouge - R) |
Manual scores |
System |
R-2 (Rec) |
R-2 (F1) |
R-SU4 (Rec) |
R-SU4 (F1) |
Readability |
Recall |
Precision |
Repetition |
simple baseline solr |
0.4464 |
0.2606 |
0.4547 |
0.2515 |
3.63 |
4.06 |
3.46 |
3.87 |
AUEB-System1 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
- |
- |
- |
NCU-IISR/AS-GIS-1 |
0.4704 |
0.4151 |
0.4772 |
0.4040 |
4.61 |
4.49 |
4.48 |
4.94 |
NCU-IISR/AS-GIS-2 |
0.4840 |
0.4283 |
0.4907 |
0.4177 |
4.53 |
4.48 |
4.46 |
4.90 |
NCU-IISR/AS-GIS-3 |
0.4824 |
0.4276 |
0.4897 |
0.4173 |
4.52 |
4.48 |
4.44 |
4.89 |
bio-answerfinder |
0.5531 |
0.3683 |
0.5600 |
0.3609 |
4.27 |
4.59 |
4.19 |
4.46 |
bio-answerfinder-2 |
0.4749 |
0.4031 |
0.4805 |
0.3963 |
4.51 |
4.44 |
4.37 |
4.89 |
orpheus_kg |
0.1411 |
0.1714 |
0.1332 |
0.1636 |
4.47 |
4.29 |
4.44 |
4.92 |
LaRSA |
- |
- |
- |
- |
- |
- |
- |
- |
MQ-1 |
0.6053 |
0.3387 |
0.6184 |
0.3256 |
4.29 |
4.70 |
4.08 |
4.34 |
MQ-2 |
0.6079 |
0.3387 |
0.6207 |
0.3258 |
4.26 |
4.70 |
4.08 |
4.37 |
UDEL-LAB1 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB2 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB3 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
UDEL-LAB4 |
- |
- |
- |
- |
0.38 |
0.38 |
0.38 |
0.38 |
KU-AAA637-system1 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system2 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system3 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system4 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system5 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
- |
- |
- |
NCU-1 |
0.4881 |
0.4187 |
0.4943 |
0.4073 |
4.57 |
4.52 |
4.48 |
4.93 |
simple truncation |
0.7085 |
0.3414 |
0.7128 |
0.3251 |
4.04 |
4.90 |
3.82 |
4.08 |
Ir_sys1 |
0.6085 |
0.3421 |
0.6169 |
0.3272 |
4.28 |
4.77 |
4.09 |
4.33 |
Ir_sys2 |
0.6128 |
0.3441 |
0.6282 |
0.3317 |
4.24 |
4.70 |
4.10 |
4.33 |
Ir_sys3 |
0.6026 |
0.3388 |
0.6153 |
0.3254 |
4.23 |
4.76 |
4.10 |
4.31 |
Ir_sys4 |
0.1114 |
0.1654 |
0.0864 |
0.1318 |
3.86 |
3.39 |
3.92 |
4.73 |
lalala |
0.3131 |
0.2698 |
0.3210 |
0.2699 |
4.12 |
3.46 |
3.68 |
4.44 |
BioASQ_Baseline |
- |
- |
- |
- |
- |
- |
- |
- |
Test batch 3
Exact Answers
|
Yes/No |
Factoid |
List |
System |
Accuracy |
F1 Yes |
F1 No |
Macro F1 |
Strict Acc. |
Lenient Acc. |
MRR |
Mean Prec. |
Recall |
F-Measure |
AUEB-System1 |
0.7600 |
0.8636 |
- |
0.4318 |
0.3750 |
0.5313 |
0.4349 |
0.1818 |
0.1584 |
0.1292 |
AUEB-System2 |
0.7600 |
0.8636 |
- |
0.4318 |
0.4688 |
0.4688 |
0.4688 |
0.1409 |
0.1662 |
0.1421 |
AUEB-System3 |
0.7600 |
0.8636 |
- |
0.4318 |
0.3438 |
0.5313 |
0.4323 |
0.1152 |
0.1351 |
0.1065 |
AUEB-System4 |
0.7600 |
0.8636 |
- |
0.4318 |
0.5000 |
0.5313 |
0.5156 |
0.1273 |
0.0766 |
0.0669 |
AUEB-System5 |
0.7600 |
0.8636 |
- |
0.4318 |
0.3750 |
0.5625 |
0.4505 |
0.1227 |
0.1403 |
0.1128 |
bio-answerfinder |
0.8800 |
0.9189 |
0.7692 |
0.8441 |
0.3750 |
0.4063 |
0.3906 |
0.6273 |
0.4472 |
0.4843 |
bio-answerfinder-2 |
0.8800 |
0.9189 |
0.7692 |
0.8441 |
0.3750 |
0.4063 |
0.3906 |
0.6273 |
0.4472 |
0.4843 |
LaRSA |
0.9200 |
0.9500 |
0.8000 |
0.8750 |
0.5313 |
0.6875 |
0.5990 |
0.4923 |
0.4128 |
0.4052 |
MQ-1 |
0.7600 |
0.8636 |
- |
0.4318 |
- | - | - |
- | - | - |
MQ-2 |
0.7600 |
0.8636 |
- |
0.4318 |
- | - | - |
- | - | - |
NCU-1 |
0.8800 |
0.9268 |
0.6667 |
0.7967 |
0.4063 |
0.4063 |
0.4063 |
0.3030 |
0.1608 |
0.1971 |
NCU-IISR/AS-GIS-1 |
0.8800 |
0.9268 |
0.6667 |
0.7967 |
0.4063 |
0.4063 |
0.4063 |
0.3030 |
0.1608 |
0.1971 |
NCU-IISR/AS-GIS-2 |
0.8800 |
0.9268 |
0.6667 |
0.7967 |
0.4063 |
0.4063 |
0.4063 |
0.3030 |
0.1608 |
0.1971 |
NCU-IISR/AS-GIS-3 |
0.8800 |
0.9268 |
0.6667 |
0.7967 |
0.4063 |
0.4063 |
0.4063 |
0.3030 |
0.1608 |
0.1971 |
BioASQ-2022_UNCC |
0.8800 |
0.9231 |
0.7273 |
0.8252 |
0.5000 |
0.6875 |
0.5714 |
- | - | - |
UDEL-LAB1 |
0.9600 |
0.9730 |
0.9231 |
0.9480 |
0.5625 |
0.6563 |
0.6042 |
0.5442 |
0.6742 |
0.5655 |
UDEL-LAB2 |
0.9600 |
0.9730 |
0.9231 |
0.9480 |
0.5313 |
0.6563 |
0.5885 |
0.5174 |
0.6591 |
0.5558 |
UDEL-LAB4 |
0.9600 |
0.9730 |
0.9231 |
0.9480 |
0.5313 |
0.6250 |
0.5729 |
0.5263 |
0.5985 |
0.5188 |
UDEL-LAB3 |
0.9600 |
0.9730 |
0.9231 |
0.9480 |
0.5000 |
0.6250 |
0.5469 |
0.5293 |
0.6439 |
0.5255 |
UDEL-LAB5 |
0.9600 |
0.9730 |
0.9231 |
0.9480 |
0.5625 |
0.6563 |
0.5948 |
0.5447 |
0.5682 |
0.5094 |
KU-AAA637-system1 |
0.8800 |
0.9189 |
0.7692 |
0.8441 |
0.5000 |
0.6875 |
0.5729 |
0.5203 |
0.4794 |
0.4201 |
KU-AAA637-system2 |
0.9200 |
0.9444 |
0.8571 |
0.9008 |
0.5313 |
0.6875 |
0.5911 |
0.5152 |
0.4794 |
0.4192 |
KU-AAA637-system3 |
0.9200 |
0.9444 |
0.8571 |
0.9008 |
0.5313 |
0.6875 |
0.5885 |
0.5152 |
0.4794 |
0.4192 |
KU-AAA637-system4 |
0.8800 |
0.9189 |
0.7692 |
0.8441 |
0.5313 |
0.6875 |
0.5964 |
0.5195 |
0.4794 |
0.4212 |
KU-AAA637-system5 |
0.8800 |
0.9189 |
0.7692 |
0.8441 |
0.4375 |
0.6875 |
0.5323 |
0.5126 |
0.4613 |
0.4083 |
Fleming-3 |
0.8000 |
0.8837 |
0.2857 |
0.5847 |
- | - | - |
- | - | - |
bioinfo-0 |
0.7200 |
0.7879 |
0.5882 |
0.6881 |
- | - | - |
- | - | - |
bioinfo-1 |
0.8000 |
0.8649 |
0.6154 |
0.7401 |
- | - | - |
- | - | - |
bioinfo-2 |
0.6800 |
0.7500 |
0.5556 |
0.6528 |
- | - | - |
- | - | - |
bioinfo-3 |
0.7600 |
0.8421 |
0.5000 |
0.6711 |
- | - | - |
- | - | - |
bioinfo-4 |
0.7200 |
0.7879 |
0.5882 |
0.6881 |
- | - | - |
- | - | - |
new kgqa for yesno |
0.7200 |
0.8293 |
0.2222 |
0.5257 |
0.3438 |
0.5625 |
0.4245 |
0.1955 |
0.2333 |
0.2030 |
extractive |
0.7600 |
0.8636 |
- |
0.4318 |
- | - | - |
- | - | - |
Ir_sys1 |
0.9200 |
0.9444 |
0.8571 |
0.9008 |
0.4688 |
0.6875 |
0.5573 |
0.5325 |
0.4552 |
0.4567 |
Ir_sys2 |
0.9200 |
0.9444 |
0.8571 |
0.9008 |
0.4375 |
0.6563 |
0.5151 |
0.5189 |
0.3697 |
0.4000 |
Ir_sys3 |
0.8400 |
0.9000 |
0.6000 |
0.7500 |
0.5000 |
0.6563 |
0.5625 |
0.5038 |
0.3652 |
0.3758 |
Ir_sys4 |
0.8400 |
0.9000 |
0.6000 |
0.7500 |
0.5313 |
0.6250 |
0.5703 |
0.5195 |
0.3392 |
0.3740 |
lalala |
0.9200 |
0.9444 |
0.8571 |
0.9008 |
0.5000 |
0.6250 |
0.5573 |
0.4418 |
0.6076 |
0.4769 |
BioASQ_Baseline |
0.2400 |
0.0952 |
0.3448 |
0.2200 |
0.1563 |
0.4063 |
0.2526 |
0.1338 |
0.3478 |
0.1752 |
Ideal Answers
|
Automatic scores (Rouge - R) |
Manual scores |
System |
R-2 (Rec) |
R-2 (F1) |
R-SU4 (Rec) |
R-SU4 (F1) |
Readability |
Recall |
Precision |
Repetition |
AUEB-System1 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
- |
- |
- |
bio-answerfinder |
0.4721 |
0.3219 |
0.4879 |
0.3190 |
4.30 |
4.64 |
4.26 |
4.58 |
bio-answerfinder-2 |
0.3752 |
0.3471 |
0.3925 |
0.3474 |
4.61 |
4.61 |
4.43 |
4.97 |
LaRSA |
0.4935 |
0.3222 |
0.4977 |
0.3086 |
4.54 |
4.40 |
3.90 |
4.73 |
MQ-1 |
0.5446 |
0.3253 |
0.5508 |
0.3119 |
4.14 |
4.74 |
4.07 |
4.26 |
MQ-2 |
0.5328 |
0.3179 |
0.5392 |
0.3046 |
4.17 |
4.76 |
4.10 |
4.23 |
NCU-1 |
0.4072 |
0.3696 |
0.4106 |
0.3613 |
4.58 |
4.48 |
4.49 |
4.94 |
NCU-IISR/AS-GIS-1 |
0.3787 |
0.3508 |
0.3786 |
0.3395 |
4.62 |
4.40 |
4.46 |
4.99 |
NCU-IISR/AS-GIS-2 |
0.4054 |
0.3690 |
0.4083 |
0.3611 |
4.58 |
4.46 |
4.49 |
4.97 |
NCU-IISR/AS-GIS-3 |
0.3910 |
0.3553 |
0.3959 |
0.3480 |
4.62 |
4.42 |
4.44 |
4.97 |
BioASQ-2022_UNCC |
- |
- |
- |
- |
- |
- |
- |
- |
UDEL-LAB1 |
- |
- |
- |
- |
0.36 |
0.36 |
0.36 |
0.36 |
UDEL-LAB2 |
- |
- |
- |
- |
0.36 |
0.36 |
0.36 |
0.36 |
UDEL-LAB4 |
- |
- |
- |
- |
0.36 |
0.36 |
0.36 |
0.36 |
UDEL-LAB3 |
- |
- |
- |
- |
0.36 |
0.36 |
0.36 |
0.36 |
UDEL-LAB5 |
- |
- |
- |
- |
0.36 |
0.36 |
0.36 |
0.36 |
KU-AAA637-system1 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system2 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system3 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system4 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system5 |
- |
- |
- |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
1.28 |
0.96 |
1.19 |
1.39 |
bioinfo-0 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
- |
- |
- |
new kgqa for yesno |
0.1960 |
0.1455 |
0.2060 |
0.1430 |
1.89 |
2.19 |
1.94 |
2.11 |
extractive |
0.1304 |
0.0979 |
0.1274 |
0.0939 |
0.80 |
1.09 |
1.01 |
1.08 |
Ir_sys1 |
0.5603 |
0.3408 |
0.5585 |
0.3238 |
4.11 |
4.72 |
4.06 |
4.28 |
Ir_sys2 |
0.5677 |
0.3403 |
0.5778 |
0.3258 |
4.13 |
4.80 |
4.08 |
4.29 |
Ir_sys3 |
0.2892 |
0.2512 |
0.3020 |
0.2548 |
4.09 |
3.81 |
3.63 |
4.64 |
Ir_sys4 |
0.5241 |
0.3178 |
0.5334 |
0.3060 |
4.11 |
4.74 |
4.08 |
4.24 |
lalala |
0.5677 |
0.3403 |
0.5778 |
0.3258 |
4.13 |
4.80 |
4.08 |
4.29 |
BioASQ_Baseline |
- |
- |
- |
- |
- |
- |
- |
- |
Test batch 4
Exact Answers
|
Yes/No |
Factoid |
List |
System |
Accuracy |
F1 Yes |
F1 No |
Macro F1 |
Strict Acc. |
Lenient Acc. |
MRR |
Mean Prec. |
Recall |
F-Measure |
AUEB-System1 |
0.7083 |
0.8293 |
- |
0.4146 |
0.2903 |
0.4516 |
0.3495 |
0.2917 |
0.1285 |
0.1571 |
AUEB-System2 |
0.7083 |
0.8293 |
- |
0.4146 |
0.3226 |
0.4516 |
0.3710 |
0.1000 |
0.0917 |
0.0933 |
AUEB-System3 |
0.7083 |
0.8293 |
- |
0.4146 |
0.2903 |
0.4516 |
0.3602 |
0.1694 |
0.1424 |
0.1446 |
AUEB-System4 |
0.7083 |
0.8293 |
- |
0.4146 |
0.2903 |
0.4839 |
0.3710 |
0.2250 |
0.2118 |
0.2032 |
AUEB-System5 |
0.7083 |
0.8293 |
- |
0.4146 |
0.2903 |
0.5484 |
0.3855 |
0.2222 |
0.1563 |
0.1741 |
extractive |
0.7083 |
0.8293 |
- |
0.4146 |
- | - | - |
- | - | - |
Fleming-3 |
0.7500 |
0.8500 |
0.2500 |
0.5500 |
- | - | - |
- | - | - |
bio-answerfinder |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.3548 |
0.4194 |
0.3871 |
0.3727 |
0.2701 |
0.2733 |
bio-answerfinder-2 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.3548 |
0.4194 |
0.3871 |
0.3727 |
0.2701 |
0.2733 |
bioinfo-0 |
0.7917 |
0.8387 |
0.7059 |
0.7723 |
- | - | - |
- | - | - |
bioinfo-1 |
0.7917 |
0.8718 |
0.4444 |
0.6581 |
- | - | - |
- | - | - |
bioinfo-2 |
0.8750 |
0.9032 |
0.8235 |
0.8634 |
- | - | - |
- | - | - |
bioinfo-3 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
- | - | - |
- | - | - |
bioinfo-4 |
0.7917 |
0.8387 |
0.7059 |
0.7723 |
- | - | - |
- | - | - |
LaRSA |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4516 |
0.6129 |
0.5129 |
0.4736 |
0.3104 |
0.3048 |
BioASQ-2022_UNCC |
0.9167 |
0.9412 |
0.8571 |
0.8992 |
0.5161 |
0.6129 |
0.5645 |
- | - | - |
KU-AAA637-system1 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4839 |
0.6452 |
0.5495 |
0.4491 |
0.3326 |
0.3068 |
KU-AAA637-system2 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4839 |
0.6129 |
0.5430 |
0.4750 |
0.3535 |
0.3268 |
KU-AAA637-system3 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4839 |
0.6452 |
0.5495 |
0.4491 |
0.3118 |
0.2846 |
KU-AAA637-system4 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.5161 |
0.6452 |
0.5656 |
0.4819 |
0.3660 |
0.3537 |
KU-AAA637-system5 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4516 |
0.5806 |
0.5108 |
0.5083 |
0.4146 |
0.3669 |
MQ-1 |
0.7083 |
0.8293 |
- |
0.4146 |
- | - | - |
- | - | - |
MQ-2 |
0.7083 |
0.8293 |
- |
0.4146 |
- | - | - |
- | - | - |
UDEL-LAB1 |
0.9583 |
0.9697 |
0.9333 |
0.9515 |
0.4839 |
0.6129 |
0.5387 |
0.5799 |
0.5017 |
0.4950 |
UDEL-LAB2 |
0.9583 |
0.9697 |
0.9333 |
0.9515 |
0.4839 |
0.6129 |
0.5484 |
0.5834 |
0.5844 |
0.5386 |
UDEL-LAB3 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5161 |
0.6129 |
0.5484 |
0.5584 |
0.4438 |
0.4501 |
UDEL-LAB4 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5484 |
0.6129 |
0.5613 |
0.6162 |
0.4753 |
0.4752 |
NCU-1 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
0.3548 |
0.3548 |
0.3548 |
0.4500 |
0.2801 |
0.3190 |
UDEL-LAB5 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5161 |
0.5806 |
0.5484 |
0.6132 |
0.4426 |
0.4434 |
NCU-IISR/AS-GIS-1 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
0.3548 |
0.3548 |
0.3548 |
0.4500 |
0.2801 |
0.3190 |
NCU-IISR/AS-GIS-2 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
0.3548 |
0.3548 |
0.3548 |
0.4500 |
0.2801 |
0.3190 |
simple truncation |
0.7083 |
0.8293 |
- |
0.4146 |
- | - | - |
- | - | - |
Ir_sys1 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.4839 |
0.6452 |
0.5495 |
0.4444 |
0.2410 |
0.2747 |
NCU-IISR-AS-GIS-4 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
0.3226 |
0.3226 |
0.3226 |
0.3778 |
0.3122 |
0.3224 |
NCU-IISR-AS-GIS-5 |
0.8333 |
0.8947 |
0.6000 |
0.7474 |
0.3226 |
0.3226 |
0.3226 |
0.3778 |
0.3053 |
0.3200 |
Ir_sys2 |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.4516 |
0.5161 |
0.4839 |
0.3889 |
0.2847 |
0.2718 |
Ir_sys3 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.5161 |
0.6774 |
0.5806 |
0.4583 |
0.3243 |
0.3314 |
Ir_sys4 |
0.9583 |
0.9714 |
0.9231 |
0.9473 |
0.4516 |
0.5806 |
0.5161 |
0.5083 |
0.2826 |
0.2841 |
lalala |
1.0000 |
1.0000 |
1.0000 |
1.0000 |
0.5806 |
0.6452 |
0.5995 |
0.4089 |
0.4507 |
0.3835 |
BioASQ_Baseline |
0.2917 |
0.1905 |
0.3704 |
0.2804 |
0.1613 |
0.3226 |
0.2177 |
0.2163 |
0.4035 |
0.2582 |
Ideal Answers
|
Automatic scores (Rouge - R) |
Manual scores |
System |
R-2 (Rec) |
R-2 (F1) |
R-SU4 (Rec) |
R-SU4 (F1) |
Readability |
Recall |
Precision |
Repetition |
AUEB-System1 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System2 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System3 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System4 |
- |
- |
- |
- |
- |
- |
- |
- |
AUEB-System5 |
- |
- |
- |
- |
- |
- |
- |
- |
extractive |
0.1809 |
0.1414 |
0.1811 |
0.1386 |
1.14 |
1.24 |
1.12 |
1.23 |
Fleming-3 |
- |
- |
- |
- |
1.10 |
0.90 |
1.11 |
1.33 |
bio-answerfinder |
0.5208 |
0.3812 |
0.5183 |
0.3703 |
4.37 |
4.44 |
4.37 |
4.42 |
bio-answerfinder-2 |
0.4459 |
0.4249 |
0.4411 |
0.4142 |
4.63 |
4.36 |
4.61 |
4.98 |
bioinfo-0 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
- |
- |
- |
LaRSA |
0.5568 |
0.4127 |
0.5584 |
0.4013 |
4.41 |
4.68 |
4.27 |
4.59 |
BioASQ-2022_UNCC |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system1 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system2 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system3 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system4 |
- |
- |
- |
- |
- |
- |
- |
- |
KU-AAA637-system5 |
- |
- |
- |
- |
- |
- |
- |
- |
MQ-1 |
0.5818 |
0.3613 |
0.5970 |
0.3520 |
4.09 |
4.69 |
4.00 |
4.31 |
MQ-2 |
0.5825 |
0.3636 |
0.5958 |
0.3536 |
4.08 |
4.69 |
4.03 |
4.30 |
UDEL-LAB1 |
- |
- |
- |
- |
0.34 |
0.34 |
0.34 |
0.34 |
UDEL-LAB2 |
- |
- |
- |
- |
0.34 |
0.34 |
0.34 |
0.34 |
UDEL-LAB3 |
- |
- |
- |
- |
0.34 |
0.34 |
0.34 |
0.34 |
UDEL-LAB4 |
- |
- |
- |
- |
0.34 |
0.34 |
0.34 |
0.34 |
NCU-1 |
0.4351 |
0.4301 |
0.4367 |
0.4213 |
4.58 |
4.44 |
4.69 |
5.00 |
UDEL-LAB5 |
- |
- |
- |
- |
0.34 |
0.34 |
0.34 |
0.34 |
NCU-IISR/AS-GIS-1 |
0.2842 |
0.2747 |
0.2868 |
0.2706 |
4.43 |
4.04 |
4.13 |
4.96 |
NCU-IISR/AS-GIS-2 |
0.4375 |
0.4419 |
0.4349 |
0.4355 |
4.62 |
4.49 |
4.70 |
5.00 |
simple truncation |
- |
- |
- |
- |
- |
- |
- |
- |
Ir_sys1 |
0.5648 |
0.3540 |
0.5762 |
0.3449 |
4.21 |
4.70 |
4.01 |
4.34 |
NCU-IISR-AS-GIS-4 |
0.4351 |
0.4301 |
0.4367 |
0.4213 |
4.58 |
4.44 |
4.69 |
5.00 |
NCU-IISR-AS-GIS-5 |
0.4351 |
0.4301 |
0.4367 |
0.4213 |
4.58 |
4.44 |
4.69 |
5.00 |
Ir_sys2 |
0.1315 |
0.1439 |
0.1406 |
0.1501 |
4.62 |
3.63 |
4.32 |
4.86 |
Ir_sys3 |
0.2489 |
0.2124 |
0.2506 |
0.2125 |
4.22 |
3.46 |
3.77 |
4.67 |
Ir_sys4 |
0.5459 |
0.3464 |
0.5630 |
0.3390 |
4.12 |
4.63 |
3.97 |
4.31 |
lalala |
0.5531 |
0.3467 |
0.5658 |
0.3388 |
4.14 |
4.60 |
3.99 |
4.29 |
BioASQ_Baseline |
- |
- |
- |
- |
- |
- |
- |
- |
Test batch 5
Exact Answers
|
Yes/No |
Factoid |
List |
System |
Accuracy |
F1 Yes |
F1 No |
Macro F1 |
Strict Acc. |
Lenient Acc. |
MRR |
Mean Prec. |
Recall |
F-Measure |
simple truncation |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
bio-answerfinder |
0.7857 |
0.8235 |
0.7273 |
0.7754 |
0.3448 |
0.4138 |
0.3736 |
0.3438 |
0.3290 |
0.3248 |
bio-answerfinder-2 |
0.7857 |
0.8235 |
0.7273 |
0.7754 |
0.3448 |
0.4138 |
0.3736 |
0.3438 |
0.3290 |
0.3248 |
AUEB-System1 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
0.0278 |
0.0278 |
0.0278 |
AUEB-System2 |
0.5357 |
0.6977 |
- |
0.3488 |
0.3103 |
0.3448 |
0.3218 |
0.1222 |
0.1012 |
0.0891 |
AUEB-System3 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
0.0278 |
0.0278 |
0.0278 |
AUEB-System4 |
0.5357 |
0.6977 |
- |
0.3488 |
0.2414 |
0.3793 |
0.3000 |
0.1037 |
0.0734 |
0.0714 |
AUEB-System5 |
0.5357 |
0.6977 |
- |
0.3488 |
0.3103 |
0.4138 |
0.3534 |
0.0870 |
0.1012 |
0.0847 |
bioinfo-0 |
0.7857 |
0.7692 |
0.8000 |
0.7846 |
- | - | - |
- | - | - |
bioinfo-1 |
0.7500 |
0.7586 |
0.7407 |
0.7497 |
- | - | - |
- | - | - |
bioinfo-2 |
0.8214 |
0.8148 |
0.8276 |
0.8212 |
- | - | - |
- | - | - |
bioinfo-3 |
0.8214 |
0.8276 |
0.8148 |
0.8212 |
- | - | - |
- | - | - |
bioinfo-4 |
0.7857 |
0.7692 |
0.8000 |
0.7846 |
- | - | - |
- | - | - |
LaRSA |
0.7500 |
0.8000 |
0.6667 |
0.7333 |
0.4138 |
0.5517 |
0.4626 |
0.4486 |
0.4188 |
0.4191 |
Fleming-4 |
0.6429 |
0.7500 |
0.3750 |
0.5625 |
- | - | - |
- | - | - |
UDEL-LAB1 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3448 |
0.5172 |
0.4149 |
0.6082 |
0.5921 |
0.5794 |
UDEL-LAB2 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3103 |
0.5172 |
0.3833 |
0.5642 |
0.6424 |
0.5860 |
UDEL-LAB3 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.4483 |
0.5862 |
0.5000 |
0.5790 |
0.6043 |
0.5793 |
UDEL-LAB4 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3103 |
0.5862 |
0.4190 |
0.6120 |
0.6427 |
0.6123 |
UDEL-LAB5 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3793 |
0.5172 |
0.4190 |
0.5881 |
0.6515 |
0.6076 |
BioASQ-2022_UNCC |
0.8929 |
0.9032 |
0.8800 |
0.8916 |
0.1034 |
0.6207 |
0.2253 |
- | - | - |
BioASQ-2022_UNCC1 |
0.8929 |
0.9032 |
0.8800 |
0.8916 |
0.4138 |
0.6207 |
0.4868 |
- | - | - |
MQ-1 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
MQ-2 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
BioASQ-2022_UNCC2 |
0.8929 |
0.9032 |
0.8800 |
0.8916 |
0.4138 |
0.6207 |
0.4868 |
- | - | - |
BioASQ-2022_UNCC3 |
0.8929 |
0.9032 |
0.8800 |
0.8916 |
0.4138 |
0.6207 |
0.4868 |
- | - | - |
KU-AAA637-system1 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3448 |
0.4828 |
0.3994 |
0.5386 |
0.5095 |
0.4885 |
KU-AAA637-system2 |
0.9286 |
0.9375 |
0.9167 |
0.9271 |
0.3448 |
0.4828 |
0.4052 |
0.5858 |
0.5373 |
0.5285 |
KU-AAA637-system3 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3448 |
0.5172 |
0.4080 |
0.5283 |
0.5049 |
0.5054 |
KU-AAA637-system4 |
0.8571 |
0.8824 |
0.8182 |
0.8503 |
0.3793 |
0.5172 |
0.4264 |
0.5361 |
0.5373 |
0.4999 |
KU-AAA637-system5 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.3448 |
0.5172 |
0.4109 |
0.5312 |
0.5595 |
0.5085 |
NCU-IISR/AS-GIS-1 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
NCU-1 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
NCU-IISR/AS-GIS-2 |
0.5357 |
0.6977 |
- |
0.3488 |
- | - | - |
- | - | - |
NCU-IISR-AS-GIS-4 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.4828 |
0.5862 |
0.5259 |
0.6500 |
0.5114 |
0.5517 |
NCU-IISR-AS-GIS-5 |
0.8929 |
0.9091 |
0.8696 |
0.8893 |
0.4828 |
0.5862 |
0.5259 |
0.6500 |
0.5114 |
0.5517 |
Ir_sys1 |
0.9286 |
0.9333 |
0.9231 |
0.9282 |
0.3793 |
0.5517 |
0.4540 |
0.6799 |
0.4716 |
0.5214 |
Ir_sys2 |
0.9286 |
0.9333 |
0.9231 |
0.9282 |
0.3448 |
0.5862 |
0.4494 |
0.6204 |
0.4660 |
0.5099 |
Ir_sys3 |
0.8929 |
0.9032 |
0.8800 |
0.8916 |
0.4828 |
0.5862 |
0.5098 |
0.5852 |
0.4114 |
0.4613 |
Ir_sys4 |
0.7857 |
0.8235 |
0.7273 |
0.7754 |
0.3793 |
0.4483 |
0.4138 |
0.6058 |
0.4354 |
0.4692 |
lalala |
0.9286 |
0.9333 |
0.9231 |
0.9282 |
0.3793 |
0.5517 |
0.4494 |
0.5209 |
0.5840 |
0.5340 |
BioASQ_Baseline |
0.4643 |
0.2857 |
0.5714 |
0.4286 |
0.0345 |
0.1034 |
0.0632 |
0.2125 |
0.4801 |
0.2572 |
Ideal Answers
|
Automatic scores (Rouge - R) |
Manual scores |
System |
R-2 (Rec) |
R-2 (F1) |
R-SU4 (Rec) |
R-SU4 (F1) |
Readability |
Recall |
Precision |
Repetition |