BioASQ Participants Area
Task 12b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *
The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented
here .
Test batch 1
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
first batch |
0.0341 |
0.1004 |
0.0484 |
0.0259 |
0.0001 |
IR1 |
0.0118 |
0.0343 |
0.0161 |
0.0297 |
0.0000 |
mibi_rag_snippet |
0.0880 |
0.1560 |
0.0951 |
0.1053 |
0.0002 |
mibi_rag_abstract |
0.0880 |
0.1560 |
0.0951 |
0.1053 |
0.0002 |
2 |
0.0275 |
0.0668 |
0.0338 |
0.0399 |
0.0000 |
UR-IW-5 |
0.0525 |
0.1093 |
0.0602 |
0.0811 |
0.0001 |
UR-IW-4 |
0.0566 |
0.0861 |
0.0477 |
0.0511 |
0.0001 |
UR-IW-2 |
0.0544 |
0.0975 |
0.0551 |
0.0600 |
0.0001 |
UR-IW-3 |
0.0539 |
0.1023 |
0.0648 |
0.0631 |
0.0001 |
UR-IW-1 |
0.0784 |
0.1525 |
0.0938 |
0.0751 |
0.0002 |
bioinfo-0 |
0.1151 |
0.3055 |
0.1570 |
0.1800 |
0.0009 |
bioinfo-1 |
0.1156 |
0.3171 |
0.1581 |
0.2018 |
0.0015 |
bioinfo-2 |
0.1294 |
0.3369 |
0.1728 |
0.2006 |
0.0019 |
bioinfo-3 |
0.1009 |
0.3047 |
0.1444 |
0.2024 |
0.0013 |
bioinfo-4 |
0.1039 |
0.3124 |
0.1485 |
0.2067 |
0.0016 |
Fleming-1 |
0.0834 |
0.2659 |
0.1198 |
0.1143 |
0.0007 |
qa |
0.0333 |
0.0343 |
0.0311 |
0.0294 |
0.0000 |
Gatech competition |
0.0461 |
0.1076 |
0.0596 |
0.0497 |
0.0001 |
GTBioASQsys2 |
0.0557 |
0.1346 |
0.0745 |
0.0599 |
0.0001 |
GTBioASQsys3 |
0.0669 |
0.1412 |
0.0816 |
0.0717 |
0.0001 |
dmiip2024 |
0.0753 |
0.2505 |
0.1092 |
0.1530 |
0.0006 |
dmiip2024_1 |
0.0718 |
0.2335 |
0.1041 |
0.1503 |
0.0007 |
dmiip2024_2 |
0.0718 |
0.2319 |
0.1040 |
0.1486 |
0.0005 |
dmiip2024_4 |
0.0706 |
0.2479 |
0.1035 |
0.1538 |
0.0006 |
dmiip2024_3 |
0.0706 |
0.2514 |
0.1039 |
0.1612 |
0.0007 |
IRIS_1 |
0.0753 |
0.2339 |
0.1081 |
0.1357 |
0.0005 |
IRIS_2 |
0.0729 |
0.2413 |
0.1055 |
0.1318 |
0.0006 |
IRIS_4 |
0.0729 |
0.2389 |
0.1055 |
0.1467 |
0.0005 |
simple truncation |
0.0337 |
0.0889 |
0.0456 |
0.0284 |
0.0001 |
IRIS_5 |
0.0718 |
0.2365 |
0.1044 |
0.1132 |
0.0005 |
IRIS_3 |
0.0753 |
0.2404 |
0.1084 |
0.1471 |
0.0006 |
HU-WBI-1 |
0.0494 |
0.1642 |
0.0712 |
0.1094 |
0.0003 |
4 |
0.0222 |
0.0737 |
0.0315 |
0.0479 |
0.0000 |
ashley2 |
0.0482 |
0.1477 |
0.0675 |
0.1048 |
0.0002 |
ashley3 |
0.0435 |
0.1175 |
0.0598 |
0.0831 |
0.0001 |
3 |
0.0326 |
0.0563 |
0.0373 |
0.0463 |
0.0000 |
CPS |
0.1119 |
0.0842 |
0.0760 |
0.0661 |
0.0001 |
Fleming-2 |
0.0881 |
0.2684 |
0.1255 |
0.1195 |
0.0008 |
Fleming-3 |
0.0851 |
0.2615 |
0.1216 |
0.1101 |
0.0008 |
Fleming-4 |
0.0834 |
0.2659 |
0.1198 |
0.1143 |
0.0007 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
first batch |
0.0129 |
0.0002 |
0.0003 |
0.0050 |
0.0000 |
IR1 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
0.0235 |
0.0268 |
0.0236 |
0.0413 |
0.0000 |
mibi_rag_abstract |
0.0235 |
0.0268 |
0.0236 |
0.0413 |
0.0000 |
2 |
- |
- |
- |
- |
- |
UR-IW-5 |
0.0450 |
0.0546 |
0.0441 |
0.0412 |
0.0000 |
UR-IW-4 |
0.0444 |
0.0527 |
0.0336 |
0.0244 |
0.0001 |
UR-IW-2 |
0.0341 |
0.0483 |
0.0276 |
0.0237 |
0.0000 |
UR-IW-3 |
0.0454 |
0.0539 |
0.0458 |
0.0452 |
0.0001 |
UR-IW-1 |
0.0480 |
0.0720 |
0.0508 |
0.0357 |
0.0001 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Fleming-1 |
0.0004 |
0.0023 |
0.0007 |
0.0002 |
0.0000 |
qa |
- |
- |
- |
- |
- |
Gatech competition |
0.0240 |
0.0238 |
0.0194 |
0.0150 |
0.0000 |
GTBioASQsys2 |
0.0096 |
0.0076 |
0.0074 |
0.0050 |
0.0000 |
GTBioASQsys3 |
0.0244 |
0.0205 |
0.0201 |
0.0204 |
0.0000 |
dmiip2024 |
0.0485 |
0.1645 |
0.0691 |
0.1061 |
0.0002 |
dmiip2024_1 |
0.0477 |
0.1568 |
0.0679 |
0.1039 |
0.0002 |
dmiip2024_2 |
0.0446 |
0.1490 |
0.0638 |
0.1149 |
0.0001 |
dmiip2024_4 |
0.0482 |
0.1601 |
0.0681 |
0.0985 |
0.0001 |
dmiip2024_3 |
0.0482 |
0.1645 |
0.0684 |
0.1062 |
0.0002 |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
ashley3 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
Fleming-3 |
0.0480 |
0.0797 |
0.0530 |
0.0528 |
0.0001 |
Fleming-4 |
0.0004 |
0.0023 |
0.0007 |
0.0002 |
0.0000 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
first batch |
- |
- |
- |
- |
- |
IR1 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
ashley3 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
first batch |
- |
- |
- |
- |
- |
IR1 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
ashley3 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
Test batch 2
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
0.0106 |
0.0461 |
0.0160 |
0.0201 |
0.0000 |
UR-IW-4 |
0.0694 |
0.0589 |
0.0517 |
0.0409 |
0.0000 |
UR-IW-1 |
0.0889 |
0.1607 |
0.0971 |
0.0875 |
0.0002 |
UR-IW-2 |
0.0502 |
0.1227 |
0.0583 |
0.0657 |
0.0001 |
UR-IW-5 |
0.0633 |
0.1048 |
0.0660 |
0.0564 |
0.0001 |
UR-IW-3 |
0.0542 |
0.1390 |
0.0716 |
0.0643 |
0.0001 |
first batch |
0.0271 |
0.1027 |
0.0396 |
0.0356 |
0.0001 |
2 |
0.0239 |
0.0543 |
0.0228 |
0.0397 |
0.0000 |
Fleming-2 |
0.0635 |
0.2440 |
0.0948 |
0.1381 |
0.0004 |
mibi_rag_snippet |
0.0708 |
0.2579 |
0.1049 |
0.1578 |
0.0006 |
mibi_rag_abstract |
0.0708 |
0.2579 |
0.1049 |
0.1578 |
0.0006 |
bioinfo-0 |
0.1012 |
0.3573 |
0.1443 |
0.2117 |
0.0018 |
bioinfo-1 |
0.0943 |
0.3455 |
0.1370 |
0.2110 |
0.0016 |
bioinfo-2 |
0.1085 |
0.3580 |
0.1524 |
0.2041 |
0.0022 |
bioinfo-3 |
0.0909 |
0.3159 |
0.1310 |
0.1947 |
0.0012 |
bioinfo-4 |
0.0872 |
0.3321 |
0.1298 |
0.2096 |
0.0017 |
run1 |
0.0871 |
0.1591 |
0.1042 |
0.1049 |
0.0002 |
Fleming-3 |
0.0553 |
0.2097 |
0.0835 |
0.1076 |
0.0003 |
GTBioASQsys2 |
0.0831 |
0.1414 |
0.0914 |
0.0826 |
0.0001 |
GTBioASQsys3 |
0.0825 |
0.1499 |
0.0994 |
0.0836 |
0.0001 |
bm25cross |
0.0071 |
0.0402 |
0.0115 |
0.0089 |
0.0000 |
gpt-bm25 |
0.0965 |
0.3382 |
0.1413 |
0.1824 |
0.0022 |
gpt-bm25-cross |
0.0988 |
0.1893 |
0.1189 |
0.1226 |
0.0004 |
gpt-bm25-cross2 |
0.0918 |
0.1713 |
0.1097 |
0.1189 |
0.0003 |
Gatech competition |
0.0540 |
0.1387 |
0.0740 |
0.0810 |
0.0001 |
qa |
0.0643 |
0.1445 |
0.0768 |
0.0856 |
0.0001 |
3 |
0.0012 |
0.0059 |
0.0020 |
0.0059 |
0.0000 |
4 |
0.0012 |
0.0059 |
0.0020 |
0.0059 |
0.0000 |
Deep ML methods for |
0.1090 |
0.2028 |
0.1188 |
0.1025 |
0.0005 |
MindLab QA Reloaded |
0.1017 |
0.2047 |
0.1176 |
0.0944 |
0.0005 |
dmiip2024 |
0.0835 |
0.3173 |
0.1246 |
0.2082 |
0.0013 |
dmiip2024_2 |
0.0882 |
0.3422 |
0.1319 |
0.2272 |
0.0017 |
dmiip2024_1 |
0.0847 |
0.3183 |
0.1258 |
0.2086 |
0.0013 |
dmiip2024_3 |
0.0824 |
0.3083 |
0.1227 |
0.1974 |
0.0012 |
dmiip2024_4 |
0.0953 |
0.3673 |
0.1428 |
0.2293 |
0.0026 |
IRIS_1 |
0.0729 |
0.2564 |
0.1062 |
0.1429 |
0.0007 |
IRIS_2 |
0.0635 |
0.2144 |
0.0922 |
0.1151 |
0.0004 |
IRIS_3 |
0.0741 |
0.2682 |
0.1086 |
0.1487 |
0.0008 |
IRIS_4 |
0.0624 |
0.2099 |
0.0911 |
0.1097 |
0.0004 |
IRIS_5 |
0.0694 |
0.2562 |
0.1018 |
0.1452 |
0.0007 |
simple truncation |
0.0385 |
0.1208 |
0.0561 |
0.0625 |
0.0001 |
kmeans |
0.0385 |
0.1208 |
0.0561 |
0.0625 |
0.0001 |
similarity measures |
0.0417 |
0.1270 |
0.0601 |
0.0476 |
0.0001 |
CPS |
0.0717 |
0.1201 |
0.0730 |
0.0845 |
0.0001 |
CPS2 |
0.0580 |
0.1240 |
0.0697 |
0.0871 |
0.0001 |
Fleming-1 |
0.0714 |
0.2408 |
0.1029 |
0.1585 |
0.0005 |
HU-WBI-2 |
0.0788 |
0.3238 |
0.1200 |
0.1553 |
0.0012 |
HU-WBI-3 |
0.0718 |
0.2968 |
0.1089 |
0.1357 |
0.0008 |
HU-WBI-4 |
0.0894 |
0.3673 |
0.1361 |
0.1878 |
0.0024 |
HU-WBI-5 |
0.0800 |
0.3198 |
0.1205 |
0.1699 |
0.0011 |
HU-WBI-1 |
0.0941 |
0.3482 |
0.1397 |
0.2223 |
0.0018 |
ashley1 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
UR-IW-4 |
0.0590 |
0.0449 |
0.0334 |
0.0230 |
0.0000 |
UR-IW-1 |
0.0568 |
0.0850 |
0.0532 |
0.0569 |
0.0001 |
UR-IW-2 |
0.0329 |
0.0713 |
0.0278 |
0.0191 |
0.0000 |
UR-IW-5 |
0.0357 |
0.0474 |
0.0333 |
0.0301 |
0.0000 |
UR-IW-3 |
0.0400 |
0.0722 |
0.0474 |
0.0345 |
0.0001 |
first batch |
0.0059 |
0.0001 |
0.0001 |
0.0022 |
0.0000 |
2 |
- |
- |
- |
- |
- |
Fleming-2 |
0.0202 |
0.0297 |
0.0221 |
0.0265 |
0.0000 |
mibi_rag_snippet |
0.0490 |
0.0952 |
0.0604 |
0.0508 |
0.0001 |
mibi_rag_abstract |
0.0490 |
0.0952 |
0.0604 |
0.0508 |
0.0001 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
Fleming-3 |
0.0255 |
0.0446 |
0.0282 |
0.0370 |
0.0001 |
GTBioASQsys2 |
0.0354 |
0.0491 |
0.0306 |
0.0261 |
0.0000 |
GTBioASQsys3 |
0.0351 |
0.0452 |
0.0352 |
0.0271 |
0.0000 |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
Gatech competition |
0.0138 |
0.0173 |
0.0145 |
0.0151 |
0.0000 |
qa |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
Deep ML methods for |
0.0506 |
0.0759 |
0.0522 |
0.0306 |
0.0001 |
MindLab QA Reloaded |
0.0650 |
0.0959 |
0.0668 |
0.0398 |
0.0002 |
dmiip2024 |
0.0520 |
0.1810 |
0.0746 |
0.1539 |
0.0003 |
dmiip2024_2 |
0.0523 |
0.2006 |
0.0760 |
0.1506 |
0.0004 |
dmiip2024_1 |
0.0514 |
0.1830 |
0.0741 |
0.1485 |
0.0004 |
dmiip2024_3 |
0.0515 |
0.1731 |
0.0732 |
0.1372 |
0.0003 |
dmiip2024_4 |
0.0553 |
0.1962 |
0.0795 |
0.1374 |
0.0005 |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
ashley1 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
ashley1 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
ashley1 |
- |
- |
- |
- |
- |
ashley2 |
- |
- |
- |
- |
- |
Test batch 3
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
0.0106 |
0.0514 |
0.0167 |
0.0218 |
0.0000 |
IR2 |
0.0094 |
0.0455 |
0.0148 |
0.0203 |
0.0000 |
Fleming-3 |
0.0855 |
0.3336 |
0.1273 |
0.1183 |
0.0013 |
Fleming-1 |
0.0729 |
0.3037 |
0.1083 |
0.1063 |
0.0012 |
CPS |
0.0576 |
0.1777 |
0.0778 |
0.1160 |
0.0002 |
CPS2 |
0.0576 |
0.1777 |
0.0778 |
0.1160 |
0.0002 |
bioinfo-0 |
0.0980 |
0.3849 |
0.1438 |
0.2487 |
0.0027 |
bioinfo-1 |
0.0956 |
0.3849 |
0.1422 |
0.2493 |
0.0024 |
bioinfo-2 |
0.0982 |
0.3535 |
0.1409 |
0.2427 |
0.0017 |
bioinfo-3 |
0.0889 |
0.3633 |
0.1338 |
0.2146 |
0.0020 |
bioinfo-4 |
0.0910 |
0.3731 |
0.1371 |
0.2415 |
0.0023 |
UR-IW-1 |
0.0524 |
0.1761 |
0.0720 |
0.1281 |
0.0002 |
UR-IW-2 |
0.0541 |
0.1569 |
0.0734 |
0.1217 |
0.0001 |
UR-IW-5 |
0.0664 |
0.1866 |
0.0854 |
0.0957 |
0.0003 |
UR-IW-4 |
0.0446 |
0.0859 |
0.0492 |
0.0480 |
0.0001 |
UR-IW-3 |
0.0687 |
0.1730 |
0.0766 |
0.0971 |
0.0002 |
mibi_rag_snippet |
0.0612 |
0.2351 |
0.0910 |
0.1385 |
0.0005 |
mibi_rag_abstract |
0.0541 |
0.2280 |
0.0816 |
0.1252 |
0.0003 |
mibi_rag_3 |
0.0612 |
0.2351 |
0.0910 |
0.1417 |
0.0005 |
mibi_rag_4 |
0.0541 |
0.2280 |
0.0816 |
0.1247 |
0.0003 |
mibi_rag_5 |
0.0612 |
0.2351 |
0.0910 |
0.1385 |
0.0005 |
Gatech competition |
0.0627 |
0.2136 |
0.0865 |
0.1385 |
0.0002 |
GTBioASQsys2 |
0.0490 |
0.1490 |
0.0699 |
0.0883 |
0.0001 |
GTBioASQsys3 |
0.0600 |
0.1647 |
0.0792 |
0.0929 |
0.0001 |
first batch |
0.0282 |
0.1295 |
0.0431 |
0.0611 |
0.0001 |
IRIS_1 |
0.0682 |
0.3155 |
0.1045 |
0.1682 |
0.0010 |
IRIS_2 |
0.0600 |
0.2245 |
0.0883 |
0.1217 |
0.0004 |
dmiip2024 |
0.0824 |
0.3345 |
0.1238 |
0.2243 |
0.0018 |
IRIS_3 |
0.0729 |
0.3059 |
0.1096 |
0.1890 |
0.0009 |
dmiip2024_1 |
0.0800 |
0.3335 |
0.1208 |
0.2168 |
0.0013 |
IRIS_4 |
0.0659 |
0.2962 |
0.1006 |
0.1777 |
0.0010 |
IRIS_5 |
0.0694 |
0.2780 |
0.1036 |
0.1734 |
0.0007 |
dmiip2024_2 |
0.0882 |
0.3833 |
0.1340 |
0.2541 |
0.0029 |
dmiip2024_3 |
0.0824 |
0.3590 |
0.1252 |
0.2240 |
0.0013 |
dmiip2024_4 |
0.0859 |
0.3835 |
0.1309 |
0.2549 |
0.0024 |
simple truncation |
0.0460 |
0.1751 |
0.0679 |
0.1041 |
0.0002 |
run1 |
0.0482 |
0.1759 |
0.0707 |
0.0881 |
0.0002 |
2 |
0.0024 |
0.0043 |
0.0030 |
0.0008 |
0.0000 |
kmeans |
0.0476 |
0.1663 |
0.0679 |
0.1026 |
0.0001 |
similarity measures |
0.0578 |
0.1506 |
0.0760 |
0.0959 |
0.0001 |
Fleming-5 |
0.0652 |
0.2610 |
0.0963 |
0.0993 |
0.0004 |
3 |
0.0059 |
0.0169 |
0.0085 |
0.0100 |
0.0000 |
4 |
0.0059 |
0.0169 |
0.0085 |
0.0100 |
0.0000 |
bm25cross |
0.0771 |
0.2037 |
0.1008 |
0.1312 |
0.0003 |
gpt-bm25 |
0.0700 |
0.1988 |
0.0946 |
0.1265 |
0.0003 |
0.6 |
0.0059 |
0.0169 |
0.0085 |
0.0100 |
0.0000 |
gpt-bm25-cross |
0.0747 |
0.2116 |
0.1005 |
0.1341 |
0.0003 |
Fleming-4 |
0.0612 |
0.2531 |
0.0908 |
0.0910 |
0.0003 |
gpt-bm25-cross2 |
0.0747 |
0.2096 |
0.0999 |
0.1344 |
0.0003 |
HU-WBI-1 |
0.0788 |
0.3128 |
0.1180 |
0.1655 |
0.0011 |
HU-WBI-2 |
0.0859 |
0.3295 |
0.1285 |
0.1863 |
0.0013 |
HU-WBI-3 |
0.0859 |
0.3321 |
0.1287 |
0.1928 |
0.0014 |
HU-WBI-4 |
0.0753 |
0.2776 |
0.1116 |
0.1418 |
0.0006 |
HU-WBI-5 |
0.0753 |
0.2875 |
0.1122 |
0.1526 |
0.0008 |
drums_bm |
0.0247 |
0.0855 |
0.0354 |
0.0572 |
0.0001 |
drums |
0.0247 |
0.0855 |
0.0354 |
0.0572 |
0.0001 |
Deep ML methods for |
0.0751 |
0.1147 |
0.0768 |
0.0654 |
0.0001 |
MindLab QA Reloaded |
0.0537 |
0.1604 |
0.0705 |
0.0581 |
0.0002 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
Fleming-3 |
0.0239 |
0.0846 |
0.0341 |
0.0552 |
0.0002 |
Fleming-1 |
0.0201 |
0.0714 |
0.0287 |
0.0443 |
0.0001 |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
UR-IW-1 |
0.0379 |
0.1251 |
0.0508 |
0.0818 |
0.0002 |
UR-IW-2 |
0.0359 |
0.0881 |
0.0456 |
0.0677 |
0.0001 |
UR-IW-5 |
0.0399 |
0.1188 |
0.0506 |
0.0736 |
0.0002 |
UR-IW-4 |
0.0316 |
0.0388 |
0.0279 |
0.0381 |
0.0000 |
UR-IW-3 |
0.0320 |
0.0819 |
0.0338 |
0.0402 |
0.0001 |
mibi_rag_snippet |
0.0378 |
0.0857 |
0.0469 |
0.0524 |
0.0001 |
mibi_rag_abstract |
0.0412 |
0.1011 |
0.0522 |
0.0549 |
0.0001 |
mibi_rag_3 |
0.0446 |
0.1143 |
0.0580 |
0.0456 |
0.0001 |
mibi_rag_4 |
0.0401 |
0.0985 |
0.0508 |
0.0585 |
0.0001 |
mibi_rag_5 |
0.0382 |
0.0879 |
0.0468 |
0.0454 |
0.0001 |
Gatech competition |
0.0387 |
0.0416 |
0.0344 |
0.0505 |
0.0001 |
GTBioASQsys2 |
0.0313 |
0.0434 |
0.0341 |
0.0312 |
0.0000 |
GTBioASQsys3 |
0.0395 |
0.0453 |
0.0369 |
0.0271 |
0.0000 |
first batch |
0.0118 |
0.0002 |
0.0003 |
0.0138 |
0.0000 |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
dmiip2024 |
0.0580 |
0.2110 |
0.0820 |
0.1897 |
0.0006 |
IRIS_3 |
- |
- |
- |
- |
- |
dmiip2024_1 |
0.0593 |
0.1983 |
0.0826 |
0.1661 |
0.0005 |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
dmiip2024_2 |
0.0645 |
0.2501 |
0.0907 |
0.2183 |
0.0009 |
dmiip2024_3 |
0.0590 |
0.2123 |
0.0828 |
0.1893 |
0.0005 |
dmiip2024_4 |
0.0666 |
0.2568 |
0.0940 |
0.2224 |
0.0009 |
simple truncation |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
Fleming-5 |
0.0189 |
0.0803 |
0.0267 |
0.0636 |
0.0001 |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
0.6 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |
Deep ML methods for |
0.0526 |
0.0509 |
0.0440 |
0.0449 |
0.0001 |
MindLab QA Reloaded |
0.0438 |
0.0776 |
0.0500 |
0.0412 |
0.0001 |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
mibi_rag_3 |
- |
- |
- |
- |
- |
mibi_rag_4 |
- |
- |
- |
- |
- |
mibi_rag_5 |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
Fleming-5 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
0.6 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
Fleming-3 |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
mibi_rag_3 |
- |
- |
- |
- |
- |
mibi_rag_4 |
- |
- |
- |
- |
- |
mibi_rag_5 |
- |
- |
- |
- |
- |
Gatech competition |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
IRIS_1 |
- |
- |
- |
- |
- |
IRIS_2 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
IRIS_3 |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
IRIS_4 |
- |
- |
- |
- |
- |
IRIS_5 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
kmeans |
- |
- |
- |
- |
- |
similarity measures |
- |
- |
- |
- |
- |
Fleming-5 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
0.6 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |
Deep ML methods for |
- |
- |
- |
- |
- |
MindLab QA Reloaded |
- |
- |
- |
- |
- |
Test batch 4
Documents
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
0.0106 |
0.0396 |
0.0158 |
0.0159 |
0.0000 |
IR2 |
0.0141 |
0.0592 |
0.0214 |
0.0180 |
0.0000 |
first batch |
0.0400 |
0.2384 |
0.0653 |
0.1210 |
0.0003 |
Fleming-1 |
0.0824 |
0.4590 |
0.1322 |
0.2558 |
0.0050 |
Fleming-2 |
0.0824 |
0.4590 |
0.1322 |
0.2558 |
0.0050 |
UR-IW-1 |
0.0952 |
0.2810 |
0.1253 |
0.1892 |
0.0006 |
UR-IW-2 |
0.1199 |
0.3769 |
0.1586 |
0.2910 |
0.0018 |
UR-IW-3 |
0.0681 |
0.1861 |
0.0844 |
0.1281 |
0.0001 |
UR-IW-4 |
0.0934 |
0.2686 |
0.1224 |
0.1819 |
0.0005 |
UR-IW-5 |
0.0870 |
0.2231 |
0.1099 |
0.1617 |
0.0003 |
mibi_rag_snippet |
0.0541 |
0.2296 |
0.0793 |
0.1379 |
0.0004 |
mibi_rag_abstract |
0.0447 |
0.2284 |
0.0701 |
0.1343 |
0.0003 |
mibi_rag_3 |
0.0435 |
0.2296 |
0.0696 |
0.1399 |
0.0004 |
bioinfo-0 |
0.1247 |
0.5353 |
0.1819 |
0.3752 |
0.0115 |
bioinfo-1 |
0.1239 |
0.5529 |
0.1833 |
0.3773 |
0.0142 |
mibi_rag_4 |
0.0447 |
0.2284 |
0.0701 |
0.1343 |
0.0003 |
bioinfo-2 |
0.1047 |
0.5347 |
0.1618 |
0.3690 |
0.0106 |
bioinfo-3 |
0.1082 |
0.5412 |
0.1671 |
0.3740 |
0.0129 |
bioinfo-4 |
0.1047 |
0.5353 |
0.1619 |
0.3903 |
0.0137 |
mibi_rag_5 |
0.0541 |
0.2296 |
0.0793 |
0.1379 |
0.0004 |
bm25cross |
0.0424 |
0.2141 |
0.0668 |
0.1138 |
0.0002 |
gpt-bm25 |
0.0482 |
0.2435 |
0.0762 |
0.1303 |
0.0003 |
gpt-bm25-cross |
0.0603 |
0.2592 |
0.0919 |
0.1436 |
0.0004 |
2 |
0.0059 |
0.0284 |
0.0092 |
0.0043 |
0.0000 |
GTBioASQsys2 |
0.0886 |
0.1959 |
0.1050 |
0.1250 |
0.0001 |
GTBioASQsys3 |
0.0954 |
0.2780 |
0.1241 |
0.1769 |
0.0004 |
3 |
0.0059 |
0.0176 |
0.0083 |
0.0125 |
0.0000 |
4 |
0.0259 |
0.1265 |
0.0412 |
0.0803 |
0.0001 |
CPS |
0.0570 |
0.2157 |
0.0800 |
0.1308 |
0.0003 |
CPS2 |
0.0606 |
0.2304 |
0.0854 |
0.1352 |
0.0003 |
dmiip2024 |
0.0929 |
0.5069 |
0.1487 |
0.3541 |
0.0088 |
dmiip2024_2 |
0.0965 |
0.5141 |
0.1536 |
0.3824 |
0.0078 |
dmiip2024_3 |
0.1000 |
0.5569 |
0.1609 |
0.3930 |
0.0148 |
dmiip2024_4 |
0.0976 |
0.5559 |
0.1576 |
0.3911 |
0.0133 |
Fleming-5 |
0.0682 |
0.3863 |
0.1102 |
0.2615 |
0.0018 |
Fleming-4 |
0.0682 |
0.3863 |
0.1102 |
0.2615 |
0.0018 |
gpt-bm25-cross2 |
0.0765 |
0.1625 |
0.0958 |
0.1233 |
0.0001 |
simple truncation |
0.0990 |
0.4651 |
0.1535 |
0.3068 |
0.0082 |
dmiip2024_1 |
0.0953 |
0.5255 |
0.1530 |
0.3634 |
0.0100 |
HU-WBI-3 |
0.0871 |
0.4665 |
0.1391 |
0.3487 |
0.0057 |
HU-WBI-4 |
0.0871 |
0.4596 |
0.1389 |
0.3339 |
0.0045 |
HU-WBI-5 |
0.0871 |
0.4586 |
0.1388 |
0.3506 |
0.0051 |
HU-WBI-1 |
0.0871 |
0.4567 |
0.1386 |
0.3388 |
0.0038 |
HU-WBI-2 |
0.0882 |
0.4625 |
0.1406 |
0.3274 |
0.0049 |
qa |
- |
- |
- |
- |
- |
mmb |
- |
- |
- |
- |
- |
run1 |
0.0459 |
0.2516 |
0.0733 |
0.1157 |
0.0004 |
drums_bm |
0.0306 |
0.1714 |
0.0492 |
0.0924 |
0.0001 |
drums |
0.0035 |
0.0353 |
0.0064 |
0.0154 |
0.0000 |
Snippets
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
first batch |
0.0035 |
0.0000 |
0.0001 |
0.0032 |
0.0000 |
Fleming-1 |
0.0432 |
0.1707 |
0.0615 |
0.0961 |
0.0005 |
Fleming-2 |
0.0154 |
0.2197 |
0.0276 |
0.0363 |
0.0003 |
UR-IW-1 |
0.0502 |
0.1398 |
0.0645 |
0.0959 |
0.0002 |
UR-IW-2 |
0.0777 |
0.1846 |
0.0888 |
0.1402 |
0.0008 |
UR-IW-3 |
0.0400 |
0.0486 |
0.0329 |
0.0428 |
0.0000 |
UR-IW-4 |
0.0559 |
0.0848 |
0.0566 |
0.0661 |
0.0001 |
UR-IW-5 |
0.0586 |
0.1208 |
0.0654 |
0.0617 |
0.0001 |
mibi_rag_snippet |
0.0357 |
0.0988 |
0.0473 |
0.0454 |
0.0001 |
mibi_rag_abstract |
0.0284 |
0.0950 |
0.0387 |
0.0369 |
0.0001 |
mibi_rag_3 |
0.0322 |
0.1000 |
0.0422 |
0.0410 |
0.0001 |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
mibi_rag_4 |
0.0284 |
0.0950 |
0.0387 |
0.0369 |
0.0001 |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
mibi_rag_5 |
0.0334 |
0.0995 |
0.0451 |
0.0472 |
0.0001 |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
GTBioASQsys2 |
0.0068 |
0.0158 |
0.0089 |
0.0153 |
0.0000 |
GTBioASQsys3 |
0.0189 |
0.0313 |
0.0181 |
0.0235 |
0.0000 |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
dmiip2024 |
0.0745 |
0.3857 |
0.1114 |
0.3180 |
0.0028 |
dmiip2024_2 |
0.0735 |
0.3694 |
0.1099 |
0.3223 |
0.0027 |
dmiip2024_3 |
0.0771 |
0.4012 |
0.1159 |
0.3352 |
0.0043 |
dmiip2024_4 |
0.0782 |
0.4162 |
0.1191 |
0.3437 |
0.0043 |
Fleming-5 |
0.0319 |
0.1383 |
0.0459 |
0.0897 |
0.0002 |
Fleming-4 |
0.0105 |
0.1912 |
0.0191 |
0.0286 |
0.0002 |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
dmiip2024_1 |
0.0766 |
0.4010 |
0.1150 |
0.3137 |
0.0034 |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
mmb |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |
Concepts
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
mibi_rag_3 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
mibi_rag_4 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
mibi_rag_5 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
Fleming-5 |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
mmb |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |
RDF Triples
System |
Mean precision |
Recall |
F-Measure |
MAP |
GMAP |
IR1 |
- |
- |
- |
- |
- |
IR2 |
- |
- |
- |
- |
- |
first batch |
- |
- |
- |
- |
- |
Fleming-1 |
- |
- |
- |
- |
- |
Fleming-2 |
- |
- |
- |
- |
- |
UR-IW-1 |
- |
- |
- |
- |
- |
UR-IW-2 |
- |
- |
- |
- |
- |
UR-IW-3 |
- |
- |
- |
- |
- |
UR-IW-4 |
- |
- |
- |
- |
- |
UR-IW-5 |
- |
- |
- |
- |
- |
mibi_rag_snippet |
- |
- |
- |
- |
- |
mibi_rag_abstract |
- |
- |
- |
- |
- |
mibi_rag_3 |
- |
- |
- |
- |
- |
bioinfo-0 |
- |
- |
- |
- |
- |
bioinfo-1 |
- |
- |
- |
- |
- |
mibi_rag_4 |
- |
- |
- |
- |
- |
bioinfo-2 |
- |
- |
- |
- |
- |
bioinfo-3 |
- |
- |
- |
- |
- |
bioinfo-4 |
- |
- |
- |
- |
- |
mibi_rag_5 |
- |
- |
- |
- |
- |
bm25cross |
- |
- |
- |
- |
- |
gpt-bm25 |
- |
- |
- |
- |
- |
gpt-bm25-cross |
- |
- |
- |
- |
- |
2 |
- |
- |
- |
- |
- |
GTBioASQsys2 |
- |
- |
- |
- |
- |
GTBioASQsys3 |
- |
- |
- |
- |
- |
3 |
- |
- |
- |
- |
- |
4 |
- |
- |
- |
- |
- |
CPS |
- |
- |
- |
- |
- |
CPS2 |
- |
- |
- |
- |
- |
dmiip2024 |
- |
- |
- |
- |
- |
dmiip2024_2 |
- |
- |
- |
- |
- |
dmiip2024_3 |
- |
- |
- |
- |
- |
dmiip2024_4 |
- |
- |
- |
- |
- |
Fleming-5 |
- |
- |
- |
- |
- |
Fleming-4 |
- |
- |
- |
- |
- |
gpt-bm25-cross2 |
- |
- |
- |
- |
- |
simple truncation |
- |
- |
- |
- |
- |
dmiip2024_1 |
- |
- |
- |
- |
- |
HU-WBI-3 |
- |
- |
- |
- |
- |
HU-WBI-4 |
- |
- |
- |
- |
- |
HU-WBI-5 |
- |
- |
- |
- |
- |
HU-WBI-1 |
- |
- |
- |
- |
- |
HU-WBI-2 |
- |
- |
- |
- |
- |
qa |
- |
- |
- |
- |
- |
mmb |
- |
- |
- |
- |
- |
run1 |
- |
- |
- |
- |
- |
drums_bm |
- |
- |
- |
- |
- |
drums |
- |
- |
- |
- |
- |