BioASQ Participants Area
Task 13b: Test Results of Phase A
* The "-" replace the scores of systems that didn't submit the corresponding annotations. *The test results are presented in separate tables for each type of annotation. They are sorted based on the scores of MAP. The "System Description" of each system is used.
The evaluation measures that are used are presented here .
Test batch 1
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.2030 | 0.3477 | 0.1528 | 0.3349 | 0.0254 |
| IR1 | 0.3024 | 0.4103 | 0.2145 | 0.4594 | 0.1415 |
| UniTor_0 | 0.2376 | 0.3617 | 0.1715 | 0.3979 | 0.0766 |
| UniTor_1 | 0.3629 | 0.3307 | 0.2073 | 0.3711 | 0.0552 |
| Baseline top 20 | 0.3953 | 0.4688 | 0.2728 | 0.5627 | 0.3121 |
| Baseline top 10 | 0.3953 | 0.4688 | 0.2728 | 0.5627 | 0.3121 |
| Using LLM alone | 0.3694 | 0.4403 | 0.2514 | 0.5219 | 0.2729 |
| Using KG for list q | 0.4641 | 0.4230 | 0.3264 | 0.4800 | 0.1905 |
| Main pipeline | 0.4659 | 0.4256 | 0.3309 | 0.4796 | 0.1766 |
| 13b_phase_a | 0.0906 | 0.1465 | 0.0810 | 0.1190 | 0.0004 |
| 13b-1 | - | - | - | - | - |
| bioinfo-0 | 0.3259 | 0.4362 | 0.2467 | 0.4865 | 0.2568 |
| bioinfo-1 | 0.3271 | 0.4360 | 0.2467 | 0.4816 | 0.2525 |
| bioinfo-2 | 0.3353 | 0.4379 | 0.2501 | 0.5057 | 0.2410 |
| bioinfo-3 | 0.3200 | 0.4334 | 0.2418 | 0.4933 | 0.2345 |
| bioinfo-4 | 0.3318 | 0.4368 | 0.2479 | 0.5015 | 0.2627 |
| UniTor_2 | 0.2906 | 0.3845 | 0.2057 | 0.4196 | 0.0799 |
| UniTor_3 | 0.3953 | 0.3501 | 0.2208 | 0.3778 | 0.0722 |
| DB_vector_&_LLM | 0.0200 | 0.0056 | 0.0085 | 0.0096 | 0.0000 |
| UR-IW-1 | 0.3565 | 0.3045 | 0.2595 | 0.3726 | 0.0277 |
| UR-IW-2 | 0.3659 | 0.2931 | 0.2589 | 0.3652 | 0.0180 |
| UR-IW-3 | 0.3650 | 0.2752 | 0.2556 | 0.3534 | 0.0221 |
| UR-IW-4 | 0.2575 | 0.1964 | 0.1724 | 0.2724 | 0.0019 |
| google_serach_&_LLM | 0.0773 | 0.0303 | 0.0380 | 0.0430 | 0.0002 |
| UR-IW-5 | 0.4079 | 0.3428 | 0.3013 | 0.4216 | 0.0577 |
| no expand only rank | 0.1618 | 0.1291 | 0.1304 | 0.1359 | 0.0004 |
| DS@GT BioASQ 1b-1 | 0.1396 | 0.2154 | 0.1066 | 0.2387 | 0.0021 |
| IRIS_1 | 0.2153 | 0.3799 | 0.1688 | 0.3693 | 0.0851 |
| IRIS_3 | 0.0082 | 0.0171 | 0.0075 | 0.0056 | 0.0000 |
| IRIS_2 | 0.0282 | 0.0168 | 0.0159 | 0.0121 | 0.0001 |
| deepseek32b-me | 0.1902 | 0.1532 | 0.1508 | 0.1474 | 0.0003 |
| dmiip2024 | 0.3047 | 0.4236 | 0.2226 | 0.4765 | 0.1741 |
| dmiip2024_1 | 0.3094 | 0.4290 | 0.2250 | 0.4844 | 0.1760 |
| dmiip2024_2 | 0.3259 | 0.4229 | 0.2308 | 0.4851 | 0.1989 |
| dmiip2024_3 | 0.3318 | 0.4270 | 0.2355 | 0.4889 | 0.2057 |
| dmiip2024_4 | 0.3294 | 0.4217 | 0.2337 | 0.4926 | 0.2062 |
| config-1 | 0.2847 | 0.3653 | 0.1973 | 0.3959 | 0.0968 |
| config-2 | 0.2247 | 0.3447 | 0.1622 | 0.2831 | 0.0431 |
| config-3 | 0.2165 | 0.3329 | 0.1588 | 0.3109 | 0.0270 |
| config-4 | 0.2626 | 0.3651 | 0.1864 | 0.2302 | 0.0633 |
| config-5 | 0.2614 | 0.3650 | 0.1861 | 0.2262 | 0.0628 |
| mistral | 0.2755 | 0.2675 | 0.1999 | 0.2909 | 0.0124 |
| deepseek32b-full | 0.2355 | 0.1966 | 0.1812 | 0.1759 | 0.0023 |
| GPT4O | 0.0853 | 0.1075 | 0.0643 | 0.0742 | 0.0004 |
| deepseek-r1:32b | 0.0946 | 0.1633 | 0.0795 | 0.0840 | 0.0006 |
| deepseek-r1:14b | 0.0630 | 0.1076 | 0.0505 | 0.0875 | 0.0002 |
| using free 7b LLM | - | - | - | - | - |
| llama | 0.2890 | 0.2904 | 0.2300 | 0.3271 | 0.0186 |
| dense | 0.1178 | 0.1746 | 0.0833 | 0.1619 | 0.0033 |
| deepseek-r1:8b | 0.0805 | 0.1682 | 0.0765 | 0.1002 | 0.0006 |
| lasigeBioTM | 0.2188 | 0.3619 | 0.1650 | 0.3827 | 0.0733 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.1412 | 0.1076 | 0.0876 | 0.1867 | 0.0055 |
| IR1 | - | - | - | - | - |
| UniTor_0 | 0.3118 | 0.2078 | 0.1945 | 0.4130 | 0.0477 |
| UniTor_1 | 0.2993 | 0.2111 | 0.1897 | 0.4199 | 0.0494 |
| Baseline top 20 | 0.3648 | 0.2422 | 0.2314 | 0.4040 | 0.1150 |
| Baseline top 10 | 0.3648 | 0.2422 | 0.2314 | 0.4040 | 0.1150 |
| Using LLM alone | 0.3389 | 0.2241 | 0.2140 | 0.3745 | 0.1158 |
| Using KG for list q | 0.3971 | 0.2100 | 0.2270 | 0.3470 | 0.0818 |
| Main pipeline | 0.4081 | 0.2176 | 0.2366 | 0.3499 | 0.0832 |
| 13b_phase_a | 0.0619 | 0.0195 | 0.0235 | 0.0605 | 0.0001 |
| 13b-1 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | 0.3109 | 0.2181 | 0.1925 | 0.4231 | 0.0579 |
| UniTor_3 | 0.3093 | 0.2291 | 0.1919 | 0.4288 | 0.0660 |
| DB_vector_&_LLM | 0.0288 | 0.0067 | 0.0105 | 0.0121 | 0.0000 |
| UR-IW-1 | 0.3017 | 0.1589 | 0.1689 | 0.4009 | 0.0152 |
| UR-IW-2 | 0.3346 | 0.1681 | 0.1865 | 0.3947 | 0.0153 |
| UR-IW-3 | 0.3082 | 0.1590 | 0.1739 | 0.3994 | 0.0137 |
| UR-IW-4 | 0.2447 | 0.1171 | 0.1311 | 0.2878 | 0.0014 |
| google_serach_&_LLM | 0.1797 | 0.0536 | 0.0726 | 0.0327 | 0.0002 |
| UR-IW-5 | 0.3447 | 0.1835 | 0.1972 | 0.4298 | 0.0217 |
| no expand only rank | 0.0805 | 0.0148 | 0.0225 | 0.0533 | 0.0001 |
| DS@GT BioASQ 1b-1 | 0.1280 | 0.1421 | 0.1011 | 0.1620 | 0.0004 |
| IRIS_1 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| deepseek32b-me | 0.1301 | 0.0499 | 0.0654 | 0.1130 | 0.0002 |
| dmiip2024 | 0.2663 | 0.3248 | 0.2003 | 0.6156 | 0.1195 |
| dmiip2024_1 | 0.2687 | 0.3308 | 0.2042 | 0.6137 | 0.1113 |
| dmiip2024_2 | 0.2763 | 0.3285 | 0.2068 | 0.6062 | 0.1003 |
| dmiip2024_3 | 0.2805 | 0.3275 | 0.2085 | 0.5935 | 0.1137 |
| dmiip2024_4 | 0.2721 | 0.3193 | 0.2027 | 0.6020 | 0.0978 |
| config-1 | 0.1688 | 0.0797 | 0.0812 | 0.1826 | 0.0067 |
| config-2 | 0.1437 | 0.0578 | 0.0625 | 0.1254 | 0.0024 |
| config-3 | 0.1283 | 0.0550 | 0.0595 | 0.1281 | 0.0015 |
| config-4 | 0.1665 | 0.0751 | 0.0778 | 0.1111 | 0.0045 |
| config-5 | 0.1662 | 0.0751 | 0.0779 | 0.1101 | 0.0044 |
| mistral | 0.2562 | 0.0749 | 0.1040 | 0.1865 | 0.0033 |
| deepseek32b-full | 0.1665 | 0.0593 | 0.0785 | 0.1340 | 0.0007 |
| GPT4O | 0.0625 | 0.0194 | 0.0240 | 0.0410 | 0.0001 |
| deepseek-r1:32b | 0.0612 | 0.0353 | 0.0337 | 0.0504 | 0.0002 |
| deepseek-r1:14b | 0.0467 | 0.0302 | 0.0277 | 0.0553 | 0.0001 |
| using free 7b LLM | - | - | - | - | - |
| llama | 0.2729 | 0.1056 | 0.1341 | 0.2407 | 0.0081 |
| dense | 0.1659 | 0.1106 | 0.0904 | 0.2330 | 0.0019 |
| deepseek-r1:8b | 0.0525 | 0.0447 | 0.0358 | 0.0609 | 0.0002 |
| lasigeBioTM | 0.1012 | 0.0615 | 0.0574 | 0.1817 | 0.0043 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| 13b_phase_a | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| DB_vector_&_LLM | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| google_serach_&_LLM | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| no expand only rank | - | - | - | - | - |
| DS@GT BioASQ 1b-1 | - | - | - | - | - |
| IRIS_1 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| config-1 | - | - | - | - | - |
| config-2 | - | - | - | - | - |
| config-3 | - | - | - | - | - |
| config-4 | - | - | - | - | - |
| config-5 | - | - | - | - | - |
| mistral | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| using free 7b LLM | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| 13b_phase_a | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| DB_vector_&_LLM | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| google_serach_&_LLM | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| no expand only rank | - | - | - | - | - |
| DS@GT BioASQ 1b-1 | - | - | - | - | - |
| IRIS_1 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| config-1 | - | - | - | - | - |
| config-2 | - | - | - | - | - |
| config-3 | - | - | - | - | - |
| config-4 | - | - | - | - | - |
| config-5 | - | - | - | - | - |
| mistral | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| using free 7b LLM | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
Test batch 2
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.2941 | 0.4245 | 0.2103 | 0.4449 | 0.0965 |
| Fleming-2 | 0.3089 | 0.4245 | 0.2240 | 0.4317 | 0.0893 |
| UniTor_0 | 0.2695 | 0.3289 | 0.1672 | 0.4210 | 0.0447 |
| UniTor_1 | 0.3982 | 0.3000 | 0.2016 | 0.3841 | 0.0413 |
| UniTor_2 | 0.2859 | 0.3360 | 0.1780 | 0.4468 | 0.0526 |
| UniTor_3 | 0.4254 | 0.3130 | 0.2140 | 0.3996 | 0.0485 |
| UR-IW-1 | 0.4596 | 0.2847 | 0.2796 | 0.4405 | 0.0476 |
| UR-IW-2 | 0.4161 | 0.2586 | 0.2417 | 0.4167 | 0.0250 |
| UR-IW-3 | 0.4085 | 0.2926 | 0.2791 | 0.4034 | 0.0245 |
| UR-IW-4 | 0.4670 | 0.3041 | 0.2804 | 0.4778 | 0.0592 |
| Baseline top 10 | 0.4635 | 0.4888 | 0.2922 | 0.6768 | 0.3837 |
| Baseline top 20 | 0.4635 | 0.4888 | 0.2922 | 0.6768 | 0.3837 |
| Main pipeline | 0.5646 | 0.4121 | 0.3636 | 0.5423 | 0.2186 |
| Using KG for list q | 0.5650 | 0.3984 | 0.3586 | 0.5299 | 0.1906 |
| Using LLM alone | 0.4082 | 0.4581 | 0.2552 | 0.5945 | 0.2823 |
| dmiip2024 | 0.4059 | 0.4834 | 0.2692 | 0.6167 | 0.3445 |
| dmiip2024_1 | 0.4047 | 0.4827 | 0.2682 | 0.6145 | 0.3393 |
| dmiip2024_2 | 0.4271 | 0.4711 | 0.2774 | 0.6263 | 0.3217 |
| dmiip2024_3 | 0.4259 | 0.4698 | 0.2759 | 0.6212 | 0.3176 |
| dmiip2024_4 | 0.4224 | 0.4701 | 0.2758 | 0.6245 | 0.3202 |
| bioinfo-0 | 0.4106 | 0.4717 | 0.2699 | 0.6183 | 0.2800 |
| bioinfo-1 | 0.4365 | 0.4869 | 0.2846 | 0.6424 | 0.3932 |
| bioinfo-2 | 0.4106 | 0.4717 | 0.2699 | 0.6183 | 0.2800 |
| bioinfo-3 | 0.4071 | 0.4704 | 0.2684 | 0.6161 | 0.3009 |
| bioinfo-4 | 0.4035 | 0.4700 | 0.2673 | 0.6066 | 0.2936 |
| IR5 | 0.2659 | 0.3824 | 0.1861 | 0.4285 | 0.0932 |
| lasigeBioTM | - | - | - | - | - |
| UR-IW-5 | 0.5113 | 0.2988 | 0.3117 | 0.4745 | 0.0561 |
| deepseek32b-full | 0.3347 | 0.2031 | 0.1910 | 0.3029 | 0.0108 |
| mistral | 0.2962 | 0.1643 | 0.1640 | 0.2716 | 0.0054 |
| llama | 0.4120 | 0.2593 | 0.2632 | 0.3936 | 0.0344 |
| dense | 0.4089 | 0.2618 | 0.2562 | 0.4055 | 0.0303 |
| GPT4O | 0.1518 | 0.1053 | 0.0792 | 0.1634 | 0.0009 |
| deepseek-r1:32b | 0.2598 | 0.2027 | 0.1533 | 0.2841 | 0.0103 |
| deepseek-r1:14b | 0.1253 | 0.0765 | 0.0641 | 0.0802 | 0.0008 |
| deepseek-r1:8b | 0.1384 | 0.1166 | 0.0880 | 0.1134 | 0.0006 |
| gpt 01 mini | 0.1660 | 0.1363 | 0.1025 | 0.1166 | 0.0016 |
| 13b-1 | 0.2455 | 0.2224 | 0.1642 | 0.2829 | 0.0071 |
| IR1 | 0.3744 | 0.4106 | 0.2318 | 0.5454 | 0.1657 |
| Fleming-3 | 0.3100 | 0.4250 | 0.2247 | 0.4329 | 0.0996 |
| deepseek32b-me | 0.3009 | 0.1465 | 0.1532 | 0.2096 | 0.0050 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.2040 | 0.1756 | 0.1335 | 0.2974 | 0.0302 |
| Fleming-2 | 0.2058 | 0.1199 | 0.1058 | 0.2411 | 0.0099 |
| UniTor_0 | 0.3075 | 0.1961 | 0.1741 | 0.4331 | 0.0376 |
| UniTor_1 | 0.2933 | 0.1971 | 0.1659 | 0.4366 | 0.0385 |
| UniTor_2 | 0.3217 | 0.2165 | 0.1879 | 0.4564 | 0.0421 |
| UniTor_3 | 0.3065 | 0.2185 | 0.1798 | 0.4604 | 0.0435 |
| UR-IW-1 | 0.3939 | 0.1680 | 0.1966 | 0.4737 | 0.0333 |
| UR-IW-2 | 0.3731 | 0.1437 | 0.1744 | 0.3897 | 0.0208 |
| UR-IW-3 | 0.3626 | 0.1605 | 0.1852 | 0.4504 | 0.0248 |
| UR-IW-4 | 0.4125 | 0.1610 | 0.1941 | 0.4654 | 0.0325 |
| Baseline top 10 | 0.4450 | 0.2813 | 0.2729 | 0.5267 | 0.2319 |
| Baseline top 20 | 0.4450 | 0.2813 | 0.2729 | 0.5267 | 0.2319 |
| Main pipeline | 0.4913 | 0.2389 | 0.2746 | 0.4330 | 0.0912 |
| Using KG for list q | 0.4909 | 0.2330 | 0.2684 | 0.4157 | 0.0858 |
| Using LLM alone | 0.3987 | 0.2489 | 0.2371 | 0.4596 | 0.1562 |
| dmiip2024 | 0.2712 | 0.3346 | 0.2099 | 0.6604 | 0.1676 |
| dmiip2024_1 | 0.2785 | 0.3376 | 0.2149 | 0.6860 | 0.1896 |
| dmiip2024_2 | 0.2649 | 0.3297 | 0.2061 | 0.6462 | 0.1411 |
| dmiip2024_3 | 0.2698 | 0.3225 | 0.2040 | 0.6464 | 0.1504 |
| dmiip2024_4 | 0.2671 | 0.3331 | 0.2089 | 0.6402 | 0.1533 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| lasigeBioTM | 0.2609 | 0.0831 | 0.0931 | 0.2184 | 0.0058 |
| UR-IW-5 | 0.4345 | 0.1654 | 0.2020 | 0.5208 | 0.0393 |
| deepseek32b-full | 0.2735 | 0.1032 | 0.1234 | 0.3136 | 0.0065 |
| mistral | 0.2587 | 0.0653 | 0.0954 | 0.1872 | 0.0022 |
| llama | 0.3801 | 0.1257 | 0.1707 | 0.3612 | 0.0206 |
| dense | 0.3690 | 0.1264 | 0.1674 | 0.3680 | 0.0186 |
| GPT4O | 0.1227 | 0.0388 | 0.0463 | 0.1126 | 0.0004 |
| deepseek-r1:32b | 0.1986 | 0.0621 | 0.0780 | 0.1999 | 0.0029 |
| deepseek-r1:14b | 0.0989 | 0.0293 | 0.0390 | 0.0551 | 0.0003 |
| deepseek-r1:8b | 0.0991 | 0.0407 | 0.0457 | 0.0795 | 0.0002 |
| gpt 01 mini | 0.1125 | 0.0399 | 0.0489 | 0.0679 | 0.0006 |
| 13b-1 | 0.2348 | 0.0942 | 0.1065 | 0.2349 | 0.0033 |
| IR1 | 0.3503 | 0.2150 | 0.1912 | 0.4698 | 0.0772 |
| Fleming-3 | 0.2058 | 0.1199 | 0.1058 | 0.2411 | 0.0099 |
| deepseek32b-me | 0.2540 | 0.0680 | 0.0906 | 0.2204 | 0.0036 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| Fleming-2 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| mistral | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| gpt 01 mini | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| Fleming-3 | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| Fleming-2 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| mistral | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| gpt 01 mini | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| Fleming-3 | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
Test batch 3
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| NLP_BioBert | 0.0576 | 0.0251 | 0.0307 | 0.0403 | 0.0001 |
| NLP_Group5_TU | 0.0576 | 0.0251 | 0.0307 | 0.0403 | 0.0001 |
| Fleming-1 | 0.3063 | 0.2996 | 0.2093 | 0.3288 | 0.0526 |
| IR5 | 0.2718 | 0.2783 | 0.1816 | 0.3250 | 0.0592 |
| IR1 | 0.4071 | 0.3471 | 0.2575 | 0.4749 | 0.2414 |
| IR4 | 0.4094 | 0.3505 | 0.2613 | 0.4681 | 0.2014 |
| UR-IW-1 | 0.4478 | 0.2473 | 0.2580 | 0.3975 | 0.0560 |
| UR-IW-2 | 0.3654 | 0.1926 | 0.2114 | 0.3224 | 0.0235 |
| UR-IW-4 | 0.3778 | 0.1824 | 0.2144 | 0.3127 | 0.0172 |
| UR-IW-5 | 0.4804 | 0.2641 | 0.2969 | 0.4066 | 0.0615 |
| lasigeBioTM | 0.2714 | 0.1486 | 0.1462 | 0.2265 | 0.0031 |
| lasigeBioTM-onto-sm | 0.2714 | 0.1486 | 0.1462 | 0.2265 | 0.0031 |
| Baseline top 10 | 0.4959 | 0.3977 | 0.3139 | 0.5757 | 0.3983 |
| Using LLM alone | 0.4641 | 0.3753 | 0.2905 | 0.5193 | 0.3428 |
| Using KG for list q | 0.5065 | 0.3308 | 0.3188 | 0.4571 | 0.1211 |
| Main pipeline | 0.5093 | 0.3261 | 0.3143 | 0.4513 | 0.1180 |
| Baseline top 20 | 0.4959 | 0.3977 | 0.3139 | 0.5757 | 0.3983 |
| UniTor_0 | 0.2855 | 0.2628 | 0.1759 | 0.3554 | 0.0408 |
| UniTor_1 | 0.3980 | 0.2422 | 0.1990 | 0.3067 | 0.0342 |
| bioinfo-0 | 0.4347 | 0.3676 | 0.2778 | 0.5229 | 0.2641 |
| bioinfo-1 | 0.4565 | 0.3784 | 0.2887 | 0.5410 | 0.2545 |
| bioinfo-2 | 0.4194 | 0.3614 | 0.2693 | 0.5123 | 0.2551 |
| bioinfo-3 | 0.4312 | 0.3651 | 0.2746 | 0.5168 | 0.2629 |
| bioinfo-4 | 0.4341 | 0.3677 | 0.2754 | 0.5212 | 0.2648 |
| UniTor_2 | 0.3271 | 0.2836 | 0.2011 | 0.3868 | 0.0515 |
| UniTor_3 | 0.4243 | 0.2518 | 0.2035 | 0.3216 | 0.0392 |
| dmiip2024 | 0.4518 | 0.3800 | 0.2892 | 0.5283 | 0.3132 |
| dmiip2024_1 | 0.4318 | 0.3724 | 0.2783 | 0.5178 | 0.2739 |
| dmiip2024_2 | 0.4659 | 0.3719 | 0.2922 | 0.5464 | 0.2869 |
| dmiip2024_4 | 0.4682 | 0.3722 | 0.2931 | 0.5490 | 0.2664 |
| dmiip2024_3 | 0.4494 | 0.3625 | 0.2808 | 0.5332 | 0.2543 |
| UR-IW-3 | 0.3596 | 0.1936 | 0.2031 | 0.3170 | 0.0300 |
| lasigeBioTM-onto-bl | 0.2714 | 0.1486 | 0.1462 | 0.2265 | 0.0031 |
| IRIS_1 | 0.3235 | 0.2895 | 0.2099 | 0.3711 | 0.0581 |
| IRIS_2 | 0.0365 | 0.0099 | 0.0145 | 0.0138 | 0.0001 |
| IRIS_3 | 0.0494 | 0.0234 | 0.0201 | 0.0180 | 0.0001 |
| 13b-1 | - | - | - | - | - |
| 13b_phase_a | 0.1421 | 0.0899 | 0.0790 | 0.1085 | 0.0006 |
| google_serach_&_LLM | 0.0376 | 0.0140 | 0.0161 | 0.0249 | 0.0000 |
| mistral | 0.4165 | 0.2132 | 0.2405 | 0.3324 | 0.0330 |
| llama | 0.4139 | 0.2084 | 0.2244 | 0.3080 | 0.0318 |
| dense | 0.4045 | 0.2066 | 0.2334 | 0.3184 | 0.0302 |
| GPT4O | 0.1399 | 0.0827 | 0.0852 | 0.0822 | 0.0007 |
| deepseek-r1:32b | 0.1304 | 0.0819 | 0.0815 | 0.0991 | 0.0005 |
| deepseek-r1:14b | 0.1695 | 0.1356 | 0.1156 | 0.1165 | 0.0015 |
| sp_lasigebiotm | 0.2714 | 0.1486 | 0.1462 | 0.2265 | 0.0031 |
| AQAMS | 0.0951 | 0.0289 | 0.0394 | 0.0430 | 0.0001 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| NLP_BioBert | 0.0907 | 0.0221 | 0.0328 | 0.0342 | 0.0001 |
| NLP_Group5_TU | 0.0907 | 0.0221 | 0.0328 | 0.0342 | 0.0001 |
| Fleming-1 | 0.2200 | 0.1020 | 0.1070 | 0.2028 | 0.0110 |
| IR5 | - | - | - | - | - |
| IR1 | 0.4157 | 0.1962 | 0.2197 | 0.4155 | 0.0725 |
| IR4 | 0.4570 | 0.2215 | 0.2469 | 0.4633 | 0.1086 |
| UR-IW-1 | 0.4221 | 0.1490 | 0.1897 | 0.3558 | 0.0338 |
| UR-IW-2 | 0.3772 | 0.1132 | 0.1508 | 0.2934 | 0.0107 |
| UR-IW-4 | 0.3739 | 0.1117 | 0.1509 | 0.2941 | 0.0104 |
| UR-IW-5 | 0.4520 | 0.1621 | 0.2115 | 0.3732 | 0.0353 |
| lasigeBioTM | 0.1918 | 0.0462 | 0.0645 | 0.1181 | 0.0006 |
| lasigeBioTM-onto-sm | 0.1918 | 0.0462 | 0.0645 | 0.1181 | 0.0006 |
| Baseline top 10 | 0.4600 | 0.2236 | 0.2501 | 0.4597 | 0.2228 |
| Using LLM alone | 0.4166 | 0.2022 | 0.2218 | 0.4111 | 0.1829 |
| Using KG for list q | 0.4565 | 0.1958 | 0.2355 | 0.3554 | 0.0660 |
| Main pipeline | 0.4531 | 0.1937 | 0.2319 | 0.3467 | 0.0684 |
| Baseline top 20 | 0.4600 | 0.2236 | 0.2501 | 0.4597 | 0.2228 |
| UniTor_0 | 0.3354 | 0.1485 | 0.1601 | 0.3495 | 0.0238 |
| UniTor_1 | 0.3327 | 0.1540 | 0.1616 | 0.3524 | 0.0246 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | 0.3357 | 0.1675 | 0.1735 | 0.3637 | 0.0277 |
| UniTor_3 | 0.3268 | 0.1677 | 0.1686 | 0.3638 | 0.0299 |
| dmiip2024 | 0.3465 | 0.2917 | 0.2391 | 0.6352 | 0.1546 |
| dmiip2024_1 | 0.3520 | 0.3001 | 0.2436 | 0.6425 | 0.1773 |
| dmiip2024_2 | 0.3519 | 0.3059 | 0.2439 | 0.6296 | 0.1365 |
| dmiip2024_4 | 0.3493 | 0.3035 | 0.2420 | 0.6235 | 0.1387 |
| dmiip2024_3 | 0.3630 | 0.3023 | 0.2467 | 0.6446 | 0.1439 |
| UR-IW-3 | 0.3207 | 0.1246 | 0.1532 | 0.3096 | 0.0141 |
| lasigeBioTM-onto-bl | 0.1918 | 0.0462 | 0.0645 | 0.1181 | 0.0006 |
| IRIS_1 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| 13b_phase_a | 0.0604 | 0.0063 | 0.0108 | 0.0356 | 0.0001 |
| google_serach_&_LLM | 0.1837 | 0.0763 | 0.0938 | 0.0533 | 0.0002 |
| mistral | 0.3965 | 0.1173 | 0.1574 | 0.2743 | 0.0193 |
| llama | 0.4109 | 0.1150 | 0.1534 | 0.2445 | 0.0161 |
| dense | 0.3843 | 0.1066 | 0.1463 | 0.2647 | 0.0176 |
| GPT4O | 0.1165 | 0.0350 | 0.0459 | 0.0470 | 0.0003 |
| deepseek-r1:32b | 0.1041 | 0.0310 | 0.0429 | 0.0503 | 0.0002 |
| deepseek-r1:14b | 0.1328 | 0.0382 | 0.0538 | 0.0642 | 0.0006 |
| sp_lasigebiotm | 0.1918 | 0.0462 | 0.0645 | 0.1181 | 0.0006 |
| AQAMS | 0.1094 | 0.1348 | 0.0985 | 0.0398 | 0.0001 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| NLP_BioBert | - | - | - | - | - |
| NLP_Group5_TU | - | - | - | - | - |
| Fleming-1 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| IR4 | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| lasigeBioTM-onto-sm | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| lasigeBioTM-onto-bl | - | - | - | - | - |
| IRIS_1 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| 13b_phase_a | - | - | - | - | - |
| google_serach_&_LLM | - | - | - | - | - |
| mistral | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| sp_lasigebiotm | - | - | - | - | - |
| AQAMS | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| NLP_BioBert | - | - | - | - | - |
| NLP_Group5_TU | - | - | - | - | - |
| Fleming-1 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| IR4 | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| lasigeBioTM-onto-sm | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| lasigeBioTM-onto-bl | - | - | - | - | - |
| IRIS_1 | - | - | - | - | - |
| IRIS_2 | - | - | - | - | - |
| IRIS_3 | - | - | - | - | - |
| 13b-1 | - | - | - | - | - |
| 13b_phase_a | - | - | - | - | - |
| google_serach_&_LLM | - | - | - | - | - |
| mistral | - | - | - | - | - |
| llama | - | - | - | - | - |
| dense | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| sp_lasigebiotm | - | - | - | - | - |
| AQAMS | - | - | - | - | - |
Test batch 4
Documents
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.3172 | 0.1289 | 0.1504 | 0.2524 | 0.0223 |
| Data Wizards BM25 | 0.0376 | 0.0109 | 0.0158 | 0.0234 | 0.0001 |
| NLP_BioBert | 0.0718 | 0.0220 | 0.0310 | 0.0467 | 0.0002 |
| Data Wizards Re-Rank | 0.1078 | 0.0062 | 0.0113 | 0.0180 | 0.0001 |
| First abrotsch | 0.0706 | 0.0232 | 0.0321 | 0.0441 | 0.0002 |
| ms-marco-MiniLML6v2 | 0.0706 | 0.0232 | 0.0321 | 0.0441 | 0.0002 |
| IR5 | 0.2918 | 0.1180 | 0.1374 | 0.2123 | 0.0249 |
| BM25-Okapi | 0.0600 | 0.0195 | 0.0276 | 0.0366 | 0.0001 |
| IR1 | 0.4471 | 0.1641 | 0.2029 | 0.3763 | 0.1165 |
| IR4 | 0.4353 | 0.1674 | 0.2037 | 0.3592 | 0.1195 |
| UniTor_0 | 0.2706 | 0.0899 | 0.1157 | 0.2195 | 0.0183 |
| UniTor_1 | 0.3880 | 0.0784 | 0.1177 | 0.1917 | 0.0119 |
| Reranking rankings | 0.1489 | 0.0641 | 0.0710 | 0.0943 | 0.0010 |
| query-engineering | 0.1242 | 0.0549 | 0.0582 | 0.0771 | 0.0005 |
| Re-Ranker with BM25 | 0.0988 | 0.0291 | 0.0415 | 0.0698 | 0.0004 |
| RetrievalRivals | 0.2247 | 0.0951 | 0.1089 | 0.1532 | 0.0074 |
| RetrievalRivals_BM25 | 0.2247 | 0.0951 | 0.1089 | 0.1532 | 0.0074 |
| RetrieverRivals | 0.2247 | 0.0951 | 0.1089 | 0.1532 | 0.0074 |
| UniTor_2 | 0.3635 | 0.1289 | 0.1606 | 0.3002 | 0.0353 |
| UniTor_3 | 0.4303 | 0.0967 | 0.1329 | 0.2216 | 0.0203 |
| TEsting123 | 0.0676 | 0.0303 | 0.0317 | 0.0300 | 0.0001 |
| Traditional IR Methd | 0.0676 | 0.0303 | 0.0317 | 0.0300 | 0.0001 |
| traditional IR | 0.1569 | 0.0315 | 0.0422 | 0.0641 | 0.0006 |
| Neural IR approach | 0.1878 | 0.0346 | 0.0534 | 0.0883 | 0.0008 |
| Machinen Results | 0.2953 | 0.1079 | 0.1360 | 0.2302 | 0.0232 |
| Tuwien Group 6 AIR | 0.1878 | 0.0346 | 0.0534 | 0.0883 | 0.0008 |
| reranking after IR | 0.1539 | 0.0111 | 0.0195 | 0.0333 | 0.0001 |
| bioinfo-0 | 0.4776 | 0.1829 | 0.2244 | 0.4139 | 0.1311 |
| bioinfo-1 | 0.5129 | 0.1983 | 0.2442 | 0.4342 | 0.2073 |
| bioinfo-2 | 0.4635 | 0.1785 | 0.2181 | 0.3998 | 0.1391 |
| bioinfo-3 | 0.4741 | 0.1817 | 0.2227 | 0.4107 | 0.1387 |
| bioinfo-4 | 0.4624 | 0.1783 | 0.2178 | 0.3992 | 0.1385 |
| Okapi BM25 IR | 0.0442 | 0.0142 | 0.0201 | 0.0229 | 0.0001 |
| NNR Model | 0.0443 | 0.0143 | 0.0203 | 0.0229 | 0.0001 |
| GoldenRetrievers | 0.1802 | 0.0715 | 0.0819 | 0.1349 | 0.0047 |
| Using KG for list q | 0.5434 | 0.1818 | 0.2336 | 0.3990 | 0.1491 |
| Baseline top 20 | 0.5622 | 0.2107 | 0.2621 | 0.4847 | 0.2452 |
| Baseline top 10 | 0.5622 | 0.2107 | 0.2621 | 0.4847 | 0.2452 |
| Using LLM alone | 0.5022 | 0.1905 | 0.2334 | 0.3930 | 0.1934 |
| Main pipeline | 0.5520 | 0.1860 | 0.2391 | 0.4051 | 0.1865 |
| dmiip2024 | 0.4635 | 0.1756 | 0.2171 | 0.3932 | 0.1655 |
| dmiip2024_1 | 0.4624 | 0.1742 | 0.2153 | 0.3917 | 0.1661 |
| dmiip2024_2 | 0.4976 | 0.1840 | 0.2340 | 0.4190 | 0.1462 |
| dmiip2024_3 | 0.4965 | 0.1819 | 0.2317 | 0.4142 | 0.1438 |
| dmiip2024_4 | 0.5012 | 0.1833 | 0.2336 | 0.4211 | 0.1332 |
| AQAMS | 0.1680 | 0.0581 | 0.0727 | 0.0903 | 0.0026 |
| deepseek32b-me | 0.3215 | 0.1005 | 0.1346 | 0.2198 | 0.0198 |
| deepseek32b-full | 0.2542 | 0.0804 | 0.1040 | 0.1488 | 0.0081 |
| DB_vector_&_LLM | 0.2298 | 0.0869 | 0.1045 | 0.1396 | 0.0086 |
| UR-IW-1 | 0.3617 | 0.1283 | 0.1605 | 0.2998 | 0.0174 |
| UR-IW-2 | 0.4047 | 0.1260 | 0.1735 | 0.3373 | 0.0376 |
| UR-IW-3 | 0.4052 | 0.1169 | 0.1640 | 0.3379 | 0.0340 |
| UR-IW-4 | 0.4419 | 0.1487 | 0.1942 | 0.3785 | 0.0569 |
| UR-IW-5 | 0.4350 | 0.1448 | 0.1918 | 0.3589 | 0.0866 |
| The Relevants | 0.1867 | 0.0350 | 0.0538 | 0.0875 | 0.0008 |
| Data Wizards NN | 0.0753 | 0.0218 | 0.0311 | 0.0499 | 0.0002 |
| API and weirdnesss | 0.1773 | 0.0704 | 0.0829 | 0.1016 | 0.0024 |
| DS@GT BioASQ 1b-1 | 0.4200 | 0.1636 | 0.1948 | 0.3483 | 0.1391 |
| Neural Re-Ranking | 0.0400 | 0.0118 | 0.0167 | 0.0164 | 0.0001 |
| Approach 1 | 0.0918 | 0.0269 | 0.0383 | 0.0638 | 0.0002 |
| DS@GT BioASQ 1b-2 | 0.4306 | 0.1661 | 0.1980 | 0.3577 | 0.1694 |
| GoldenRetrievers-2 | 0.1910 | 0.0617 | 0.0802 | 0.1043 | 0.0020 |
| dl approach | 0.0419 | 0.0138 | 0.0192 | 0.0238 | 0.0001 |
| GPT4O | 0.3242 | 0.1271 | 0.1536 | 0.2789 | 0.0319 |
| DS@GTBioASQT13b5 | 0.4318 | 0.1643 | 0.1970 | 0.3600 | 0.1550 |
| sp_lasigebiotm | 0.2573 | 0.0711 | 0.0952 | 0.1313 | 0.0043 |
| mistral | 0.3605 | 0.0999 | 0.1400 | 0.2626 | 0.0341 |
| lasigeBioTM | 0.2573 | 0.0711 | 0.0952 | 0.1313 | 0.0043 |
| llama | 0.3150 | 0.0833 | 0.1163 | 0.2065 | 0.0234 |
| lasigeBioTM-onto-bl | 0.2573 | 0.0711 | 0.0952 | 0.1313 | 0.0043 |
| lasigeBioTM-onto-sm | 0.2573 | 0.0711 | 0.0952 | 0.1313 | 0.0043 |
| 1.PhaseA_System | 0.0688 | 0.0267 | 0.0298 | 0.0368 | 0.0001 |
| deepseek-r1:32b | 0.1906 | 0.0779 | 0.0944 | 0.0942 | 0.0038 |
| deepseek-r1:14b | 0.1906 | 0.0779 | 0.0944 | 0.0942 | 0.0038 |
| deepseek-r1:8b | 0.1561 | 0.0538 | 0.0672 | 0.0747 | 0.0011 |
| DS@GTBioASQT13b4 | 0.4759 | 0.1874 | 0.2260 | 0.4096 | 0.1968 |
| DS@GTBioASQT13b3 | 0.4800 | 0.1875 | 0.2264 | 0.4137 | 0.1993 |
| dense | 0.3677 | 0.1186 | 0.1602 | 0.2694 | 0.0418 |
| Fleming-2 | 0.3172 | 0.1289 | 0.1504 | 0.2579 | 0.0224 |
Snippets
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | 0.2216 | 0.0569 | 0.0775 | 0.1788 | 0.0070 |
| Data Wizards BM25 | 0.0392 | 0.0074 | 0.0117 | 0.0398 | 0.0000 |
| NLP_BioBert | 0.0981 | 0.0147 | 0.0242 | 0.0420 | 0.0001 |
| Data Wizards Re-Rank | - | - | - | - | - |
| First abrotsch | - | - | - | - | - |
| ms-marco-MiniLML6v2 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| BM25-Okapi | - | - | - | - | - |
| IR1 | 0.4392 | 0.1158 | 0.1568 | 0.3702 | 0.0935 |
| IR4 | 0.4621 | 0.1302 | 0.1763 | 0.4061 | 0.1265 |
| UniTor_0 | 0.3216 | 0.0796 | 0.1101 | 0.2327 | 0.0105 |
| UniTor_1 | 0.3156 | 0.0826 | 0.1144 | 0.2419 | 0.0108 |
| Reranking rankings | 0.1544 | 0.0452 | 0.0567 | 0.1272 | 0.0011 |
| query-engineering | 0.0935 | 0.0220 | 0.0329 | 0.0751 | 0.0002 |
| Re-Ranker with BM25 | 0.0369 | 0.0200 | 0.0238 | 0.0329 | 0.0001 |
| RetrievalRivals | 0.2082 | 0.0033 | 0.0064 | 0.0185 | 0.0001 |
| RetrievalRivals_BM25 | 0.2082 | 0.0033 | 0.0064 | 0.0185 | 0.0001 |
| RetrieverRivals | 0.2082 | 0.0033 | 0.0064 | 0.0185 | 0.0001 |
| UniTor_2 | 0.3456 | 0.0981 | 0.1306 | 0.2797 | 0.0217 |
| UniTor_3 | 0.3393 | 0.0981 | 0.1283 | 0.2801 | 0.0218 |
| TEsting123 | 0.0512 | 0.0128 | 0.0174 | 0.0123 | 0.0001 |
| Traditional IR Methd | 0.0512 | 0.0128 | 0.0174 | 0.0123 | 0.0001 |
| traditional IR | 0.0749 | 0.0068 | 0.0118 | 0.0351 | 0.0001 |
| Neural IR approach | 0.0958 | 0.0172 | 0.0278 | 0.0760 | 0.0004 |
| Machinen Results | - | - | - | - | - |
| Tuwien Group 6 AIR | 0.0958 | 0.0172 | 0.0278 | 0.0760 | 0.0004 |
| reranking after IR | 0.0749 | 0.0068 | 0.0118 | 0.0448 | 0.0001 |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| Okapi BM25 IR | 0.0163 | 0.0234 | 0.0175 | 0.0082 | 0.0001 |
| NNR Model | 0.0171 | 0.0243 | 0.0185 | 0.0082 | 0.0001 |
| GoldenRetrievers | - | - | - | - | - |
| Using KG for list q | 0.5055 | 0.1364 | 0.1916 | 0.3557 | 0.1237 |
| Baseline top 20 | 0.5210 | 0.1606 | 0.2164 | 0.4178 | 0.2039 |
| Baseline top 10 | 0.5210 | 0.1606 | 0.2164 | 0.4178 | 0.2039 |
| Using LLM alone | 0.4695 | 0.1451 | 0.1942 | 0.3481 | 0.1551 |
| Main pipeline | 0.5040 | 0.1381 | 0.1926 | 0.3599 | 0.1402 |
| dmiip2024 | 0.3161 | 0.1185 | 0.1354 | 0.2930 | 0.0524 |
| dmiip2024_1 | 0.3152 | 0.1187 | 0.1358 | 0.3011 | 0.0547 |
| dmiip2024_2 | 0.3249 | 0.1159 | 0.1417 | 0.3003 | 0.0496 |
| dmiip2024_3 | 0.3420 | 0.1197 | 0.1472 | 0.3169 | 0.0568 |
| dmiip2024_4 | 0.3237 | 0.1142 | 0.1397 | 0.3017 | 0.0441 |
| AQAMS | 0.2091 | 0.1769 | 0.1541 | 0.0750 | 0.0006 |
| deepseek32b-me | 0.3039 | 0.0700 | 0.1013 | 0.2198 | 0.0121 |
| deepseek32b-full | 0.2371 | 0.0467 | 0.0665 | 0.1310 | 0.0045 |
| DB_vector_&_LLM | 0.1726 | 0.0371 | 0.0577 | 0.0860 | 0.0020 |
| UR-IW-1 | 0.3390 | 0.0941 | 0.1219 | 0.2982 | 0.0127 |
| UR-IW-2 | 0.4131 | 0.0845 | 0.1309 | 0.3405 | 0.0187 |
| UR-IW-3 | 0.3980 | 0.0913 | 0.1303 | 0.3558 | 0.0211 |
| UR-IW-4 | 0.4440 | 0.1013 | 0.1473 | 0.3616 | 0.0435 |
| UR-IW-5 | 0.4034 | 0.0983 | 0.1389 | 0.3413 | 0.0542 |
| The Relevants | - | - | - | - | - |
| Data Wizards NN | 0.0158 | 0.0016 | 0.0029 | 0.0147 | 0.0000 |
| API and weirdnesss | 0.1203 | 0.0329 | 0.0457 | 0.0544 | 0.0005 |
| DS@GT BioASQ 1b-1 | 0.2921 | 0.0665 | 0.0900 | 0.1776 | 0.0040 |
| Neural Re-Ranking | 0.0585 | 0.0099 | 0.0157 | 0.0442 | 0.0001 |
| Approach 1 | 0.0546 | 0.0098 | 0.0158 | 0.0379 | 0.0001 |
| DS@GT BioASQ 1b-2 | 0.2921 | 0.0665 | 0.0900 | 0.1776 | 0.0040 |
| GoldenRetrievers-2 | - | - | - | - | - |
| dl approach | 0.0202 | 0.0244 | 0.0202 | 0.0097 | 0.0000 |
| GPT4O | 0.2423 | 0.0545 | 0.0761 | 0.2108 | 0.0110 |
| DS@GTBioASQT13b5 | 0.2921 | 0.0665 | 0.0900 | 0.1776 | 0.0040 |
| sp_lasigebiotm | 0.1561 | 0.0363 | 0.0531 | 0.0852 | 0.0007 |
| mistral | 0.3680 | 0.0732 | 0.1106 | 0.2413 | 0.0260 |
| lasigeBioTM | 0.1561 | 0.0363 | 0.0531 | 0.0852 | 0.0007 |
| llama | 0.3081 | 0.0529 | 0.0822 | 0.1686 | 0.0116 |
| lasigeBioTM-onto-bl | 0.1561 | 0.0363 | 0.0531 | 0.0852 | 0.0007 |
| lasigeBioTM-onto-sm | 0.1561 | 0.0363 | 0.0531 | 0.0852 | 0.0007 |
| 1.PhaseA_System | 0.0419 | 0.0111 | 0.0142 | 0.0223 | 0.0001 |
| deepseek-r1:32b | 0.1433 | 0.0366 | 0.0503 | 0.0770 | 0.0014 |
| deepseek-r1:14b | 0.1433 | 0.0366 | 0.0503 | 0.0770 | 0.0014 |
| deepseek-r1:8b | 0.1146 | 0.0202 | 0.0317 | 0.0481 | 0.0003 |
| DS@GTBioASQT13b4 | 0.2921 | 0.0665 | 0.0900 | 0.1776 | 0.0040 |
| DS@GTBioASQT13b3 | 0.2921 | 0.0665 | 0.0900 | 0.1776 | 0.0040 |
| dense | 0.3550 | 0.0854 | 0.1257 | 0.2465 | 0.0201 |
| Fleming-2 | 0.2216 | 0.0569 | 0.0775 | 0.1788 | 0.0070 |
Concepts
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| Data Wizards BM25 | - | - | - | - | - |
| NLP_BioBert | - | - | - | - | - |
| Data Wizards Re-Rank | - | - | - | - | - |
| First abrotsch | - | - | - | - | - |
| ms-marco-MiniLML6v2 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| BM25-Okapi | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| IR4 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| Reranking rankings | - | - | - | - | - |
| query-engineering | - | - | - | - | - |
| Re-Ranker with BM25 | - | - | - | - | - |
| RetrievalRivals | - | - | - | - | - |
| RetrievalRivals_BM25 | - | - | - | - | - |
| RetrieverRivals | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| TEsting123 | - | - | - | - | - |
| Traditional IR Methd | - | - | - | - | - |
| traditional IR | - | - | - | - | - |
| Neural IR approach | - | - | - | - | - |
| Machinen Results | - | - | - | - | - |
| Tuwien Group 6 AIR | - | - | - | - | - |
| reranking after IR | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| Okapi BM25 IR | - | - | - | - | - |
| NNR Model | - | - | - | - | - |
| GoldenRetrievers | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| AQAMS | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| DB_vector_&_LLM | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| The Relevants | - | - | - | - | - |
| Data Wizards NN | - | - | - | - | - |
| API and weirdnesss | - | - | - | - | - |
| DS@GT BioASQ 1b-1 | - | - | - | - | - |
| Neural Re-Ranking | - | - | - | - | - |
| Approach 1 | - | - | - | - | - |
| DS@GT BioASQ 1b-2 | - | - | - | - | - |
| GoldenRetrievers-2 | - | - | - | - | - |
| dl approach | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| DS@GTBioASQT13b5 | - | - | - | - | - |
| sp_lasigebiotm | - | - | - | - | - |
| mistral | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| llama | - | - | - | - | - |
| lasigeBioTM-onto-bl | - | - | - | - | - |
| lasigeBioTM-onto-sm | - | - | - | - | - |
| 1.PhaseA_System | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| DS@GTBioASQT13b4 | - | - | - | - | - |
| DS@GTBioASQT13b3 | - | - | - | - | - |
| dense | - | - | - | - | - |
| Fleming-2 | - | - | - | - | - |
RDF Triples
| System | Mean precision | Recall | F-Measure | MAP | GMAP |
|---|---|---|---|---|---|
| Fleming-1 | - | - | - | - | - |
| Data Wizards BM25 | - | - | - | - | - |
| NLP_BioBert | - | - | - | - | - |
| Data Wizards Re-Rank | - | - | - | - | - |
| First abrotsch | - | - | - | - | - |
| ms-marco-MiniLML6v2 | - | - | - | - | - |
| IR5 | - | - | - | - | - |
| BM25-Okapi | - | - | - | - | - |
| IR1 | - | - | - | - | - |
| IR4 | - | - | - | - | - |
| UniTor_0 | - | - | - | - | - |
| UniTor_1 | - | - | - | - | - |
| Reranking rankings | - | - | - | - | - |
| query-engineering | - | - | - | - | - |
| Re-Ranker with BM25 | - | - | - | - | - |
| RetrievalRivals | - | - | - | - | - |
| RetrievalRivals_BM25 | - | - | - | - | - |
| RetrieverRivals | - | - | - | - | - |
| UniTor_2 | - | - | - | - | - |
| UniTor_3 | - | - | - | - | - |
| TEsting123 | - | - | - | - | - |
| Traditional IR Methd | - | - | - | - | - |
| traditional IR | - | - | - | - | - |
| Neural IR approach | - | - | - | - | - |
| Machinen Results | - | - | - | - | - |
| Tuwien Group 6 AIR | - | - | - | - | - |
| reranking after IR | - | - | - | - | - |
| bioinfo-0 | - | - | - | - | - |
| bioinfo-1 | - | - | - | - | - |
| bioinfo-2 | - | - | - | - | - |
| bioinfo-3 | - | - | - | - | - |
| bioinfo-4 | - | - | - | - | - |
| Okapi BM25 IR | - | - | - | - | - |
| NNR Model | - | - | - | - | - |
| GoldenRetrievers | - | - | - | - | - |
| Using KG for list q | - | - | - | - | - |
| Baseline top 20 | - | - | - | - | - |
| Baseline top 10 | - | - | - | - | - |
| Using LLM alone | - | - | - | - | - |
| Main pipeline | - | - | - | - | - |
| dmiip2024 | - | - | - | - | - |
| dmiip2024_1 | - | - | - | - | - |
| dmiip2024_2 | - | - | - | - | - |
| dmiip2024_3 | - | - | - | - | - |
| dmiip2024_4 | - | - | - | - | - |
| AQAMS | - | - | - | - | - |
| deepseek32b-me | - | - | - | - | - |
| deepseek32b-full | - | - | - | - | - |
| DB_vector_&_LLM | - | - | - | - | - |
| UR-IW-1 | - | - | - | - | - |
| UR-IW-2 | - | - | - | - | - |
| UR-IW-3 | - | - | - | - | - |
| UR-IW-4 | - | - | - | - | - |
| UR-IW-5 | - | - | - | - | - |
| The Relevants | - | - | - | - | - |
| Data Wizards NN | - | - | - | - | - |
| API and weirdnesss | - | - | - | - | - |
| DS@GT BioASQ 1b-1 | - | - | - | - | - |
| Neural Re-Ranking | - | - | - | - | - |
| Approach 1 | - | - | - | - | - |
| DS@GT BioASQ 1b-2 | - | - | - | - | - |
| GoldenRetrievers-2 | - | - | - | - | - |
| dl approach | - | - | - | - | - |
| GPT4O | - | - | - | - | - |
| DS@GTBioASQT13b5 | - | - | - | - | - |
| sp_lasigebiotm | - | - | - | - | - |
| mistral | - | - | - | - | - |
| lasigeBioTM | - | - | - | - | - |
| llama | - | - | - | - | - |
| lasigeBioTM-onto-bl | - | - | - | - | - |
| lasigeBioTM-onto-sm | - | - | - | - | - |
| 1.PhaseA_System | - | - | - | - | - |
| deepseek-r1:32b | - | - | - | - | - |
| deepseek-r1:14b | - | - | - | - | - |
| deepseek-r1:8b | - | - | - | - | - |
| DS@GTBioASQT13b4 | - | - | - | - | - |
| DS@GTBioASQT13b3 | - | - | - | - | - |
| dense | - | - | - | - | - |
| Fleming-2 | - | - | - | - | - |