BioASQ Participants Area
Test Results for Task 7a
The evaluation measures indicating the performance of the systems that submitted results are presented below. The evaluation is incremental; as new MeSH become available the tables are updated.
Annotated articles:16780/17711
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
BioASQ_Baseline |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
BioASQ_Baseline |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
0.0000 |
+ Dry Run 2
Annotated articles:5408/5577
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
NLM System 4 |
0.6561 |
0.6900 |
0.6371 |
0.6449 |
0.6237 |
0.5146 |
0.5090 |
0.6838 |
0.6306 |
0.4920 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
NLM System 4 |
0.5252 |
0.8082 |
0.7248 |
0.7475 |
0.5668 |
0.5121 |
+ Test batch 1, week 1
Annotated articles:7194/7358
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6268 |
0.6730 |
0.6050 |
0.6170 |
0.6284 |
0.5552 |
0.5320 |
0.6610 |
0.5960 |
0.4640 |
Default MTI |
0.6397 |
0.6419 |
0.6586 |
0.6322 |
0.5937 |
0.5953 |
0.5524 |
0.6328 |
0.6468 |
0.4797 |
DeepMeSH1 |
0.6628 |
0.7286 |
0.6226 |
0.6564 |
0.6886 |
0.5137 |
0.5139 |
0.7319 |
0.6056 |
0.5053 |
DeepMeSH2 |
0.6382 |
0.7160 |
0.5883 |
0.6305 |
0.6705 |
0.4526 |
0.4599 |
0.7213 |
0.5724 |
0.4767 |
DeepMeSH3 |
0.6607 |
0.7260 |
0.6209 |
0.6544 |
0.6870 |
0.5108 |
0.5116 |
0.7293 |
0.6039 |
0.5034 |
MeSHProbeNet-P |
0.6849 |
0.7141 |
0.6714 |
0.6753 |
0.6728 |
0.5670 |
0.5556 |
0.7090 |
0.6624 |
0.5264 |
NLM System 4 |
0.6515 |
0.6840 |
0.6343 |
0.6404 |
0.6125 |
0.5045 |
0.4967 |
0.6768 |
0.6280 |
0.4873 |
MeSHProbeNet-P0 |
0.6848 |
0.7024 |
0.6817 |
0.6753 |
0.6604 |
0.5781 |
0.5609 |
0.6970 |
0.6730 |
0.5262 |
MeSHProbeNet-P1 |
0.6839 |
0.6903 |
0.6917 |
0.6746 |
0.6486 |
0.5893 |
0.5657 |
0.6846 |
0.6832 |
0.5252 |
MeSHProbeNet-P2 |
0.6849 |
0.7140 |
0.6714 |
0.6753 |
0.6728 |
0.5670 |
0.5556 |
0.7089 |
0.6624 |
0.5264 |
MeSHProbeNet-P3 |
0.6843 |
0.7128 |
0.6711 |
0.6746 |
0.6700 |
0.5651 |
0.5529 |
0.7077 |
0.6624 |
0.5256 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5035 |
0.7889 |
0.7105 |
0.7277 |
0.5508 |
0.4909 |
Default MTI |
0.5207 |
0.7601 |
0.7567 |
0.7408 |
0.5358 |
0.5333 |
DeepMeSH1 |
0.5298 |
0.8304 |
0.7120 |
0.7507 |
0.5905 |
0.5015 |
DeepMeSH2 |
0.5068 |
0.8239 |
0.6815 |
0.7291 |
0.5772 |
0.4722 |
DeepMeSH3 |
0.5288 |
0.8293 |
0.7099 |
0.7491 |
0.5895 |
0.5006 |
MeSHProbeNet-P |
0.5510 |
0.8186 |
0.7572 |
0.7704 |
0.5874 |
0.5429 |
NLM System 4 |
0.5228 |
0.8013 |
0.7231 |
0.7420 |
0.5624 |
0.5130 |
MeSHProbeNet-P0 |
0.5523 |
0.8099 |
0.7667 |
0.7715 |
0.5812 |
0.5508 |
MeSHProbeNet-P1 |
0.5535 |
0.8011 |
0.7761 |
0.7725 |
0.5746 |
0.5587 |
MeSHProbeNet-P2 |
0.5509 |
0.8186 |
0.7572 |
0.7703 |
0.5874 |
0.5429 |
MeSHProbeNet-P3 |
0.5509 |
0.8178 |
0.7576 |
0.7701 |
0.5870 |
0.5432 |
+ Test batch 1, week 2
Annotated articles:7021/7166
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6272 |
0.6688 |
0.6083 |
0.6160 |
0.6383 |
0.5640 |
0.5421 |
0.6613 |
0.5964 |
0.4644 |
Default MTI |
0.6405 |
0.6391 |
0.6621 |
0.6313 |
0.6068 |
0.6072 |
0.5650 |
0.6343 |
0.6469 |
0.4792 |
DeepMeSH1 |
0.6330 |
0.7105 |
0.5857 |
0.6246 |
0.6636 |
0.4596 |
0.4664 |
0.7177 |
0.5662 |
0.4714 |
DeepMeSH2 |
0.6382 |
0.7252 |
0.5866 |
0.6313 |
0.6801 |
0.4886 |
0.4915 |
0.7324 |
0.5655 |
0.4786 |
DeepMeSH3 |
0.6134 |
0.7367 |
0.5462 |
0.6084 |
0.6781 |
0.4732 |
0.4782 |
0.7428 |
0.5223 |
0.4547 |
MetaEmb |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
0.0004 |
0.0000 |
0.0001 |
0.0001 |
0.0001 |
MeSHProbeNet-P |
0.6853 |
0.7109 |
0.6751 |
0.6748 |
0.6828 |
0.5806 |
0.5699 |
0.7090 |
0.6632 |
0.5266 |
NLM System 4 |
0.6499 |
0.6898 |
0.6254 |
0.6373 |
0.6281 |
0.5000 |
0.4978 |
0.6855 |
0.6177 |
0.4848 |
MeSHProbeNet-P0 |
0.6868 |
0.7108 |
0.6768 |
0.6758 |
0.6855 |
0.5754 |
0.5668 |
0.7090 |
0.6659 |
0.5273 |
MeSHProbeNet-P1 |
0.6819 |
0.7027 |
0.6731 |
0.6703 |
0.6801 |
0.5588 |
0.5533 |
0.7005 |
0.6642 |
0.5207 |
MeSHProbeNet-P2 |
0.6815 |
0.6837 |
0.6861 |
0.6705 |
0.6557 |
0.5903 |
0.5737 |
0.6891 |
0.6741 |
0.5205 |
MeSHProbeNet-P3 |
0.6799 |
0.6888 |
0.6772 |
0.6687 |
0.6701 |
0.5677 |
0.5588 |
0.6939 |
0.6665 |
0.5185 |
baseline |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
0.0004 |
0.0000 |
0.0001 |
0.0001 |
0.0001 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5015 |
0.7858 |
0.7097 |
0.7252 |
0.5460 |
0.4913 |
Default MTI |
0.5169 |
0.7579 |
0.7567 |
0.7387 |
0.5315 |
0.5311 |
DeepMeSH1 |
0.5008 |
0.8195 |
0.6812 |
0.7262 |
0.5695 |
0.4687 |
DeepMeSH2 |
0.5079 |
0.8295 |
0.6937 |
0.7386 |
0.5747 |
0.4763 |
DeepMeSH3 |
0.4868 |
0.8358 |
0.6636 |
0.7217 |
0.5708 |
0.4457 |
MetaEmb |
0.1224 |
0.1192 |
0.1926 |
0.1429 |
0.1059 |
0.1529 |
MeSHProbeNet-P |
0.5501 |
0.8177 |
0.7583 |
0.7701 |
0.5849 |
0.5438 |
NLM System 4 |
0.5193 |
0.8089 |
0.7146 |
0.7403 |
0.5648 |
0.5045 |
MeSHProbeNet-P0 |
0.5509 |
0.8194 |
0.7596 |
0.7717 |
0.5848 |
0.5449 |
MeSHProbeNet-P1 |
0.5474 |
0.8167 |
0.7577 |
0.7694 |
0.5811 |
0.5416 |
MeSHProbeNet-P2 |
0.5481 |
0.8005 |
0.7701 |
0.7703 |
0.5703 |
0.5502 |
MeSHProbeNet-P3 |
0.5477 |
0.8066 |
0.7637 |
0.7698 |
0.5741 |
0.5455 |
baseline |
0.1211 |
0.1223 |
0.1828 |
0.1421 |
0.1052 |
0.1506 |
+ Test batch 1, week 3
Annotated articles:10831/11019
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6191 |
0.6682 |
0.5952 |
0.6083 |
0.6416 |
0.5624 |
0.5423 |
0.6576 |
0.5849 |
0.4556 |
Default MTI |
0.6329 |
0.6352 |
0.6521 |
0.6245 |
0.6072 |
0.6058 |
0.5634 |
0.6279 |
0.6379 |
0.4716 |
DeepMeSH1 |
0.6649 |
0.7277 |
0.6300 |
0.6591 |
0.7049 |
0.5469 |
0.5527 |
0.7309 |
0.6098 |
0.5084 |
DeepMeSH2 |
0.6647 |
0.7250 |
0.6316 |
0.6587 |
0.7032 |
0.5470 |
0.5533 |
0.7280 |
0.6115 |
0.5079 |
DeepMeSH3 |
0.6494 |
0.7181 |
0.6093 |
0.6430 |
0.6937 |
0.4991 |
0.5075 |
0.7224 |
0.5898 |
0.4899 |
DeepMeSH4 |
0.6450 |
0.7170 |
0.6021 |
0.6382 |
0.6912 |
0.4910 |
0.5006 |
0.7215 |
0.5832 |
0.4849 |
DeepMeSH5 |
0.6453 |
0.7133 |
0.6044 |
0.6381 |
0.6884 |
0.4890 |
0.4981 |
0.7182 |
0.5858 |
0.4848 |
MeSHProbeNet-P |
0.6738 |
0.7033 |
0.6634 |
0.6639 |
0.6780 |
0.5674 |
0.5590 |
0.6977 |
0.6515 |
0.5134 |
NLM System 4 |
0.6384 |
0.6839 |
0.6145 |
0.6277 |
0.6254 |
0.4911 |
0.4912 |
0.6762 |
0.6045 |
0.4737 |
MeSHProbeNet-P0 |
0.6747 |
0.6983 |
0.6694 |
0.6648 |
0.6735 |
0.5673 |
0.5581 |
0.6926 |
0.6577 |
0.5142 |
MeSHProbeNet-P1 |
0.6737 |
0.6996 |
0.6661 |
0.6636 |
0.6760 |
0.5589 |
0.5527 |
0.6938 |
0.6547 |
0.5129 |
MeSHProbeNet-P2 |
0.6709 |
0.6799 |
0.6727 |
0.6610 |
0.6631 |
0.5699 |
0.5593 |
0.6817 |
0.6605 |
0.5092 |
MeSHProbeNet-P3 |
0.6693 |
0.6747 |
0.6750 |
0.6595 |
0.6489 |
0.5849 |
0.5663 |
0.6764 |
0.6622 |
0.5076 |
UNCC_IR_01 |
0.5101 |
0.5572 |
0.4855 |
0.4939 |
0.5511 |
0.3250 |
0.3388 |
0.5468 |
0.4781 |
0.3434 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.4924 |
0.7864 |
0.7044 |
0.7229 |
0.5407 |
0.4793 |
Default MTI |
0.5077 |
0.7564 |
0.7529 |
0.7366 |
0.5237 |
0.5208 |
DeepMeSH1 |
0.5252 |
0.8292 |
0.7199 |
0.7542 |
0.5831 |
0.5000 |
DeepMeSH2 |
0.5253 |
0.8272 |
0.7216 |
0.7541 |
0.5820 |
0.5011 |
DeepMeSH3 |
0.5122 |
0.8252 |
0.7035 |
0.7427 |
0.5759 |
0.4831 |
DeepMeSH4 |
0.5083 |
0.8236 |
0.6975 |
0.7384 |
0.5739 |
0.4778 |
DeepMeSH5 |
0.5084 |
0.8211 |
0.6994 |
0.7384 |
0.5718 |
0.4793 |
MeSHProbeNet-P |
0.5361 |
0.8111 |
0.7529 |
0.7630 |
0.5725 |
0.5299 |
NLM System 4 |
0.5062 |
0.8038 |
0.7081 |
0.7335 |
0.5543 |
0.4909 |
MeSHProbeNet-P0 |
0.5379 |
0.8083 |
0.7580 |
0.7646 |
0.5713 |
0.5342 |
MeSHProbeNet-P1 |
0.5375 |
0.8101 |
0.7555 |
0.7641 |
0.5729 |
0.5319 |
MeSHProbeNet-P2 |
0.5375 |
0.7965 |
0.7647 |
0.7649 |
0.5632 |
0.5375 |
MeSHProbeNet-P3 |
0.5343 |
0.7908 |
0.7663 |
0.7629 |
0.5573 |
0.5373 |
UNCC_IR_01 |
0.4107 |
0.7163 |
0.5929 |
0.6212 |
0.4736 |
0.3923 |
+ Test batch 1, week 4
Annotated articles:5482/5566
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6202 |
0.6693 |
0.6017 |
0.6137 |
0.6277 |
0.5596 |
0.5364 |
0.6590 |
0.5856 |
0.4623 |
Default MTI |
0.6387 |
0.6420 |
0.6622 |
0.6341 |
0.5965 |
0.6000 |
0.5598 |
0.6357 |
0.6419 |
0.4817 |
DeepMeSH1 |
0.6702 |
0.7368 |
0.6357 |
0.6670 |
0.6986 |
0.5427 |
0.5460 |
0.7403 |
0.6122 |
0.5164 |
DeepMeSH2 |
0.6687 |
0.7350 |
0.6346 |
0.6654 |
0.6968 |
0.5400 |
0.5429 |
0.7382 |
0.6110 |
0.5147 |
DeepMeSH3 |
0.6518 |
0.7250 |
0.6088 |
0.6463 |
0.6873 |
0.4903 |
0.4969 |
0.7300 |
0.5887 |
0.4936 |
DeepMeSH4 |
0.6522 |
0.7246 |
0.6096 |
0.6467 |
0.6863 |
0.4931 |
0.4997 |
0.7292 |
0.5899 |
0.4938 |
MeSHProbeNet-P |
0.6816 |
0.7164 |
0.6689 |
0.6747 |
0.6800 |
0.5662 |
0.5583 |
0.7127 |
0.6532 |
0.5253 |
NLM System 4 |
0.6476 |
0.6930 |
0.6238 |
0.6380 |
0.6246 |
0.4936 |
0.4914 |
0.6862 |
0.6130 |
0.4843 |
MeSHProbeNet-P0 |
0.6831 |
0.7074 |
0.6790 |
0.6761 |
0.6715 |
0.5724 |
0.5614 |
0.7035 |
0.6639 |
0.5265 |
MeSHProbeNet-P1 |
0.6829 |
0.7100 |
0.6750 |
0.6754 |
0.6754 |
0.5629 |
0.5551 |
0.7064 |
0.6608 |
0.5257 |
MeSHProbeNet-P2 |
0.6819 |
0.7045 |
0.6796 |
0.6751 |
0.6689 |
0.5783 |
0.5652 |
0.7008 |
0.6640 |
0.5255 |
MeSHProbeNet-P3 |
0.6693 |
0.7128 |
0.6494 |
0.6620 |
0.6773 |
0.5347 |
0.5288 |
0.7097 |
0.6333 |
0.5113 |
UNCC_IR_01 |
0.5249 |
0.5744 |
0.5008 |
0.5120 |
0.5433 |
0.3289 |
0.3377 |
0.5640 |
0.4909 |
0.3595 |
MeSHedPotato |
0.2839 |
0.7562 |
0.1770 |
0.2591 |
0.6752 |
0.0523 |
0.0679 |
0.7873 |
0.1732 |
0.1668 |
txt_sys0 |
0.4508 |
0.4401 |
0.4851 |
0.4420 |
0.2706 |
0.3207 |
0.2847 |
0.4280 |
0.4760 |
0.2964 |
txt_sys1 |
0.4672 |
0.5123 |
0.4444 |
0.4550 |
0.3536 |
0.2818 |
0.2759 |
0.5027 |
0.4363 |
0.3085 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5038 |
0.7870 |
0.7097 |
0.7273 |
0.5489 |
0.4911 |
Default MTI |
0.5212 |
0.7613 |
0.7582 |
0.7424 |
0.5353 |
0.5343 |
DeepMeSH1 |
0.5370 |
0.8359 |
0.7214 |
0.7584 |
0.5968 |
0.5093 |
DeepMeSH2 |
0.5356 |
0.8349 |
0.7208 |
0.7576 |
0.5957 |
0.5078 |
DeepMeSH3 |
0.5207 |
0.8298 |
0.6996 |
0.7426 |
0.5882 |
0.4880 |
DeepMeSH4 |
0.5209 |
0.8310 |
0.6993 |
0.7431 |
0.5879 |
0.4882 |
MeSHProbeNet-P |
0.5514 |
0.8209 |
0.7536 |
0.7694 |
0.5902 |
0.5408 |
NLM System 4 |
0.5192 |
0.8082 |
0.7127 |
0.7394 |
0.5654 |
0.5029 |
MeSHProbeNet-P0 |
0.5541 |
0.8139 |
0.7619 |
0.7708 |
0.5862 |
0.5488 |
MeSHProbeNet-P1 |
0.5532 |
0.8167 |
0.7579 |
0.7699 |
0.5881 |
0.5450 |
MeSHProbeNet-P2 |
0.5526 |
0.8117 |
0.7633 |
0.7706 |
0.5836 |
0.5485 |
MeSHProbeNet-P3 |
0.5420 |
0.8216 |
0.7366 |
0.7594 |
0.5891 |
0.5255 |
UNCC_IR_01 |
0.4281 |
0.7295 |
0.6014 |
0.6340 |
0.4919 |
0.4063 |
MeSHedPotato |
0.1959 |
0.8160 |
0.2045 |
0.2963 |
0.4259 |
0.1386 |
txt_sys0 |
0.3866 |
0.5834 |
0.6241 |
0.5799 |
0.3904 |
0.4142 |
txt_sys1 |
0.3883 |
0.6539 |
0.5656 |
0.5816 |
0.4379 |
0.3757 |
+ Test batch 1, week 5
Annotated articles:6353/6729
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6288 |
0.6719 |
0.6132 |
0.6207 |
0.6319 |
0.5629 |
0.5393 |
0.6612 |
0.5995 |
0.4692 |
Default MTI |
0.6424 |
0.6431 |
0.6662 |
0.6358 |
0.6003 |
0.6035 |
0.5621 |
0.6361 |
0.6489 |
0.4842 |
DeepMeSH1 |
0.6857 |
0.7491 |
0.6502 |
0.6805 |
0.7249 |
0.5518 |
0.5579 |
0.7537 |
0.6289 |
0.5329 |
DeepMeSH2 |
0.6771 |
0.7408 |
0.6409 |
0.6718 |
0.7138 |
0.5290 |
0.5353 |
0.7455 |
0.6202 |
0.5227 |
DeepMeSH3 |
0.6729 |
0.7431 |
0.6312 |
0.6671 |
0.7206 |
0.5140 |
0.5239 |
0.7488 |
0.6110 |
0.5176 |
DeepMeSH4 |
0.6728 |
0.7463 |
0.6288 |
0.6669 |
0.7207 |
0.5098 |
0.5202 |
0.7516 |
0.6090 |
0.5175 |
DeepMeSH5 |
0.6993 |
0.7628 |
0.6638 |
0.6954 |
0.7419 |
0.5677 |
0.5754 |
0.7674 |
0.6423 |
0.5546 |
MeSHProbeNet-P |
0.6854 |
0.7146 |
0.6750 |
0.6763 |
0.6787 |
0.5709 |
0.5616 |
0.7109 |
0.6617 |
0.5281 |
NLM System 4 |
0.6522 |
0.6926 |
0.6315 |
0.6420 |
0.6260 |
0.4985 |
0.4957 |
0.6860 |
0.6216 |
0.4892 |
MeSHProbeNet-P0 |
0.6854 |
0.7025 |
0.6854 |
0.6763 |
0.6670 |
0.5830 |
0.5685 |
0.6991 |
0.6723 |
0.5278 |
MeSHProbeNet-P1 |
0.6847 |
0.6905 |
0.6958 |
0.6759 |
0.6542 |
0.5941 |
0.5735 |
0.6869 |
0.6825 |
0.5268 |
MeSHProbeNet-P2 |
0.6862 |
0.6997 |
0.6894 |
0.6771 |
0.6667 |
0.5813 |
0.5670 |
0.6960 |
0.6767 |
0.5284 |
MeSHProbeNet-P3 |
0.6736 |
0.6999 |
0.6659 |
0.6644 |
0.6649 |
0.5476 |
0.5370 |
0.6963 |
0.6523 |
0.5143 |
UNCC_IR_01 |
0.5203 |
0.5323 |
0.5243 |
0.5051 |
0.5083 |
0.3476 |
0.3492 |
0.5244 |
0.5163 |
0.3532 |
MeSHedPotato1 |
0.3344 |
0.7613 |
0.2253 |
0.3040 |
0.6267 |
0.1267 |
0.1517 |
0.7445 |
0.2156 |
0.2021 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5075 |
0.7883 |
0.7163 |
0.7312 |
0.5509 |
0.4976 |
Default MTI |
0.5210 |
0.7621 |
0.7621 |
0.7446 |
0.5349 |
0.5359 |
DeepMeSH1 |
0.5472 |
0.8430 |
0.7344 |
0.7697 |
0.6043 |
0.5218 |
DeepMeSH2 |
0.5401 |
0.8369 |
0.7279 |
0.7633 |
0.5968 |
0.5148 |
DeepMeSH3 |
0.5359 |
0.8398 |
0.7186 |
0.7589 |
0.5973 |
0.5073 |
DeepMeSH4 |
0.5353 |
0.8424 |
0.7165 |
0.7588 |
0.5991 |
0.5051 |
DeepMeSH5 |
0.5682 |
0.8521 |
0.7448 |
0.7805 |
0.6243 |
0.5424 |
MeSHProbeNet-P |
0.5508 |
0.8216 |
0.7593 |
0.7726 |
0.5875 |
0.5441 |
NLM System 4 |
0.5199 |
0.8097 |
0.7210 |
0.7449 |
0.5633 |
0.5077 |
MeSHProbeNet-P0 |
0.5520 |
0.8122 |
0.7690 |
0.7737 |
0.5807 |
0.5519 |
MeSHProbeNet-P1 |
0.5523 |
0.8027 |
0.7784 |
0.7742 |
0.5734 |
0.5587 |
MeSHProbeNet-P2 |
0.5527 |
0.8103 |
0.7724 |
0.7746 |
0.5794 |
0.5544 |
MeSHProbeNet-P3 |
0.5435 |
0.8142 |
0.7539 |
0.7659 |
0.5791 |
0.5374 |
UNCC_IR_01 |
0.4244 |
0.6958 |
0.6323 |
0.6371 |
0.4632 |
0.4224 |
MeSHedPotato1 |
0.2323 |
0.8265 |
0.2771 |
0.3638 |
0.4243 |
0.1802 |
+ Test batch 2, week 1
Annotated articles:6098/6380
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6265 |
0.6690 |
0.6039 |
0.6161 |
0.6296 |
0.5627 |
0.5394 |
0.6598 |
0.5965 |
0.4635 |
Default MTI |
0.6403 |
0.6394 |
0.6590 |
0.6322 |
0.5985 |
0.6051 |
0.5614 |
0.6332 |
0.6476 |
0.4793 |
iria-mix |
0.5202 |
0.4616 |
0.5971 |
0.5091 |
0.4434 |
0.4807 |
0.4318 |
0.4658 |
0.5890 |
0.3545 |
DeepMeSH1 |
0.6917 |
0.7214 |
0.6745 |
0.6841 |
0.6845 |
0.5961 |
0.5863 |
0.7232 |
0.6629 |
0.5364 |
DeepMeSH2 |
0.6900 |
0.7215 |
0.6711 |
0.6824 |
0.6838 |
0.5907 |
0.5828 |
0.7237 |
0.6593 |
0.5346 |
DeepMeSH3 |
0.6895 |
0.7214 |
0.6701 |
0.6820 |
0.6834 |
0.5888 |
0.5810 |
0.7239 |
0.6582 |
0.5341 |
DeepMeSH4 |
0.6786 |
0.7078 |
0.6585 |
0.6690 |
0.6681 |
0.5637 |
0.5561 |
0.7111 |
0.6490 |
0.5188 |
DeepMeSH5 |
0.7128 |
0.7391 |
0.6959 |
0.7047 |
0.7070 |
0.6228 |
0.6141 |
0.7424 |
0.6855 |
0.5668 |
MeSHProbeNet-P |
0.6848 |
0.6897 |
0.6899 |
0.6742 |
0.6563 |
0.6020 |
0.5780 |
0.6865 |
0.6832 |
0.5243 |
NLM System 4 |
0.6509 |
0.6901 |
0.6254 |
0.6386 |
0.6198 |
0.5029 |
0.4978 |
0.6840 |
0.6210 |
0.4855 |
MeSHProbeNet-P0 |
0.6855 |
0.7017 |
0.6796 |
0.6746 |
0.6679 |
0.5902 |
0.5719 |
0.6986 |
0.6729 |
0.5251 |
MeSHProbeNet-P1 |
0.6871 |
0.7104 |
0.6746 |
0.6759 |
0.6776 |
0.5791 |
0.5657 |
0.7072 |
0.6681 |
0.5267 |
MeSHProbeNet-P2 |
0.6824 |
0.7002 |
0.6747 |
0.6711 |
0.6655 |
0.5827 |
0.5657 |
0.6974 |
0.6680 |
0.5211 |
MeSHProbeNet-P3 |
0.6730 |
0.6977 |
0.6598 |
0.6614 |
0.6616 |
0.5549 |
0.5404 |
0.6943 |
0.6530 |
0.5106 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5021 |
0.7851 |
0.7076 |
0.7257 |
0.5460 |
0.4899 |
Default MTI |
0.5190 |
0.7578 |
0.7567 |
0.7407 |
0.5322 |
0.5322 |
iria-mix |
0.4406 |
0.6151 |
0.7302 |
0.6538 |
0.4182 |
0.4882 |
DeepMeSH1 |
0.5533 |
0.8192 |
0.7579 |
0.7738 |
0.5900 |
0.5412 |
DeepMeSH2 |
0.5525 |
0.8199 |
0.7552 |
0.7727 |
0.5912 |
0.5387 |
DeepMeSH3 |
0.5522 |
0.8203 |
0.7537 |
0.7721 |
0.5912 |
0.5380 |
DeepMeSH4 |
0.5424 |
0.8115 |
0.7458 |
0.7632 |
0.5805 |
0.5296 |
DeepMeSH5 |
0.5825 |
0.8303 |
0.7748 |
0.7890 |
0.6161 |
0.5714 |
MeSHProbeNet-P |
0.5511 |
0.8003 |
0.7742 |
0.7717 |
0.5730 |
0.5548 |
NLM System 4 |
0.5169 |
0.8045 |
0.7127 |
0.7377 |
0.5626 |
0.5017 |
MeSHProbeNet-P0 |
0.5496 |
0.8091 |
0.7642 |
0.7704 |
0.5791 |
0.5464 |
MeSHProbeNet-P1 |
0.5498 |
0.8162 |
0.7589 |
0.7708 |
0.5837 |
0.5428 |
MeSHProbeNet-P2 |
0.5464 |
0.8092 |
0.7602 |
0.7681 |
0.5776 |
0.5421 |
MeSHProbeNet-P3 |
0.5393 |
0.8094 |
0.7481 |
0.7613 |
0.5754 |
0.5316 |
+ Test batch 2, week 2
Annotated articles:6621/6785
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6355 |
0.6747 |
0.6213 |
0.6269 |
0.6375 |
0.5707 |
0.5450 |
0.6638 |
0.6094 |
0.4742 |
Default MTI |
0.6456 |
0.6441 |
0.6690 |
0.6390 |
0.6064 |
0.6117 |
0.5669 |
0.6375 |
0.6539 |
0.4856 |
iria-mix |
0.5365 |
0.4751 |
0.6227 |
0.5269 |
0.4713 |
0.5036 |
0.4524 |
0.4788 |
0.6099 |
0.3701 |
DeepMeSH1 |
0.6963 |
0.7228 |
0.6859 |
0.6904 |
0.6898 |
0.6025 |
0.5921 |
0.7241 |
0.6706 |
0.5427 |
DeepMeSH2 |
0.6961 |
0.7237 |
0.6839 |
0.6897 |
0.6906 |
0.5993 |
0.5901 |
0.7257 |
0.6688 |
0.5421 |
DeepMeSH3 |
0.6957 |
0.7242 |
0.6834 |
0.6897 |
0.6892 |
0.5982 |
0.5885 |
0.7260 |
0.6678 |
0.5420 |
DeepMeSH4 |
0.6850 |
0.7126 |
0.6711 |
0.6777 |
0.6755 |
0.5717 |
0.5635 |
0.7149 |
0.6575 |
0.5280 |
DeepMeSH5 |
0.7288 |
0.7534 |
0.7180 |
0.7232 |
0.7309 |
0.6444 |
0.6356 |
0.7560 |
0.7036 |
0.5917 |
MeSHProbeNet-P |
0.6916 |
0.7061 |
0.6931 |
0.6829 |
0.6703 |
0.5949 |
0.5759 |
0.7017 |
0.6818 |
0.5337 |
NLM System 4 |
0.6581 |
0.6909 |
0.6405 |
0.6473 |
0.6192 |
0.5133 |
0.5072 |
0.6847 |
0.6335 |
0.4944 |
MeSHProbeNet-P0 |
0.6922 |
0.7184 |
0.6829 |
0.6834 |
0.6826 |
0.5833 |
0.5706 |
0.7140 |
0.6716 |
0.5346 |
MeSHProbeNet-P1 |
0.6928 |
0.7249 |
0.6778 |
0.6838 |
0.6906 |
0.5729 |
0.5647 |
0.7206 |
0.6671 |
0.5352 |
MeSHProbeNet-P2 |
0.6933 |
0.7228 |
0.6801 |
0.6840 |
0.6907 |
0.5705 |
0.5629 |
0.7183 |
0.6700 |
0.5354 |
MeSHProbeNet-P3 |
0.6794 |
0.7036 |
0.6720 |
0.6701 |
0.6669 |
0.5583 |
0.5433 |
0.6986 |
0.6613 |
0.5193 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5078 |
0.7878 |
0.7230 |
0.7348 |
0.5472 |
0.5009 |
Default MTI |
0.5215 |
0.7606 |
0.7653 |
0.7461 |
0.5322 |
0.5379 |
iria-mix |
0.4520 |
0.6289 |
0.7495 |
0.6701 |
0.4271 |
0.5038 |
DeepMeSH1 |
0.5593 |
0.8191 |
0.7681 |
0.7791 |
0.5907 |
0.5523 |
DeepMeSH2 |
0.5581 |
0.8197 |
0.7666 |
0.7785 |
0.5908 |
0.5498 |
DeepMeSH3 |
0.5576 |
0.8201 |
0.7659 |
0.7783 |
0.5904 |
0.5493 |
DeepMeSH4 |
0.5474 |
0.8133 |
0.7573 |
0.7704 |
0.5806 |
0.5389 |
DeepMeSH5 |
0.6071 |
0.8388 |
0.7920 |
0.8024 |
0.6361 |
0.6001 |
MeSHProbeNet-P |
0.5555 |
0.8094 |
0.7760 |
0.7765 |
0.5796 |
0.5581 |
NLM System 4 |
0.5245 |
0.8032 |
0.7268 |
0.7458 |
0.5632 |
0.5148 |
MeSHProbeNet-P0 |
0.5546 |
0.8190 |
0.7662 |
0.7756 |
0.5868 |
0.5503 |
MeSHProbeNet-P1 |
0.5552 |
0.8247 |
0.7609 |
0.7753 |
0.5911 |
0.5473 |
MeSHProbeNet-P2 |
0.5556 |
0.8233 |
0.7623 |
0.7754 |
0.5904 |
0.5488 |
MeSHProbeNet-P3 |
0.5452 |
0.8093 |
0.7592 |
0.7667 |
0.5768 |
0.5422 |
+ Test batch 2, week 3
Annotated articles:5927/6207
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6244 |
0.6874 |
0.5905 |
0.6134 |
0.6358 |
0.5381 |
0.5236 |
0.6754 |
0.5805 |
0.4627 |
Default MTI |
0.6378 |
0.6453 |
0.6491 |
0.6284 |
0.5969 |
0.5898 |
0.5526 |
0.6400 |
0.6357 |
0.4761 |
iria-mix |
0.5324 |
0.4680 |
0.6214 |
0.5214 |
0.4553 |
0.4913 |
0.4419 |
0.4732 |
0.6087 |
0.3656 |
DeepMeSH1 |
0.6909 |
0.7188 |
0.6755 |
0.6828 |
0.6812 |
0.5895 |
0.5821 |
0.7230 |
0.6614 |
0.5354 |
DeepMeSH2 |
0.6895 |
0.7180 |
0.6736 |
0.6814 |
0.6806 |
0.5872 |
0.5801 |
0.7227 |
0.6593 |
0.5337 |
DeepMeSH3 |
0.6898 |
0.7194 |
0.6730 |
0.6817 |
0.6804 |
0.5862 |
0.5797 |
0.7240 |
0.6587 |
0.5343 |
DeepMeSH4 |
0.6791 |
0.7058 |
0.6636 |
0.6704 |
0.6619 |
0.5634 |
0.5551 |
0.7107 |
0.6503 |
0.5208 |
DeepMeSH5 |
0.7186 |
0.7461 |
0.7036 |
0.7120 |
0.7212 |
0.6264 |
0.6214 |
0.7504 |
0.6894 |
0.5781 |
MeSHProbeNet-P |
0.6865 |
0.7180 |
0.6684 |
0.6761 |
0.6785 |
0.5723 |
0.5630 |
0.7170 |
0.6585 |
0.5282 |
NLM System 4 |
0.6541 |
0.6871 |
0.6341 |
0.6422 |
0.6142 |
0.5105 |
0.5048 |
0.6819 |
0.6286 |
0.4895 |
MeSHProbeNet-P0 |
0.6871 |
0.7069 |
0.6795 |
0.6769 |
0.6668 |
0.5848 |
0.5696 |
0.7056 |
0.6696 |
0.5287 |
MeSHProbeNet-P1 |
0.6869 |
0.6952 |
0.6901 |
0.6769 |
0.6546 |
0.5969 |
0.5756 |
0.6937 |
0.6803 |
0.5284 |
MeSHProbeNet-P2 |
0.6880 |
0.7137 |
0.6746 |
0.6776 |
0.6746 |
0.5744 |
0.5639 |
0.7121 |
0.6655 |
0.5294 |
MeSHProbeNet-P3 |
0.6880 |
0.7108 |
0.6765 |
0.6771 |
0.6741 |
0.5703 |
0.5609 |
0.7090 |
0.6683 |
0.5286 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5019 |
0.8047 |
0.6925 |
0.7232 |
0.5603 |
0.4809 |
Default MTI |
0.5175 |
0.7681 |
0.7473 |
0.7394 |
0.5380 |
0.5252 |
iria-mix |
0.4535 |
0.6292 |
0.7496 |
0.6701 |
0.4290 |
0.5048 |
DeepMeSH1 |
0.5537 |
0.8223 |
0.7605 |
0.7761 |
0.5896 |
0.5422 |
DeepMeSH2 |
0.5527 |
0.8216 |
0.7590 |
0.7750 |
0.5893 |
0.5406 |
DeepMeSH3 |
0.5522 |
0.8224 |
0.7576 |
0.7746 |
0.5889 |
0.5400 |
DeepMeSH4 |
0.5451 |
0.8137 |
0.7528 |
0.7680 |
0.5811 |
0.5333 |
DeepMeSH5 |
0.5955 |
0.8406 |
0.7813 |
0.7971 |
0.6291 |
0.5838 |
MeSHProbeNet-P |
0.5520 |
0.8250 |
0.7551 |
0.7725 |
0.5910 |
0.5403 |
NLM System 4 |
0.5235 |
0.8075 |
0.7224 |
0.7453 |
0.5646 |
0.5109 |
MeSHProbeNet-P0 |
0.5536 |
0.8162 |
0.7650 |
0.7739 |
0.5849 |
0.5484 |
MeSHProbeNet-P1 |
0.5549 |
0.8067 |
0.7751 |
0.7751 |
0.5783 |
0.5566 |
MeSHProbeNet-P2 |
0.5537 |
0.8222 |
0.7604 |
0.7742 |
0.5894 |
0.5446 |
MeSHProbeNet-P3 |
0.5540 |
0.8214 |
0.7616 |
0.7746 |
0.5894 |
0.5454 |
+ Test batch 2, week 4
Annotated articles:7079/7382
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6383 |
0.6712 |
0.6308 |
0.6305 |
0.6282 |
0.5892 |
0.5581 |
0.6601 |
0.6179 |
0.4794 |
Default MTI |
0.6485 |
0.6389 |
0.6826 |
0.6422 |
0.5951 |
0.6304 |
0.5780 |
0.6318 |
0.6660 |
0.4907 |
iria-mix |
0.5358 |
0.4672 |
0.6335 |
0.5242 |
0.4583 |
0.5098 |
0.4526 |
0.4719 |
0.6197 |
0.3682 |
DeepMeSH1 |
0.6925 |
0.7101 |
0.6929 |
0.6862 |
0.6715 |
0.6137 |
0.5969 |
0.7110 |
0.6749 |
0.5391 |
DeepMeSH2 |
0.6918 |
0.7117 |
0.6902 |
0.6856 |
0.6769 |
0.6084 |
0.5941 |
0.7128 |
0.6720 |
0.5383 |
DeepMeSH3 |
0.6913 |
0.7128 |
0.6881 |
0.6850 |
0.6765 |
0.6062 |
0.5928 |
0.7137 |
0.6701 |
0.5376 |
DeepMeSH4 |
0.6783 |
0.7030 |
0.6692 |
0.6702 |
0.6644 |
0.5706 |
0.5613 |
0.7054 |
0.6531 |
0.5205 |
DeepMeSH5 |
0.6954 |
0.7122 |
0.6955 |
0.6887 |
0.6745 |
0.6176 |
0.6007 |
0.7135 |
0.6782 |
0.5429 |
MeSHProbeNet-P |
0.6917 |
0.7147 |
0.6869 |
0.6835 |
0.6709 |
0.5953 |
0.5785 |
0.7097 |
0.6746 |
0.5360 |
NLM System 4 |
0.6573 |
0.6899 |
0.6418 |
0.6469 |
0.6137 |
0.5190 |
0.5099 |
0.6836 |
0.6329 |
0.4948 |
MeSHProbeNet-P0 |
0.6915 |
0.7089 |
0.6916 |
0.6832 |
0.6650 |
0.6001 |
0.5805 |
0.7040 |
0.6794 |
0.5356 |
MeSHProbeNet-P1 |
0.6913 |
0.7028 |
0.6970 |
0.6830 |
0.6596 |
0.6055 |
0.5828 |
0.6979 |
0.6848 |
0.5352 |
MeSHProbeNet-P2 |
0.6919 |
0.7203 |
0.6815 |
0.6833 |
0.6791 |
0.5836 |
0.5721 |
0.7156 |
0.6696 |
0.5359 |
MeSHProbeNet-P3 |
0.6803 |
0.7006 |
0.6780 |
0.6713 |
0.6538 |
0.5699 |
0.5515 |
0.6953 |
0.6659 |
0.5221 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5123 |
0.7835 |
0.7289 |
0.7357 |
0.5474 |
0.5087 |
Default MTI |
0.5256 |
0.7541 |
0.7730 |
0.7461 |
0.5319 |
0.5470 |
iria-mix |
0.4506 |
0.6195 |
0.7544 |
0.6647 |
0.4222 |
0.5098 |
DeepMeSH1 |
0.5563 |
0.8102 |
0.7724 |
0.7754 |
0.5839 |
0.5545 |
DeepMeSH2 |
0.5556 |
0.8118 |
0.7700 |
0.7747 |
0.5850 |
0.5520 |
DeepMeSH3 |
0.5545 |
0.8125 |
0.7682 |
0.7741 |
0.5854 |
0.5498 |
DeepMeSH4 |
0.5427 |
0.8080 |
0.7532 |
0.7635 |
0.5773 |
0.5352 |
DeepMeSH5 |
0.5599 |
0.8115 |
0.7745 |
0.7772 |
0.5870 |
0.5583 |
MeSHProbeNet-P |
0.5568 |
0.8174 |
0.7682 |
0.7756 |
0.5881 |
0.5534 |
NLM System 4 |
0.5260 |
0.8049 |
0.7257 |
0.7451 |
0.5661 |
0.5158 |
MeSHProbeNet-P0 |
0.5568 |
0.8128 |
0.7728 |
0.7759 |
0.5846 |
0.5566 |
MeSHProbeNet-P1 |
0.5571 |
0.8080 |
0.7775 |
0.7762 |
0.5811 |
0.5600 |
MeSHProbeNet-P2 |
0.5560 |
0.8216 |
0.7632 |
0.7746 |
0.5916 |
0.5489 |
MeSHProbeNet-P3 |
0.5479 |
0.8093 |
0.7625 |
0.7678 |
0.5791 |
0.5455 |
+ Test batch 2, week 5
Annotated articles:6756/7240
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6406 |
0.6869 |
0.6260 |
0.6371 |
0.6425 |
0.5693 |
0.5483 |
0.6726 |
0.6115 |
0.4908 |
Default MTI |
0.6450 |
0.6409 |
0.6739 |
0.6387 |
0.6014 |
0.6081 |
0.5644 |
0.6332 |
0.6571 |
0.4875 |
iria-mix |
0.5179 |
0.4579 |
0.6037 |
0.5082 |
0.4298 |
0.4757 |
0.4243 |
0.4606 |
0.5915 |
0.3540 |
DeepMeSH1 |
0.6895 |
0.7181 |
0.6807 |
0.6836 |
0.6855 |
0.5870 |
0.5786 |
0.7178 |
0.6634 |
0.5359 |
DeepMeSH2 |
0.6880 |
0.7190 |
0.6769 |
0.6819 |
0.6879 |
0.5791 |
0.5731 |
0.7192 |
0.6594 |
0.5341 |
DeepMeSH3 |
0.6883 |
0.7204 |
0.6758 |
0.6819 |
0.6895 |
0.5797 |
0.5750 |
0.7207 |
0.6586 |
0.5340 |
DeepMeSH4 |
0.6756 |
0.7108 |
0.6578 |
0.6677 |
0.6748 |
0.5440 |
0.5418 |
0.7122 |
0.6427 |
0.5169 |
DeepMeSH5 |
0.7120 |
0.7420 |
0.7041 |
0.7085 |
0.7142 |
0.6165 |
0.6095 |
0.7407 |
0.6854 |
0.5730 |
MeSHProbeNet-P |
0.6839 |
0.6850 |
0.6986 |
0.6746 |
0.6480 |
0.5985 |
0.5741 |
0.6807 |
0.6872 |
0.5254 |
NLM System 4 |
0.6514 |
0.6873 |
0.6329 |
0.6405 |
0.6175 |
0.5024 |
0.4989 |
0.6805 |
0.6246 |
0.4869 |
MeSHProbeNet-P0 |
0.6852 |
0.7038 |
0.6828 |
0.6755 |
0.6669 |
0.5817 |
0.5666 |
0.6994 |
0.6716 |
0.5268 |
MeSHProbeNet-P1 |
0.6848 |
0.7200 |
0.6673 |
0.6748 |
0.6847 |
0.5631 |
0.5561 |
0.7161 |
0.6562 |
0.5263 |
MeSHProbeNet-P2 |
0.6857 |
0.7149 |
0.6727 |
0.6754 |
0.6811 |
0.5647 |
0.5567 |
0.7112 |
0.6620 |
0.5268 |
MeSHProbeNet-P3 |
0.6832 |
0.6980 |
0.6848 |
0.6738 |
0.6591 |
0.5872 |
0.5685 |
0.6936 |
0.6731 |
0.5249 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
MTI First Line Index |
0.5260 |
0.7992 |
0.7244 |
0.7424 |
0.5691 |
0.5131 |
Default MTI |
0.5246 |
0.7613 |
0.7656 |
0.7463 |
0.5367 |
0.5401 |
iria-mix |
0.4401 |
0.6124 |
0.7339 |
0.6531 |
0.4161 |
0.4915 |
DeepMeSH1 |
0.5535 |
0.8178 |
0.7613 |
0.7739 |
0.5891 |
0.5439 |
DeepMeSH2 |
0.5513 |
0.8191 |
0.7574 |
0.7723 |
0.5885 |
0.5402 |
DeepMeSH3 |
0.5515 |
0.8202 |
0.7558 |
0.7718 |
0.5897 |
0.5397 |
DeepMeSH4 |
0.5366 |
0.8137 |
0.7409 |
0.7602 |
0.5779 |
0.5226 |
DeepMeSH5 |
0.5903 |
0.8332 |
0.7785 |
0.7914 |
0.6242 |
0.5801 |
MeSHProbeNet-P |
0.5520 |
0.7977 |
0.7793 |
0.7722 |
0.5708 |
0.5597 |
NLM System 4 |
0.5180 |
0.8052 |
0.7167 |
0.7402 |
0.5607 |
0.5057 |
MeSHProbeNet-P0 |
0.5507 |
0.8125 |
0.7646 |
0.7713 |
0.5812 |
0.5481 |
MeSHProbeNet-P1 |
0.5476 |
0.8254 |
0.7496 |
0.7689 |
0.5888 |
0.5359 |
MeSHProbeNet-P2 |
0.5494 |
0.8215 |
0.7540 |
0.7694 |
0.5873 |
0.5404 |
MeSHProbeNet-P3 |
0.5496 |
0.8079 |
0.7670 |
0.7705 |
0.5770 |
0.5495 |
+ Test batch 3, week 1
Annotated articles:5835/6266
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
NLM System 4 |
0.0967 |
0.6878 |
0.5990 |
0.6183 |
0.0364 |
0.4866 |
0.1232 |
0.0527 |
0.5867 |
0.4653 |
MTI First Line Index |
0.6499 |
0.6993 |
0.6377 |
0.6469 |
0.6475 |
0.5872 |
0.5646 |
0.6842 |
0.6189 |
0.5024 |
Default MTI |
0.6547 |
0.6519 |
0.6869 |
0.6492 |
0.6037 |
0.6266 |
0.5804 |
0.6436 |
0.6661 |
0.4993 |
iria-mix |
0.5221 |
0.4613 |
0.6129 |
0.5122 |
0.4250 |
0.4863 |
0.4335 |
0.4632 |
0.5983 |
0.3582 |
DeepMeSH1 |
0.6985 |
0.7423 |
0.6818 |
0.6944 |
0.6962 |
0.5994 |
0.5922 |
0.7412 |
0.6604 |
0.5492 |
DeepMeSH2 |
0.6977 |
0.7418 |
0.6808 |
0.6936 |
0.6986 |
0.5958 |
0.5894 |
0.7407 |
0.6594 |
0.5483 |
DeepMeSH3 |
0.6992 |
0.7227 |
0.6990 |
0.6947 |
0.6814 |
0.6110 |
0.5971 |
0.7223 |
0.6776 |
0.5492 |
DeepMeSH4 |
0.7277 |
0.7493 |
0.7261 |
0.7229 |
0.7148 |
0.6429 |
0.6317 |
0.7500 |
0.7068 |
0.5908 |
DeepMeSH5 |
0.7279 |
0.7671 |
0.7107 |
0.7227 |
0.7292 |
0.6328 |
0.6285 |
0.7684 |
0.6914 |
0.5909 |
MeSHProbeNet-P |
0.6912 |
0.7294 |
0.6790 |
0.6841 |
0.6851 |
0.5745 |
0.5647 |
0.7235 |
0.6616 |
0.5375 |
MeSHProbeNet-P0 |
0.6921 |
0.7190 |
0.6897 |
0.6850 |
0.6735 |
0.5865 |
0.5717 |
0.7131 |
0.6723 |
0.5382 |
MeSHProbeNet-P1 |
0.6926 |
0.7078 |
0.7007 |
0.6855 |
0.6610 |
0.5981 |
0.5780 |
0.7020 |
0.6834 |
0.5385 |
MeSHProbeNet-P2 |
0.6925 |
0.7142 |
0.6940 |
0.6853 |
0.6705 |
0.5854 |
0.5707 |
0.7084 |
0.6772 |
0.5382 |
MeSHProbeNet-P3 |
0.6913 |
0.7107 |
0.6943 |
0.6839 |
0.6704 |
0.5819 |
0.5683 |
0.7050 |
0.6781 |
0.5363 |
lcy_test |
0.1748 |
0.3593 |
0.1338 |
0.1758 |
0.2689 |
0.1668 |
0.1424 |
0.3032 |
0.1228 |
0.1022 |
Hierarchical Measures
System |
LCA-F |
HiP |
HiR |
HiF |
LCA-P |
LCA-R |
NLM System 4 |
0.5007 |
0.8007 |
0.6974 |
0.7231 |
0.5559 |
0.4844 |
MTI First Line Index |
0.5299 |
0.8030 |
0.7310 |
0.7455 |
0.5733 |
0.5201 |
Default MTI |
0.5284 |
0.7619 |
0.7742 |
0.7494 |
0.5389 |
0.5476 |
iria-mix |
0.4399 |
0.6113 |
0.7413 |
0.6541 |
0.4135 |
0.4966 |
DeepMeSH1 |
0.5569 |
0.8311 |
0.7596 |
0.7777 |
0.5988 |
0.5438 |
DeepMeSH2 |
0.5566 |
0.8318 |
0.7585 |
0.7774 |
0.5990 |
0.5433 |
DeepMeSH3 |
0.5597 |
0.8156 |
0.7750 |
0.7791 |
0.5887 |
0.5573 |
DeepMeSH4 |
0.6004 |
0.8327 |
0.7947 |
0.7988 |
0.6273 |
0.5979 |
DeepMeSH5 |
0.5979 |
0.8469 |
0.7811 |
0.7978 |
0.6362 |
0.5859 |
MeSHProbeNet-P |
0.5523 |
0.8259 |
0.7574 |
0.7718 |
0.5932 |
0.5429 |
MeSHProbeNet-P0 |
0.5541 |
0.8177 |
0.7675 |
0.7736 |
0.5875 |
0.5511 |
MeSHProbeNet-P1 |
0.5557 |
0.8087 |
0.7776 |
0.7750 |
0.5810 |
0.5596 |
MeSHProbeNet-P2 |
0.5549 |
0.8146 |
0.7711 |
0.7744 |
0.5851 |
0.5544 |
MeSHProbeNet-P3 |
0.5547 |
0.8125 |
0.7718 |
0.7740 |
0.5837 |
0.5549 |
lcy_test |
0.2042 |
0.6267 |
0.2584 |
0.3375 |
0.3522 |
0.1596 |
+ Test batch 3, week 2
Annotated articles:10386/11455
Flat Measures
System |
MiF |
EBP |
EBR |
EBF |
MaP |
MaR |
MaF |
MiP |
MiR |
Acc. |
MTI First Line Index |
0.6413 |
0.6887 |
0.6304 |
0.6379 |
0.6446 |
0.5767 |
0.5523 |
0.6740 |
0.6116 |
0.4914 |
Default MTI |
0.6454 |
0.6392 |
0.6810 |
0.6392 |
0.6015 |
0.6185 |
0.5695 |
0.6317 |
0.6598 |
0.4878 |
iria-mix |
0.5139 |
0.4500 |
0.6094 |
0.5039 |
0.4261 |
0.4774 |
0.4231 |
0.4529 |
0.5939 |
0.3500 |
DeepMeSH1 |
0.6956 |
0.7326 |
0.6832 |
0.6900 |
0.7005 |
0.6006 |
0.5945 |
0.7325 |
0.6622 |
0.5438 |
DeepMeSH2 |
0.6950 |
0.7180 |
0.6938 |
0.6893 |
0.6945 |
0.6049 |
0.5950 |
0.7183 |
0.6731 |
0.5429 |