{"id":695,"date":"2024-06-20T23:33:17","date_gmt":"2024-06-20T15:33:17","guid":{"rendered":"https:\/\/ainlp.tw\/?p=695"},"modified":"2024-11-09T18:58:05","modified_gmt":"2024-11-09T10:58:05","slug":"nycu-nlp-at-semeval-2024-task-2-aggregating-large-language-models-in-biomedical-natural-language-inference-for-clinical-trials","status":"publish","type":"post","link":"https:\/\/ainlp.tw\/index.php\/2024\/06\/20\/nycu-nlp-at-semeval-2024-task-2-aggregating-large-language-models-in-biomedical-natural-language-inference-for-clinical-trials\/","title":{"rendered":"NYCU-NLP at SemEval-2024 Task 2: Aggregating Large Language Models in Biomedical Natural Language Inference for Clinical Trials"},"content":{"rendered":"\n<p>Lung-Hao Lee, Chen-Ya Chiou and Tzu-Mi Lin. <\/p>\n\n\n\n<p>In <em>Proceedings of the 18th International Workshop on Semantic Evaluation (<strong>SemEval-2024<\/strong>)<\/em>, pages 1455\u20131462.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity is-style-wide\"\/>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Abstract<\/strong><\/p>\n\n\n\n<p>This study describes the model design of the NYCU-NLP system for the SemEval- 2024 Task 2 that focuses on natural language inference for clinical trials. We aggregate several large language models to determine the inference relation (i.e., entailment or contradiction) between clinical trial reports and statements that may be manipulated with designed interventions to investigate the faithfulness and consistency of the developed models. First, we use ChatGPT v3.5 to augment original statements in training data and then fine-tune the SOLAR model with all augmented data. During the testing inference phase, we fine-tune the OpenChat model to reduce the influence of interventions and fed a cleaned statement into the fine-tuned SOLAR model for label prediction. Our submission produced a faithfulness score of 0.9236, ranking second of 32 participating teams, and ranked first for consistency with a score of 0.8092.<\/p>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"549\" src=\"https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23-1024x549.png\" alt=\"\" class=\"wp-image-696\" srcset=\"https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23-1024x549.png 1024w, https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23-300x161.png 300w, https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23-768x411.png 768w, https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23-1536x823.png 1536w, https:\/\/ainlp.tw\/wp-content\/uploads\/2024\/06\/\u622a\u5716-2024-06-30-\u4e0a\u534812.36.23.png 1624w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Lung-Hao Lee, Chen-Ya Chiou and Tzu-Mi Lin<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[6],"tags":[33,44,17],"class_list":["post-695","post","type-post","status-publish","format-standard","hentry","category-achievements","tag-33","tag-healthcare","tag-semeval"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/posts\/695","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/comments?post=695"}],"version-history":[{"count":3,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/posts\/695\/revisions"}],"predecessor-version":[{"id":768,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/posts\/695\/revisions\/768"}],"wp:attachment":[{"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/media?parent=695"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/categories?post=695"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ainlp.tw\/index.php\/wp-json\/wp\/v2\/tags?post=695"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}