{"id":84,"date":"2021-06-05T16:49:41","date_gmt":"2021-06-05T07:49:41","guid":{"rendered":"https:\/\/sociocom.naist.jp\/real-mednlp\/?page_id=84"},"modified":"2025-05-14T19:22:05","modified_gmt":"2025-05-14T10:22:05","slug":"real-mednlp","status":"publish","type":"page","link":"https:\/\/sociocom.naist.jp\/real-mednlp\/","title":{"rendered":"Frontpage (English)"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"84\" class=\"elementor elementor-84\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e88512f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e88512f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-e70ae4c\" data-id=\"e70ae4c\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2ac9ae4 elementor-widget elementor-widget-html\" data-id=\"2ac9ae4\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\n<div class=\"wp-block-cover alignfull has-nv-dark-bg-background-color has-background-dim\" style=\"min-height: 400px;aspect-ratio:unset;\"><img fetchpriority=\"high\" decoding=\"async\" width=\"2560\" height=\"1707\" class=\"wp-block-cover__image-background wp-image-135\" style=\"object-position: 55% 61%\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-scaled.jpg\" alt=\"\" data-object-fit=\"cover\" data-object-position=\"55% 61%\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-scaled.jpg 2560w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-300x200.jpg 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-1024x683.jpg 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-768x512.jpg 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-1536x1024.jpg 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2048x1365.jpg 2048w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-930x620.jpg 930w\" sizes=\"(max-width: 2560px) 100vw, 2560px\" \/>\n<div class=\"wp-block-cover__inner-container is-layout-flow wp-block-cover-is-layout-flow\">\n<h1 class=\"has-text-align-center has-nv-text-dark-bg-color has-text-color wp-block-heading\">NTCIR-16 Real-MedNLP<\/h1>\n\n\n\n<p style=\"font-size: 18px\">With the advent of the internet age, medical records are increasingly being written in electronic formats in place of paper, which leads to a higher importance of information processing techniques in medical fields. However, the amount of privacy-free medical text data is still small in non-English languages, such as Japanese and Chinese. In such a situation, we had proposed a series of previous four medical natural language processing (MedNLP) tasks, MedNLP-1, MedNLP-2, MedNLPDoc, and MedWeb. This task will yield promising technologies to develop practical computational systems for supporting a wide range of medical services.<\/p>\n<\/div>\n<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-02d45fd elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"02d45fd\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f657212\" data-id=\"f657212\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5a86985 elementor-widget elementor-widget-heading\" data-id=\"5a86985\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">NEWS<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ee81c43 elementor-widget elementor-widget-wp-widget-mtphr-dnt-widget\" data-id=\"ee81c43\" data-element_type=\"widget\" data-widget_type=\"wp-widget-mtphr-dnt-widget.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"mtphr-dnt-901-widget\" class=\"mtphr-dnt mtphr-dnt-901 mtphr-dnt-default mtphr-dnt-scroll mtphr-dnt-scroll-left\"><div class=\"mtphr-dnt-wrapper mtphr-dnt-clearfix\"><div class=\"mtphr-dnt-tick-container\"><div class=\"mtphr-dnt-tick-contents\"><div class=\"mtphr-dnt-tick mtphr-dnt-default-tick mtphr-dnt-clearfix \">12.22\u3000Updated Schedule\u3000\/\u3000\u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u3092\u66f4\u65b0\u3057\u307e\u3057\u305f<\/div><\/div><\/div><\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c59bace elementor-widget elementor-widget-wp-widget-mtphr-dnt-widget\" data-id=\"c59bace\" data-element_type=\"widget\" data-widget_type=\"wp-widget-mtphr-dnt-widget.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"mtphr-dnt-887-widget\" class=\"mtphr-dnt mtphr-dnt-887 mtphr-dnt-default mtphr-dnt-scroll mtphr-dnt-scroll-left\"><div class=\"mtphr-dnt-wrapper mtphr-dnt-clearfix\"><div class=\"mtphr-dnt-tick-container\"><div class=\"mtphr-dnt-tick-contents\"><div class=\"mtphr-dnt-tick mtphr-dnt-default-tick mtphr-dnt-clearfix \">12.20\u3000Updated Annotation Guidelines\u3000\/\u3000\u30a2\u30ce\u30c6\u30fc\u30b7\u30e7\u30f3\u30ac\u30a4\u30c9\u30e9\u30a4\u30f3\u3092\u66f4\u65b0\u3057\u307e\u3057\u305f<\/div><\/div><\/div><\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2b294e7 elementor-widget elementor-widget-wp-widget-mtphr-dnt-widget\" data-id=\"2b294e7\" data-element_type=\"widget\" data-widget_type=\"wp-widget-mtphr-dnt-widget.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"mtphr-dnt-845-widget\" class=\"mtphr-dnt mtphr-dnt-845 mtphr-dnt-default mtphr-dnt-scroll mtphr-dnt-scroll-left\"><div class=\"mtphr-dnt-wrapper mtphr-dnt-clearfix\"><div class=\"mtphr-dnt-tick-container\"><div class=\"mtphr-dnt-tick-contents\"><div class=\"mtphr-dnt-tick mtphr-dnt-default-tick mtphr-dnt-clearfix \">11.10\u3000Updated FAQ\u3000\/\u3000FAQ\u3092\u66f4\u65b0\u3057\u307e\u3057\u305f<\/div><\/div><\/div><\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a5eac71 elementor-widget elementor-widget-wp-widget-mtphr-dnt-widget\" data-id=\"a5eac71\" data-element_type=\"widget\" data-widget_type=\"wp-widget-mtphr-dnt-widget.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div id=\"mtphr-dnt-194-widget\" class=\"mtphr-dnt mtphr-dnt-194 mtphr-dnt-default mtphr-dnt-scroll mtphr-dnt-scroll-left\"><div class=\"mtphr-dnt-wrapper mtphr-dnt-clearfix\"><div class=\"mtphr-dnt-tick-container\"><div class=\"mtphr-dnt-tick-contents\"><div class=\"mtphr-dnt-tick mtphr-dnt-default-tick mtphr-dnt-clearfix \">10.22\u3000Removed irrelevant pages from Annotation Guidelines\u3000\/\u3000\u30a2\u30ce\u30c6\u30fc\u30b7\u30e7\u30f3\u30ac\u30a4\u30c9\u30e9\u30a4\u30f3\u3092\u7e2e\u5c0f\u3057\u307e\u3057\u305f<\/div><\/div><\/div><\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-49f35c1 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"49f35c1\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-151eb20\" data-id=\"151eb20\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-bc6229c elementor-widget elementor-widget-heading\" data-id=\"bc6229c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">About this workshop<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2cb90b0 elementor-widget elementor-widget-text-editor\" data-id=\"2cb90b0\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Real-MedNLP is a shared task workshop for medical language processing using actual medical documents (case reports and radiology reports). The goal of this task is to promote the development of practical systems that support various medical services.<\/p>\n<p>The Real-MedNLP task has two corpus-based tracks (MedTxt-CR Track and MedTxt-RR Track), each with three subtasks.<\/p>\n<p>The task overview paper is available <a href=\"http:\/\/research.nii.ac.jp\/ntcir\/workshop\/OnlineProceedings16\/pdf\/ntcir\/01-NTCIR16-OV-MEDNLP-YadaS.pdf\">here<\/a>.\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ec7a8a2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ec7a8a2\" data-element_type=\"section\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d8910b4\" data-id=\"d8910b4\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f6d6338 elementor-widget elementor-widget-heading\" data-id=\"f6d6338\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Datasets<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a3c2dcb elementor-align-center elementor-widget elementor-widget-button\" data-id=\"a3c2dcb\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/sociocom.naist.jp\/download\/real-mednlp-test-collection\/\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-external-link-alt\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Test Collection<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-5682d6f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5682d6f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-e933ec9\" data-id=\"e933ec9\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-473542a elementor-widget elementor-widget-heading\" data-id=\"473542a\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">MedTxt-CR Corpus<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-458dabf elementor-widget elementor-widget-text-editor\" data-id=\"458dabf\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This dataset comprises a set of open-access case reports available at CiNii. Typically, open-access case reports are biased in the reporting of patients and diseases, due to differences in the policies various medical societies have towards open-access publications.<\/p>\n<p>To reduce such bias caused by each medical society\u2019s publication policy, we select case reports based on actual frequencies of patients and diseases.<\/p>\n<ul>\n<li>Training set:\u00a0<strong>100<\/strong> reports<\/li>\n<li>Test set: <strong>100<\/strong>\u00a0reports<\/li>\n<\/ul>\n<p>\u00a0<\/p>\n<p><strong>A case report<\/strong> is detailed descriptions of a patient\u2019s medical condition for research purposes. They track the onset and temporal progression of the patient\u2019s disease and are large in quantity as medical societies typically have dedicated submission tracks for consolidating these reports. Considering these advantages, case reports possess enormous potential as a rich source of information. Furthermore, the format of a case report is similar to that of a discharge summary, which is frequently used in healthcare contexts. Techniques developed here for case report analysis could also be applied to analyze discharge summaries.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-fd78d88\" data-id=\"fd78d88\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c5e75c9 elementor-widget elementor-widget-image\" data-id=\"c5e75c9\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample.png\" data-elementor-open-lightbox=\"yes\" data-elementor-lightbox-title=\"medtxt-cr-en-sample\" data-e-action-hash=\"#elementor-action%3Aaction%3Dlightbox%26settings%3DeyJpZCI6IjU5NSIsInVybCI6Imh0dHBzOlwvXC9zb2Npb2NvbS5uYWlzdC5qcFwvcmVhbC1tZWRubHBcL3dwLWNvbnRlbnRcL3VwbG9hZHNcL3NpdGVzXC8zXC8yMDIxXC8wOFwvbWVkdHh0LWNyLWVuLXNhbXBsZS5wbmcifQ%3D%3D\">\n\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"408\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample-1024x408.png\" class=\"attachment-large size-large wp-image-595\" alt=\"\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample-1024x408.png 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample-300x119.png 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample-768x306.png 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample-1536x612.png 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/medtxt-cr-en-sample.png 1788w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">An example of a case report with named entities recognized<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f093572 elementor-button-info elementor-align-center elementor-widget__width-auto elementor-widget elementor-widget-button\" data-id=\"f093572\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/MedTxt-CR-JA.zip\" target=\"_blank\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-download\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">MedTxt-CR Corpus Sample<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-1a4d883 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"1a4d883\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-b06cfbd\" data-id=\"b06cfbd\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c002354 elementor-widget elementor-widget-heading\" data-id=\"c002354\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">MedTxt-RR Corpus<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-48b80eb elementor-widget elementor-widget-text-editor\" data-id=\"48b80eb\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This dataset comprises a set of\u00a0<strong>15<\/strong>\u00a0cases, in which 9 different radiologists describe the findings for each report. In total,\u00a0<strong>135<\/strong>\u00a0texts will be made available.<\/p>\n<ul>\n<li>Training set: <strong>72<\/strong> texts<\/li>\n<li>Test set: <strong>63<\/strong>\u00a0texts<\/li>\n<\/ul>\n<p>\u00a0<\/p>\n<p><strong>A radiology report<\/strong>\u00a0is a type of clinical document that is written by a radiologist. Basically, they focus on a single radiology image and describes all potential findings (including potential diseases) that can be expected from the image. While reports and target images are paired, most research on radiology reports tends to focus only on images, due to the hype surrounding image-based AI \u00a0(such as automatic diagnosis of X-rays, CT, and MRI). One of the biggest problems when handling radiology reports is in the variety of writing styles. Although a diagnosis can be written in a variety of ways (diversity of expression), conventionally, only one report is created per image. As such, simply collecting reports from medical institutions may not yield enough information on the variability in reporting styles for the same diagnosis. Consequently, we included independent reports from multiple doctors for the same CT image.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-d0b59af\" data-id=\"d0b59af\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b1a4187 elementor-widget elementor-widget-image\" data-id=\"b1a4187\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17.png\" data-elementor-open-lightbox=\"yes\" data-elementor-lightbox-title=\"image-17\" data-e-action-hash=\"#elementor-action%3Aaction%3Dlightbox%26settings%3DeyJpZCI6IjI4MiIsInVybCI6Imh0dHBzOlwvXC9zb2Npb2NvbS5uYWlzdC5qcFwvcmVhbC1tZWRubHBcL3dwLWNvbnRlbnRcL3VwbG9hZHNcL3NpdGVzXC8zXC8yMDIxXC8wNlwvaW1hZ2UtMTcucG5nIn0%3D\">\n\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"280\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17-1024x280.png\" class=\"attachment-large size-large wp-image-282\" alt=\"\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17-1024x280.png 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17-300x82.png 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17-768x210.png 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17-1536x419.png 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/image-17.png 1634w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">An example of a radiology report with named entities recognized<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e6db48d elementor-button-info elementor-align-center elementor-widget__width-auto elementor-widget elementor-widget-button\" data-id=\"e6db48d\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/MedTxt-RR-JA.zip\" target=\"_blank\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-download\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">MedTxt-RR Corpus Sample<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-dc0e4b3 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"dc0e4b3\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-021ccf0\" data-id=\"021ccf0\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a20a70f elementor-widget elementor-widget-heading\" data-id=\"a20a70f\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Task Overview<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0bd73d5 elementor-widget elementor-widget-heading\" data-id=\"0bd73d5\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Few-resource Named Entity Recognition (NER)<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-1d7c443 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"1d7c443\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-3c9a853\" data-id=\"3c9a853\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-13fb996 elementor-widget elementor-widget-text-editor\" data-id=\"13fb996\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Since NER is arguably the most fundamental information extraction problem for MedNLP, we designed challenges pertaining to NER on our real-world clinical documents, with a sample of only 100-200 documents. This scale of corpus size tends to be regarded as a \u201cfew-resource machine learning\u201d, which is a de-facto standard for any sort of MedNLP in general.<\/p>\n<h4>Subtask 1: Just 100 Training<\/h4>\n<ul>\n<li>NER using the training set consisting of 100 documents<\/li>\n<li>This subtask is equivalent to standard supervised learning with few resources<\/li>\n<\/ul>\n<h4>Subtask 2: Guideline Learning<\/h4>\n<ul>\n<li>NER using the example sentence(s) for each tag in Annotation Guideline and not using the training set for Subtask 1<\/li>\n<li>This simulates the training of human annotators, who often learn from the annotation guidelines provided by researchers.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-4d481cd\" data-id=\"4d481cd\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-67c9edb elementor-widget elementor-widget-html\" data-id=\"67c9edb\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<!-- HTML generated using hilite.me --><div style=\"background: #ffffff;overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em\"><pre style=\"margin: 0;line-height: 125%\"><span style=\"color: #007700\">&lt;article<\/span> <span style=\"color: #0000CC\">id=<\/span><span style=\"background-color: #fff0f0\">&quot;JP0217-29&quot;<\/span> <span style=\"color: #0000CC\">title=<\/span><span style=\"background-color: #fff0f0\">&quot;\u8457\u660e\u306a\u597d\u9178\u7403\u5897\u591a\u3092\u4f34\u3063\u305f\u975e\u660f\u7761\u578b\u6025\u6027\u809d\u4e0d\u5168\u306e\u4e00\u4f8b&quot;<\/span><span style=\"color: #007700\">&gt;<\/span> \nCase Study: <span style=\"color: #007700\">&lt;TIMEX3<\/span> <span style=\"color: #0000CC\">type=<\/span><span style=\"background-color: #fff0f0\">&quot;AGE&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>53 year old<span style=\"color: #007700\">&lt;\/TIMEX3&gt;<\/span> female patient.\nChief Complaint: <span style=\"color: #007700\">&lt;d<\/span> <span style=\"color: #0000CC\">certainty=<\/span><span style=\"background-color: #fff0f0\">&quot;positive&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>Fever<span style=\"color: #007700\">&lt;\/d&gt;<\/span>.\nProgress: Patient was <span style=\"color: #007700\">&lt;cc<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;executed&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>seen<span style=\"color: #007700\">&lt;\/cc&gt;<\/span> at the dermatology department of our hospital <span style=\"color: #007700\">&lt;TIMEX3<\/span> <span style=\"color: #0000CC\">type=<\/span><span style=\"background-color: #fff0f0\">&quot;DATE&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>2 years before 20XX<span style=\"color: #007700\">&lt;\/TIMEX3&gt;<\/span> and presented a <span style=\"color: #007700\">&lt;d<\/span> <span style=\"color: #0000CC\">certainty=<\/span><span style=\"background-color: #fff0f0\">&quot;positive&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>skin rash<span style=\"color: #007700\">&lt;\/d&gt;<\/span> that the diagnosis identified as <span style=\"color: #007700\">&lt;d<\/span> <span style=\"color: #0000CC\">certainty=<\/span><span style=\"background-color: #fff0f0\">&quot;positive&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>bullous pemphigoid<span style=\"color: #007700\">&lt;\/d&gt;<\/span>.\n<span style=\"color: #007700\">&lt;m-key<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;executed&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>Prednisolone (PSL)<span style=\"color: #007700\">&lt;\/m-key&gt;<\/span> <span style=\"color: #007700\">&lt;m-val&gt;<\/span>1 mg\/kg\/day<span style=\"color: #007700\">&lt;\/m-val&gt;<\/span> was introduced and the patient <span style=\"color: #007700\">&lt;TIMEX3<\/span> <span style=\"color: #0000CC\">type=<\/span><span style=\"background-color: #fff0f0\">&quot;TIME&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>was<span style=\"color: #007700\">&lt;\/TIMEX3&gt;<\/span> managed with concomitant <span style=\"color: #007700\">&lt;m-key<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;executed&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>immunomodulators<span style=\"color: #007700\">&lt;\/m-key&gt;<\/span> with <span style=\"color: #007700\">&lt;c&gt;<\/span>a gradual decrease<span style=\"color: #007700\">&lt;\/c&gt;<\/span> in the <span style=\"color: #007700\">&lt;m-key<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;executed&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>PSL<span style=\"color: #007700\">&lt;\/m-key&gt;<\/span> level.\n<span style=\"color: #007700\">&lt;m-key<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;negated&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>PSL<span style=\"color: #007700\">&lt;\/m-key&gt;<\/span> was voluntarily discontinued in <span style=\"color: #007700\">&lt;TIMEX3<\/span> <span style=\"color: #0000CC\">type=<\/span><span style=\"background-color: #fff0f0\">&quot;DATE&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>August, 20XX<span style=\"color: #007700\">&lt;\/TIMEX3&gt;<\/span> when the <span style=\"color: #007700\">&lt;m-key<\/span> <span style=\"color: #0000CC\">state=<\/span><span style=\"background-color: #fff0f0\">&quot;executed&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>PSL<span style=\"color: #007700\">&lt;\/m-key&gt;<\/span> dosage had been <span style=\"color: #007700\">&lt;c&gt;<\/span>reduced<span style=\"color: #007700\">&lt;\/c&gt;<\/span> to <span style=\"color: #007700\">&lt;m-val&gt;<\/span>6 mg\/day<span style=\"color: #007700\">&lt;\/m-val&gt;<\/span>, but there was no <span style=\"color: #007700\">&lt;d<\/span> <span style=\"color: #0000CC\">certainty=<\/span><span style=\"background-color: #fff0f0\">&quot;negative&quot;<\/span><span style=\"color: #007700\">&gt;<\/span>worsening of the skin rash<span style=\"color: #007700\">&lt;\/d&gt;<\/span>.\n...(skipped)...\n<span style=\"color: #007700\">&lt;\/article&gt;<\/span>\n<\/pre><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-e999653 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e999653\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-ac852a6\" data-id=\"ac852a6\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-70f450a elementor-button-info elementor-align-center elementor-widget elementor-widget-button\" data-id=\"70f450a\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/Yada2020.lrec-1.561-1.pdf\" target=\"_blank\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-download\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">NE Annotation Scheme (Yada et al., 2020)<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-5b8b803\" data-id=\"5b8b803\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2b4cce5 elementor-align-center elementor-widget elementor-widget-button\" data-id=\"2b4cce5\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm\" href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/12\/Real-MedNLP_Annotation_Guidelines_English.pdf\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-book\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">Annotation Guidelines (v8)<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1583b3d elementor-widget elementor-widget-text-editor\" data-id=\"1583b3d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"background-color: var(--nv-site-bg)\">[Updated on\u00a0October 22, 2021] Irrelevant pages were removed from the Annotation Guidelines<\/span><\/p>\n<p><span style=\"background-color: var(--nv-site-bg)\">[Updated on December 20, 2021] A small portion of text and annotation examples were fixed\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<div class=\"elementor-element elementor-element-7219393 elementor-widget elementor-widget-spacer\" data-id=\"7219393\" data-element_type=\"widget\" data-widget_type=\"spacer.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-spacer\">\n\t\t\t<div class=\"elementor-spacer-inner\"><\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-920ec54 elementor-widget elementor-widget-heading\" data-id=\"920ec54\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Application<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-0fdbfa7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0fdbfa7\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-33dae94\" data-id=\"33dae94\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5409b99 elementor-widget elementor-widget-text-editor\" data-id=\"5409b99\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h4>Subtask 3: Application<\/h4>\n<h5>[MedTxt-CR Track] Adverse Drug Event detection (ADE)<\/h5>\n<p>Extract adverse drug event (ADE) information from case reports and fill out a table<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-0eac386\" data-id=\"0eac386\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c967bdb elementor-widget elementor-widget-image\" data-id=\"c967bdb\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"451\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/ADE-en-1024x451.png\" class=\"attachment-large size-large wp-image-555\" alt=\"Task image of Subtask 3 ADE\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/ADE-en-1024x451.png 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/ADE-en-300x132.png 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/ADE-en-768x338.png 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/ADE-en.png 1394w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<div class=\"elementor-element elementor-element-f650afd elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"f650afd\" data-element_type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<section class=\"elementor-section elementor-inner-section elementor-element elementor-element-ea752d8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ea752d8\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-d2b9933\" data-id=\"d2b9933\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a9591c0 elementor-widget elementor-widget-text-editor\" data-id=\"a9591c0\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h5 style=\"letter-spacing: normal\">[MedTxt-RR Track] Case Identi\ufb01cation (CI)<\/h5>\n<p>Identify the radiology reports for the same case<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-inner-column elementor-element elementor-element-2675aa3\" data-id=\"2675aa3\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-376d4bc elementor-widget elementor-widget-image\" data-id=\"376d4bc\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"534\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en-1024x534.png\" class=\"attachment-large size-large wp-image-566\" alt=\"\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en-1024x534.png 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en-300x156.png 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en-768x400.png 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en-1536x800.png 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/08\/CI-en.png 1996w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5050f7c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5050f7c\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-224e38c\" data-id=\"224e38c\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5b2a468 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"5b2a468\" data-element_type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-92b8c1c elementor-widget elementor-widget-heading\" data-id=\"92b8c1c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Schedule<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9d3b6e5 elementor-widget elementor-widget-text-editor\" data-id=\"9d3b6e5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n<li><del>September 30, 2021: Datasets release (email to registered teams)<\/del><\/li>\n<li><del datetime=\"2021-07-01T05:14:34+00:00\">September 1 December 1, 2021: Registration Due<\/del><\/li>\n<li><del>January 10-17, 2022: Formal run of Subtasks 1 and 2<\/del>\n<ul>\n<li><del>January 10, 2022: Test data release (the organizer to participants)<\/del><\/li>\n<li><del>January 17, 2022: Result submission (participants to the organizer)<\/del><\/li>\n<\/ul>\n<\/li>\n<li><del>January 18-25, 2022: Formal run of Subtask 3 ADE and CI<\/del>\n<ul>\n<li><del>January 18, 2022: Test data release (the organizer to participants)<\/del><\/li>\n<li><del>January 25, 2022: Result submission (participants to the organizer)<\/del><\/li>\n<\/ul>\n<\/li>\n<li><del>February 1, 2022: Evaluation Result Release<\/del><\/li>\n<li><del>February 1, 2022: Draft Task Overview Paper Release<\/del><\/li>\n<li><del>March 1, 2022: Draft Participant Paper Submission Due<\/del><\/li>\n<li><del>May 1, 2022: All Camera-ready Paper Submission Due<\/del><\/li>\n<li><del>June 14-17, 2022: NTCIR-16 Conference (Online) [<a href=\"https:\/\/research.nii.ac.jp\/ntcir\/ntcir-16\/program.html\">Program<\/a>] [<a href=\"https:\/\/research.nii.ac.jp\/ntcir\/ntcir-16\/participation.html\">Registration<\/a>]\n<\/del><ul>\n<li><del>June 17, 2022: <a href=\"https:\/\/research.nii.ac.jp\/ntcir\/ntcir-16\/program.html#day4-realmednlp\">Real-MedNLP Session<\/a><\/del><\/li>\n<\/ul>\n<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-01f4158 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"01f4158\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a4dcb9e\" data-id=\"a4dcb9e\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f41b439 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"f41b439\" data-element_type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-efc7e89 elementor-widget elementor-widget-heading\" data-id=\"efc7e89\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Entry Form<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3e9050e elementor-button-success elementor-align-center elementor-widget elementor-widget-button\" data-id=\"3e9050e\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t\t\t<a class=\"elementor-button elementor-size-lg\" role=\"button\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-icon\">\n\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-window-close\"><\/i>\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t<span class=\"elementor-button-text\">NTCIR-16 Entry Form (Closed)<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a3f0de7 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"a3f0de7\" data-element_type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c7d330f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c7d330f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-bf854dd\" data-id=\"bf854dd\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fcd18f3 elementor-widget elementor-widget-heading\" data-id=\"fcd18f3\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">FAQ<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-108601b elementor-widget elementor-widget-toggle\" data-id=\"108601b\" data-element_type=\"widget\" data-widget_type=\"toggle.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-toggle\">\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1731\" class=\"elementor-tab-title\" data-tab=\"1\" role=\"button\" aria-controls=\"elementor-tab-content-1731\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.1\u3000What is the major difference between Subtasks 1 and 2?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1731\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"1\" role=\"region\" aria-labelledby=\"elementor-tab-title-1731\"><p>In Subtask 1, participants can use the training dataset to supervise your model. In Subtask 2, participants cannot use the training dataset to supervise a model, whereas participants may use the example sentences appearing in the annotation guidelines (as if a model is a human annotator).<br \/>Both subtasks allow participants to include any external resource (i.e., semi-supervision is OK).<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1732\" class=\"elementor-tab-title\" data-tab=\"2\" role=\"button\" aria-controls=\"elementor-tab-content-1732\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.2\u3000What is the format of test data for Subtasks 1 and 2?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1732\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"2\" role=\"region\" aria-labelledby=\"elementor-tab-title-1732\"><p>Clear reports (removed all tags) are provided as the test dataset. Participants should add tags to the test data like the training data.\u00a0<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1733\" class=\"elementor-tab-title\" data-tab=\"3\" role=\"button\" aria-controls=\"elementor-tab-content-1733\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.3\u3000What are the evaluation criteria of Subtasks 1 and 2?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1733\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"3\" role=\"region\" aria-labelledby=\"elementor-tab-title-1733\"><p>In Subtasks 1 and 2, we plan to perform two-level evaluations: the entity level and the entity+attribute level (joint). In the joint evaluation, it is regarded as correct if both the entity and the attribute match, and incorrect if the attribute matches but the entity does not. We recommend to recognize both the entity and the attribute from a practical perspective, though whether or not to deal with the attribute is optional.<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1734\" class=\"elementor-tab-title\" data-tab=\"4\" role=\"button\" aria-controls=\"elementor-tab-content-1734\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.4\u3000What are the evaluation metrics about Subtasks 1 and 2?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1734\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"4\" role=\"region\" aria-labelledby=\"elementor-tab-title-1734\"><p>We plan to employ various evaluation metrics for Subtasks 1 and 2 such as precision, recall, F1 (micro, macro). The details will be announced later.\u00a0<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1735\" class=\"elementor-tab-title\" data-tab=\"5\" role=\"button\" aria-controls=\"elementor-tab-content-1735\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.5\u3000Does the ADE challenge include the NER task?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1735\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"5\" role=\"region\" aria-labelledby=\"elementor-tab-title-1735\"><p>No, it does not include the NER task in the ADE challenge. We provide the reports with tags for the ADE challenge. Since \u201carticleID,\u201d \u201ctag,\u201d and \u201ctext\u201d are given, participants should answer the value of \u201cADEval.\u201d<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1736\" class=\"elementor-tab-title\" data-tab=\"6\" role=\"button\" aria-controls=\"elementor-tab-content-1736\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.6\u3000What is the evaluation metric for the CI challenge?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1736\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"6\" role=\"region\" aria-labelledby=\"elementor-tab-title-1736\"><p>Since the CI challenge is a clustering task, we evaluate the performance with Normalized Mutual Information (NMI).<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1737\" class=\"elementor-tab-title\" data-tab=\"7\" role=\"button\" aria-controls=\"elementor-tab-content-1737\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.7\u3000How long is the formal run (from the release of the test set to the submission of the final result)?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1737\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"7\" role=\"region\" aria-labelledby=\"elementor-tab-title-1737\"><p>The formal run period would be about one week just after the test set release.<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1738\" class=\"elementor-tab-title\" data-tab=\"8\" role=\"button\" aria-controls=\"elementor-tab-content-1738\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.8\u3000Are the schedules of the three subtasks the same? <\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1738\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"8\" role=\"region\" aria-labelledby=\"elementor-tab-title-1738\"><p>The schedules of the three subtasks are the same, excepting the ADE challenge in Subtask 3, which uses the results of NER (Subtasks 1 and 2) in the formal run. Therefore, the test set for the ADE challenge of Subtask 3 will release after the formal run of Subtasks 1 and 2. That is, the flow in the formal run is as follows: Release of the test set for Subtasks 1 and 2 -&gt; The formal run of Subtasks 1 and 2 (for a week) -&gt; Release of the test set for Subtask 3 -&gt; The formal run of Subtask 3 (for a week).\u00a0<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-toggle-item\">\n\t\t\t\t\t<div id=\"elementor-tab-title-1739\" class=\"elementor-tab-title\" data-tab=\"9\" role=\"button\" aria-controls=\"elementor-tab-content-1739\" aria-expanded=\"false\">\n\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon elementor-toggle-icon-left\" aria-hidden=\"true\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-closed\"><i class=\"fas fa-caret-right\"><\/i><\/span>\n\t\t\t\t\t\t\t\t<span class=\"elementor-toggle-icon-opened\"><i class=\"elementor-toggle-icon-opened fas fa-caret-up\"><\/i><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t<a class=\"elementor-toggle-title\" tabindex=\"0\">Q.9\u3000Do all types of entities need to be recognized in subtasks 1 and 2?<\/a>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t<div id=\"elementor-tab-content-1739\" class=\"elementor-tab-content elementor-clearfix\" data-tab=\"9\" role=\"region\" aria-labelledby=\"elementor-tab-title-1739\"><p>No, we plan to evaluate the following tag sets for Subtasks 1 and 2:<\/p>\n<p><br \/>CR: &lt;d&gt;, &lt;a&gt;, &lt;timex3&gt;, &lt;t-test&gt;, &lt;t-key&gt;, &lt;t-val&gt;, &lt;m-key&gt;, and &lt;m-val&gt;<br \/>RR: &lt;d&gt;, &lt;a&gt;, &lt;timex3&gt;, and &lt;t-test&gt;<\/p><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dabc8c8 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"dabc8c8\" data-element_type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d5dede0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d5dede0\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ceb8564\" data-id=\"ceb8564\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8e18093 elementor-widget elementor-widget-heading\" data-id=\"8e18093\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Organizer<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f9035a5 elementor-widget elementor-widget-text-editor\" data-id=\"f9035a5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n<li>Eiji Aramaki (Nara Institute of Science and Technology, Japan)<\/li>\n<li>Shoko Wakamiya (Nara Institute of Science and Technology, Japan)<\/li>\n<li>Shuntaro Yada (Nara Institute of Science and Technology, Japan)<\/li>\n<li>Yuta Nakamura (The University of Tokyo, Japan)<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-289d5c6 elementor-widget elementor-widget-heading\" data-id=\"289d5c6\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Collaborators<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-653dab0 elementor-widget elementor-widget-image-gallery\" data-id=\"653dab0\" data-element_type=\"widget\" data-widget_type=\"image-gallery.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-image-gallery\">\n\t\t\t<div id='gallery-1' class='gallery galleryid-84 gallery-columns-4 gallery-size-medium_large'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a data-elementor-open-lightbox=\"yes\" data-elementor-lightbox-slideshow=\"653dab0\" data-elementor-lightbox-title=\"sociocom_logo\" data-e-action-hash=\"#elementor-action%3Aaction%3Dlightbox%26settings%3DeyJpZCI6NTI1LCJ1cmwiOiJodHRwczpcL1wvc29jaW9jb20ubmFpc3QuanBcL3JlYWwtbWVkbmxwXC93cC1jb250ZW50XC91cGxvYWRzXC9zaXRlc1wvM1wvMjAyMVwvMDdcL3NvY2lvY29tX2xvZ28tMS5wbmciLCJzbGlkZXNob3ciOiI2NTNkYWIwIn0%3D\" href='https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1.png'><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"596\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1-768x596.png\" class=\"attachment-medium_large size-medium_large\" alt=\"\" aria-describedby=\"gallery-1-525\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1-768x596.png 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1-300x233.png 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1-1024x795.png 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/sociocom_logo-1.png 1136w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/><\/a>\n\t\t\t<\/div>\n\t\t\t\t<figcaption class='wp-caption-text gallery-caption' id='gallery-1-525'>\n\t\t\t\tSocial Computing Laboratory, NAIST\n\t\t\t\t<\/figcaption><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a data-elementor-open-lightbox=\"yes\" data-elementor-lightbox-slideshow=\"653dab0\" data-elementor-lightbox-title=\"\u30ef\u30a4\u30ba\u30ea\u30fc\u30c7\u30a3\u30f3\u30b0 \u30ed\u30b4\u30de\u30fc\u30af\" data-e-action-hash=\"#elementor-action%3Aaction%3Dlightbox%26settings%3DeyJpZCI6NTE2LCJ1cmwiOiJodHRwczpcL1wvc29jaW9jb20ubmFpc3QuanBcL3JlYWwtbWVkbmxwXC93cC1jb250ZW50XC91cGxvYWRzXC9zaXRlc1wvM1wvMjAyMVwvMDdcLzM3Y2U4OTdjYmZmZGFiOTk4NDRjOTljNDE5MjZiNDE3LXNjYWxlZC5qcGciLCJzbGlkZXNob3ciOiI2NTNkYWIwIn0%3D\" href='https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-scaled.jpg'><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"599\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-768x599.jpg\" class=\"attachment-medium_large size-medium_large\" alt=\"\" aria-describedby=\"gallery-1-516\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-768x599.jpg 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-300x234.jpg 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-1024x798.jpg 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-1536x1198.jpg 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/07\/37ce897cbffdab99844c99c41926b417-2048x1597.jpg 2048w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/><\/a>\n\t\t\t<\/div>\n\t\t\t\t<figcaption class='wp-caption-text gallery-caption' id='gallery-1-516'>\n\t\t\t\tY&#8217;s READING inc.\n\t\t\t\t<\/figcaption><\/figure>\n\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-77bd3df elementor-widget elementor-widget-heading\" data-id=\"77bd3df\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Acknowledgements<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-69020f9 elementor-widget elementor-widget-text-editor\" data-id=\"69020f9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>MedTxt-RR Track and annotation were supported by JST AIP-PRISM, Japan. MedTxt-CR Track\u00a0was supported by KEEPHA project of JST, AIP Trilateral AI Research, Grant Number JPMJCR20G9, Japan.<span class=\"Apple-converted-space\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7c59e22 elementor-widget elementor-widget-heading\" data-id=\"7c59e22\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Call for Sponsors<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-97a3315 elementor-widget elementor-widget-text-editor\" data-id=\"97a3315\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Real-MedNLP is looking for sponsors. If you are interested in sponsoring Real-MedNLP, please contact us by e-mail.<br \/>Sponsors are also welcome to participate in the shared task.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-53261b0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"53261b0\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-903d84a\" data-id=\"903d84a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4a9ef91 elementor-widget elementor-widget-heading\" data-id=\"4a9ef91\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Inquiry<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3585dfd elementor-icon-list--layout-traditional elementor-list-item-link-full_width elementor-widget elementor-widget-icon-list\" data-id=\"3585dfd\" data-element_type=\"widget\" data-widget_type=\"icon-list.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<ul class=\"elementor-icon-list-items\">\n\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item\">\n\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-icon\">\n\t\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"fas fa-mail-bulk\"><\/i>\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text\">Real-MedNLP office email: real-mednlp[at]is.naist.jp<\/span>\n\t\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t\t\t<\/ul>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f52f13d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f52f13d\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f89bd27\" data-id=\"f89bd27\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-69e2fec elementor-widget elementor-widget-html\" data-id=\"69e2fec\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\n<div class=\"wp-block-cover alignfull has-nv-dark-bg-background-color has-background-dim\" style=\"min-height: 400px;aspect-ratio:unset;\"><img loading=\"lazy\" decoding=\"async\" width=\"2560\" height=\"1707\" class=\"wp-block-cover__image-background wp-image-243\" src=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-scaled.jpg\" alt=\"\" data-object-fit=\"cover\" srcset=\"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-scaled.jpg 2560w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-300x200.jpg 300w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-1024x683.jpg 1024w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-768x512.jpg 768w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-1536x1024.jpg 1536w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-2048x1365.jpg 2048w, https:\/\/sociocom.naist.jp\/real-mednlp\/wp-content\/uploads\/sites\/3\/2021\/06\/christopher-burns-Kj2SaNHG-hg-unsplash-2-930x620.jpg 930w\" sizes=\"(max-width: 2560px) 100vw, 2560px\" \/>\n<div class=\"wp-block-cover__inner-container is-layout-flow wp-block-cover-is-layout-flow\">\n<p class=\"has-text-align-center\" style=\"font-size: 25px\">\u00a0<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>NEWS About this workshop Real-MedNLP is a shared task workshop for medical language processing using actual medical documents (case reports and radiology reports). The goal of this task is to promote the development of practical systems that support various medical services. The Real-MedNLP task has two corpus-based tracks (MedTxt-CR Track and MedTxt-RR Track), each with three subtasks. The task overview paper is available here.\u00a0 Datasets Test Collection MedTxt-CR Corpus This dataset comprises a set of open-access case reports available at CiNii. Typically, open-access case reports are biased in the reporting of patients and diseases, due to differences in the policies various medical societies have towards open-access publications. To reduce such&hellip;&nbsp;<a href=\"https:\/\/sociocom.naist.jp\/real-mednlp\/\" rel=\"bookmark\">Read More &raquo;<span class=\"screen-reader-text\">Frontpage (English)<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"full-width","neve_meta_container":"","neve_meta_enable_content_width":"on","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"on","footnotes":""},"_links":{"self":[{"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/pages\/84"}],"collection":[{"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/comments?post=84"}],"version-history":[{"count":283,"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/pages\/84\/revisions"}],"predecessor-version":[{"id":1008,"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/pages\/84\/revisions\/1008"}],"wp:attachment":[{"href":"https:\/\/sociocom.naist.jp\/real-mednlp\/wp-json\/wp\/v2\/media?parent=84"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}