A Novel Fundus Image Reading Tool for Efficient Generation of a Multi-dimensional Categorical Image Database for Machine Learning Algorithm Training.

Abstract:

Background:We described a novel multi-step retinal fundus image reading system for providing high-quality large data for machine learning algorithms, and assessed the grader variability in the large-scale dataset generated with this system. Methods:A 5-step retinal fundus image reading tool was developed that rates image quality, presence of abnormality, findings with location information, diagnoses, and clinical significance. Each image was evaluated by 3 different graders. Agreements among graders for each decision were evaluated. Results:The 234,242 readings of 79,458 images were collected from 55 licensed ophthalmologists during 6 months. The 34,364 images were graded as abnormal by at-least one rater. Of these, all three raters agreed in 46.6% in abnormality, while 69.9% of the images were rated as abnormal by two or more raters. Agreement rate of at-least two raters on a certain finding was 26.7%-65.2%, and complete agreement rate of all-three raters was 5.7%-43.3%. As for diagnoses, agreement of at-least two raters was 35.6%-65.6%, and complete agreement rate was 11.0%-40.0%. Agreement of findings and diagnoses were higher when restricted to images with prior complete agreement on abnormality. Retinal/glaucoma specialists showed higher agreements on findings and diagnoses of their corresponding subspecialties. Conclusion:This novel reading tool for retinal fundus images generated a large-scale dataset with high level of information, which can be utilized in future development of machine learning-based algorithms for automated identification of abnormal conditions and clinical decision supporting system. These results emphasize the importance of addressing grader variability in algorithm developments.

journal_name

J Korean Med Sci

authors

Park SJ,Shin JY,Kim S,Son J,Jung KH,Park KH

doi

10.3346/jkms.2018.33.e239

subject

Has Abstract

pub_date

2018-08-08 00:00:00

pages

e239

issue

43

eissn

1011-8934

issn

1598-6357

journal_volume

33

pub_type

杂志文章
  • Influenza Vaccination Coverage and Its Associated Factors among North Korean Defectors Living in the Republic of Korea.

    abstract::This study aimed to examine influenza vaccination coverage of North Korean defectors (NKD) in the Republic of Korea (Korea) and explore the factors affected the vaccination coverage. Total 378 NKD were analyzed. Four Korean control subjects were randomly matched by age and gender from the Korea National Health and Nut...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2015.30.9.1226

    authors: Song IG,Lee H,Yi J,Kim MS,Park SM

    更新日期:2015-09-01 00:00:00

  • Relationship between chronic kidney disease and risk of coronary heart disease in Korean men.

    abstract::There have been many epidemiological researches of chronic kidney disease (CKD), accompanied by an increase in the incidence of coronary heart disease (CHD). However, as far as we know, little research has been done to examine the extent of the relationship between CKD and CHD as estimated by Framingham risk score (FR...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2011.26.6.753

    authors: Ryoo JH,Kim SG,Suh BS,Kim DI,Park SK

    更新日期:2011-06-01 00:00:00

  • The Effect of Emergency Medical Service Use and Inter-hospital Transfer on Prehospital Delay among Ischemic Stroke Patients: A Multicenter Observational Study.

    abstract::The time between symptom onset and arrival at an emergency department (ED) (S2D) is a crucial time for optimal intravenous reperfusion care for ischemic stroke. We aimed to analyze the effect of emergency medical services (EMS) utilization and inter-hospital transfer on S2D in Korea. Ischemic stroke patients were pros...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2016.31.1.139

    authors: Park HA,Ahn KO,Shin SD,Cha WC,Ro YS

    更新日期:2016-01-01 00:00:00

  • Response-Guided Therapy for Hepatitis C Virus Recurrence Based on Early Protocol Biopsy after Liver Transplantation.

    abstract::Hepatitis C virus (HCV) recurrence after liver transplantation (LT) is universal and progressive. Here, we report recent results of response-guided therapy for HCV recurrence based on early protocol biopsy after LT. We reviewed patients who underwent LT for HCV related liver disease between 2010 and 2012. Protocol bio...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2015.30.11.1577

    authors: Kim H,Lee KW,Yi NJ,Lee HW,Choi Y,Suh SW,Jeong J,Suh KS

    更新日期:2015-11-01 00:00:00

  • Predictable ultrasonographic findings of early abortion.

    abstract::Early fetal growth delay and early oligohydramnios have been suspected as signs of embryonal jeopardy. However, little information is available for the prediction of early abortion. Sonographic examination of 111 early pregnancies between the sixth and ninth gestational week with regular, 28 day menstrual cycles was p...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.1992.7.1.34

    authors: Jun SA,Ahn MO,Lee YD,Cha KY

    更新日期:1992-03-01 00:00:00

  • Blood Pressure Curve for Children Less than 10 Years of Age: Findings from the Ewha Birth and Growth Cohort Study.

    abstract:BACKGROUND:Routine blood pressure (BP) measurement is recommended to begin at 3 years of age, but there are no national BP reference values for Korean children less than 7 years of age. Therefore, we developed sex-, age-, and height-specific BP reference values for non-overweight children aged 3-9 years. METHODS:We an...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2020.35.e91

    authors: Lee HA,Park B,Park EA,Cho SJ,Kim HS,Choi EJ,Kim NE,Park H

    更新日期:2020-03-30 00:00:00

  • Clinical presentations and neurodevelopmental outcomes of perinatal stroke in preterm and term neonates: a case series.

    abstract::Perinatal stroke in neonates can lead to disability in later life. However, its etiology and prognosis are poorly understood. The aim of this study was to describe clinical presentations and neurodevelopmental outcomes of our case series of perinatal stroke in Korea. Thirteen term and preterm neonates who were diagnos...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2010.25.6.888

    authors: Lee HJ,Lim BC,Hwang H,Hong JS,Kim EK,Kim HS,Kim BI,Choi JH,Choi CW

    更新日期:2010-06-01 00:00:00

  • National Trends of Meniscectomy and Meniscus Repair in Korea.

    abstract:BACKGROUND:Meniscus surgeries are frequently performed in orthopaedics. However, their current status is not well known in many countries, including Korea. This study aimed to investigate the national trends of meniscus surgery in Korea. METHODS:Information from the national database was acquired through the Korean He...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2019.34.e206

    authors: Chung KS,Ha JK,Kim YS,Kim JH,Ra HJ,Kong DH,Wang PW,Choi CH,Kim JG

    更新日期:2019-08-19 00:00:00

  • Histiocyte-rich reactive lymphoid hyperplasia of the liver: unusual morphologic features.

    abstract::Reactive lymphoid hyperplasia (RLH) of the liver is a rare entity and has also been termed nodular lymphoid lesion or pseudolymphoma of the liver. We report a case of hepatic RLH exhibiting unusual histiocyte-rich histologic features in a 47-yr-old woman in conjunction with a renal cell carcinoma. A follow-up computed...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章,评审

    doi:10.3346/jkms.2008.23.1.156

    authors: Park HS,Jang KY,Kim YK,Cho BH,Moon WS

    更新日期:2008-02-01 00:00:00

  • Randomized Comparison of Actual and Ideal Body Weight for Size Selection of the Laryngeal Mask Airway Classic in Overweight Patients.

    abstract::Size selection of the laryngeal mask airway (LMA) Classic based on actual body weight remains a common practice. However, ideal body weight might allow for a better size selection in obese patients. The purpose of our study was to compare the utility of ideal body weight and actual body weight when choosing the approp...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章,随机对照试验

    doi:10.3346/jkms.2015.30.8.1197

    authors: Kim MS,Lee JS,Nam SB,Kang HJ,Kim JE

    更新日期:2015-08-01 00:00:00

  • Prognostic factors of the long-term survival after transjugular intrahepatic portosystemic shunt in the treatment of gastric and esophageal variceal bleeding.

    abstract::Transjugular intrahepatic portosystemic shunt (TIPSS) is a promising method of treatment for gastric or esophageal variceal bleeding. This study was performed to determine the prognostic factors contributing to the survival of patients after TIPSS for gastric or esophageal variceal bleeding. One hundred and fifty-five...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2002.17.6.772

    authors: Kim YT,Jung MK,Cho CM,Tak WY,Kweon YO,Kim SK,Choi YH,Chung JM

    更新日期:2002-12-01 00:00:00

  • LPS-induced migration of peritoneal B-1 cells is associated with upregulation of CXCR4 and increased migratory sensitivity to CXCL12.

    abstract::B-1 cells, which constitute a predominant lymphocyte subset in serosal cavities and produce most of natural antibodies, are subdivided into the CD5(+) B-1a and CD5(-) B-1b cell subpopulations, but the differential roles of B-1a and B-1b cells are not well understood. We report that B-1a cells preferentially migrate ou...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2012.27.1.27

    authors: Moon H,Lee JG,Shin SH,Kim TJ

    更新日期:2012-01-01 00:00:00

  • Percutaneous papillary large balloon dilation during percutaneous cholangioscopic lithotripsy for the treatment of large bile-duct stones: a feasibility study.

    abstract::When access to a major duodenal papilla or endoscopic retrograde cholangiopancreatography has failed, percutaneous transhepatic cholangioscopic lithotripsy (PTCS-L) may be useful for removing common bile duct (CBD) stones. However, the feasibility and usefulness of percutaneous transhepatic papillary large-balloon dil...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2015.30.3.278

    authors: Han JY,Jeong S,Lee DH

    更新日期:2015-03-01 00:00:00

  • Removal of alpha-Gal epitopes from porcine aortic valve and pericardium using recombinant human alpha galactosidase A.

    abstract::It has been reported that the immune response due to alpha-Gal epitopes is an important factor in tissue valve failure. The elimination of the interaction between the natural anti-Gal antibodies and alpha-gal epitopes on the xenografts is a prerequisite to the success of xenografts in humans. Previously, we reported t...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2009.24.6.1126

    authors: Park S,Kim WH,Choi SY,Kim YJ

    更新日期:2009-12-01 00:00:00

  • Process of Obtaining Social Consensus and 3-Year Functional Outcomes of the First Hand Allotransplantation in Korea.

    abstract:BACKGROUND:On February 2, 2017, the surgical team of ten board-certified hand specialists of W Hospital in Korea successfully performed the nation's first hand transplantation at Yeungnam University Medical Center (YUMC). This paper reports on the legal, financial, and cultural hurdles that were overcome to open the wa...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2021.36.e6

    authors: Woo SH,Kim YW,Cheon HJ,Jang S,Do JY,Choi JH,Kang DH,Nam HJ,Yoo MJ,Lee YS,Choi JH

    更新日期:2021-01-11 00:00:00

  • Predictive value of Kushida index and acoustic pharyngometry for the evaluation of upper airway in subjects with or without obstructive sleep apnea.

    abstract::Acoustic pharyngometry is a relatively new noninvasive method that quantifies geometrically complexed pharyngeal dimensions. Our study aimed to investigate the predictability and usefulness of acoustic pharyngometry in diagnosis of obstructive sleep apnea (OSA), and we developed a prospective clinical trial in 16 subj...

    journal_title:Journal of Korean medical science

    pub_type: 临床试验,杂志文章

    doi:10.3346/jkms.2004.19.5.662

    authors: Jung DG,Cho HY,Grunstein RR,Yee B

    更新日期:2004-10-01 00:00:00

  • Human Urine-derived Stem Cells Seeded Surface Modified Composite Scaffold Grafts for Bladder Reconstruction in a Rat Model.

    abstract::We conducted this study to investigate the synergistic effect of human urine-derived stem cells (USCs) and surface modified composite scaffold for bladder reconstruction in a rat model. The composite scaffold (Polycaprolactone/Pluronic F127/3 wt% bladder submucosa matrix) was fabricated using an immersion precipitatio...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2015.30.12.1754

    authors: Lee JN,Chun SY,Lee HJ,Jang YJ,Choi SH,Kim DH,Oh SH,Song PH,Lee JH,Kim JK,Kwon TG

    更新日期:2015-12-01 00:00:00

  • The effect of probiotics for preventing radiation-induced morphological changes in intestinal mucosa of rats.

    abstract::Radiation therapy is an important treatment modality for abdominal or pelvic cancer, but there is a common and serious complication such as radiation-induced enteritis. Probiotics is reported to have positive effects against radiation-induced enteropathy. In this study, morphological changes of bowel mucosa were analy...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2014.29.10.1372

    authors: Ki Y,Kim W,Cho H,Ahn K,Choi Y,Kim D

    更新日期:2014-10-01 00:00:00

  • Giant cystic chondroid hamartoma.

    abstract::We report a case of huge pulmonary chondroid hamartoma with multilocular cysts in a 38-yr-old male patient. The lobectomy specimen of the left lower lobe showed a large multilocular cystic mass, 11.5 x 10 cm in size. The mass had thin-walled, variable-sized cysts and areas of solid cartilaginous small nodules in the i...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2005.20.3.509

    authors: Kim GY,Han J,Kim DH,Kim J,Lee KS

    更新日期:2005-06-01 00:00:00

  • Tenosynovial giant cell tumor of finger, localized type: a case report.

    abstract::The authors report a typical case of tenosynovial giant cell tumor of the right middle finger of a 31-year-old man. Histologically, this tumor is characterized by a discrete proliferation of rounded synovial-like cells accompanied by a variable number of multinucleated giant cells, inflammatory cells, and xanthoma cel...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.1999.14.6.682

    authors: Hwang K,Lee DK,Lee SI

    更新日期:1999-12-01 00:00:00

  • Differential pattern of perivascular type IV collagen deposits in phyllodes tumors of the breast.

    abstract::Deposition of basement membrane extracellular matrix is influenced by adjacent tumor cells, and in some cases, the pattern of type IV collagen deposit is characteristic in malignant tumors. In this report, we analyzed the difference in type IV collagen deposition patterns between benign and malignant phyllodes tumors ...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.1992.7.4.360

    authors: Kim WH,Kim CW,Noh DY,Kim YI

    更新日期:1992-12-01 00:00:00

  • Reliability and validity of the Korean version of the internet addiction test among college students.

    abstract::We developed a Korean translation of the Internet Addiction Test (KIAT), widely used self-report for internet addiction and tested its reliability and validity in a sample of college students. Two hundred seventy-nine college students at a national university completed the KIAT. Internal consistency and two week test-...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2013.28.5.763

    authors: Lee K,Lee HK,Gyeong H,Yu B,Song YM,Kim D

    更新日期:2013-05-01 00:00:00

  • Validity of the Korean Version of the Face, Legs, Activity, Cry, and Consolability Scale for Assessment of Pain in Dementia Patients.

    abstract::Pain is often associated with a more rapid progression of cognitive and functional decline, and behavioral disturbance in dementia. Therefore, it is essential to accurately assesses pain for proper intervention in patients with dementia. The Face, Legs, Activity, Cry, and Consolability (FLACC) scale is an excellent be...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2017.32.11.1852

    authors: Moon Y,Kim YS,Lee J,Han SH

    更新日期:2017-11-01 00:00:00

  • Congenital insensitivity to pain with anhidrosis: a case report.

    abstract::Congenital insensitivity to pain with anhidrosis (CIPA) is a very rare genetic disorder of the peripheral nervous system characterized by recurrent episodes of unexplained fever, generalized anhidrosis, insensitivity to pain and temperature, and accompanied by self-mutilating behavior and mental retardation. We report...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.1999.14.4.460

    authors: Kim JS,Woo YJ,Kim GM,Kim CJ,Ma JS,Hwang TJ,Lee MC

    更新日期:1999-08-01 00:00:00

  • A case of renal crisis in a Korean scleroderma patient with anti-RNA polymerase I and III antibodies.

    abstract::Scleroderma (SSc) renal crisis has been reported to be associated with anti-RNA polymerase I and III (RNAP I/III) antibodies in Caucasians and the Japanese. However, no report is available for Korean SSc patients. Here, we describe the case of a 65-yr-old female SSc patient who developed renal crisis and whose serum c...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2006.21.6.1121

    authors: Kang EH,Im CH,Kim SH,Chung JR,Lee EY,Kim DJ,Lee EB,Song YW

    更新日期:2006-12-01 00:00:00

  • Association between Plasma Pentraxin 3 Levels and Bone Mineral Density in Elderly Koreans: the Dong-gu Study.

    abstract:Background:We investigated the association between pentraxin 3 (PTX3), a novel inflammatory marker, and bone mineral density (BMD) in the general Korean population. Methods:We selected a sub-cohort of 1,440 subjects (757 men and 683 women) from participants in the community-based Dong-gu Study. The mean age was 66.0 ±...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2018.33.e165

    authors: Lee R,Ahn HR,Shin MH,Kim HN,Lee YH,Choi SW,Kweon SS

    更新日期:2018-05-09 00:00:00

  • Susceptibility of the Index Urinary Tract Infection to Prophylactic Antibiotics Is a Predictive Factor of Breakthrough Urinary Tract Infection in Children with Primary Vesicoureteral Reflux Receiving Continuous Antibiotic Prophylaxis.

    abstract:BACKGROUND:Few studies have reported on breakthrough urinary tract infection (UTI) associated with the susceptibility of index UTI to prophylactic antibiotics in children with primary vesicoureteral reflux (VUR) receiving continuous antibiotic prophylaxis (CAP). We assessed the impact of the susceptibility of index UTI...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2019.34.e156

    authors: Lee JN,Byeon KH,Woo MJ,Baek HS,Cho MH,Jeong SY,Lee SM,Ham JY,Ha YS,Kim HT,Yoo ES,Kwon TG,Chung SK

    更新日期:2019-06-02 00:00:00

  • Compound nerve action potential of common peroneal nerve and sural nerve action potential in common peroneal neuropathy.

    abstract::To enhance the accuracy for determining the precise localization, the findings of the compound nerve action potentials (CNAPs) of the common peroneal nerve (CPN) were investigated in patients with common peroneal mononeuropathy (CPM) in the knee, and the sural sensory nerve action potentials (SNAPs) were also analyzed...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2008.23.1.117

    authors: Kwon HK,Kim L,Park YK

    更新日期:2008-02-01 00:00:00

  • Use of nafamostat mesilate as an anticoagulant during extracorporeal membrane oxygenation.

    abstract::Although the incidence of bleeding complications during extracorporeal membrane oxygenator (ECMO) support has decreased in various trials, bleeding is still the most fatal complication. We investigated the ideal dosage and efficacy of nafamostat mesilate for use with ECMO in patients with acute cardiac or respiratory ...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2011.26.7.945

    authors: Han SJ,Kim HS,Kim KI,Whang SM,Hong KS,Lee WK,Lee SH

    更新日期:2011-07-01 00:00:00

  • Laparoscopic-assisted resection of ileal lipoma causing ileo-ileo-colic intussusception.

    abstract::Adult intussusception is rare, and the majority of cases has an underlying cause that requires surgical resection. We report a case of a 39 yr-old man with ileo-ileo-colic intussusception caused by ileal lipoma that was successfully managed by a laparoscopic-assisted surgical maneuver. Using a three-cannula technique,...

    journal_title:Journal of Korean medical science

    pub_type: 杂志文章

    doi:10.3346/jkms.2001.16.1.119

    authors: Park KT,Kim SH,Song TJ,Moon HY

    更新日期:2001-02-01 00:00:00