Experimental design and data analysis in receiver operating characteristic studies: lessons learned from reports in radiology from 1997 to 2006.

Abstract:

PURPOSE:To provide a broad perspective concerning the recent use of receiver operating characteristic (ROC) analysis in medical imaging by reviewing ROC studies published in Radiology between 1997 and 2006 for experimental design, imaging modality, medical condition, and ROC paradigm. MATERIALS AND METHODS:Two hundred ninety-five studies were obtained by conducting a literature search with PubMed with two criteria: publication in Radiology between 1997 and 2006 and occurrence of the phrase "receiver operating characteristic." Studies returned by the query that were not diagnostic imaging procedure performance evaluations were excluded. Characteristics of the remaining studies were tabulated. RESULTS:Two hundred thirty-three (79.0%) of the 295 studies reported findings based on observers' diagnostic judgments or objective measurements. Forty-three (14.6%) did not include human observers, with most of these reporting an evaluation of a computer-aided diagnosis system or functional data obtained with computed tomography (CT) or magnetic resonance (MR) imaging. The remaining 19 (6.4%) studies were classified as reviews or meta-analyses and were excluded from our subsequent analysis. Among the various imaging modalities, MR imaging (46.0%) and CT (25.7%) were investigated most frequently. Approximately 60% (144 of 233) of ROC studies with human observers published in Radiology included three or fewer observers. CONCLUSION:ROC analysis is widely used in radiologic research, confirming its fundamental role in assessing diagnostic performance. However, the ROC studies reported in Radiology were not always adequate to support clear and clinically relevant conclusions.

journal_name

Radiology

journal_title

Radiology

authors

Shiraishi J,Pesce LL,Metz CE,Doi K

doi

10.1148/radiol.2533081632

subject

Has Abstract

pub_date

2009-12-01 00:00:00

pages

822-30

issue

3

eissn

0033-8419

issn

1527-1315

pii

radiol.2533081632

journal_volume

253

pub_type

杂志文章
  • Optical imaging of cancer heterogeneity with multispectral optoacoustic tomography.

    abstract:PURPOSE:To investigate whether multispectral optoacoustic tomography (MSOT) can reveal the heterogeneous distributions of exogenous agents of interest and vascular characteristics through tumors of several millimeters in diameter in vivo. MATERIALS AND METHODS:Procedures involving animals were approved by the governme...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.11111646

    authors: Herzog E,Taruttis A,Beziere N,Lutich AA,Razansky D,Ntziachristos V

    更新日期:2012-05-01 00:00:00

  • The influence of hypoxia and acidity on the hyperthermic response of malignant cells in vitro.

    abstract::Colony formation of JB-1-E tumor cells was studied after hyperthermic treatment (42.5 degrees C) at a pH of 6.4 or 7.2 under hypoxic and euoxic conditions. At a pH of 7.2 and normal oxygen tension, there was a moderate decrease in colony formation with increasing duration of hyperthermic treatment (To = 65 min.). This...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/123.2.511

    authors: Overgaard J,Bichel P

    更新日期:1977-05-01 00:00:00

  • Porous-coated anatomic total hip prostheses: radiographic analysis and clinical correlation.

    abstract::The radiographic changes of 46 bone-ingrowth, porous-coated anatomic total hip prostheses were reviewed and compared to the clinical outcome. The average postoperative follow-up was 32 months. The radiographic features that were assessed included subsidence; periprosthetic lucencies; periosteal, endosteal, and heterot...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.174.3.2305070

    authors: Kattapuram SV,Lodwick GS,Chandler H,Khurana JS,Ehara S,Rosenthal DI

    更新日期:1990-03-01 00:00:00

  • Gray-scale ultrasound detection of carotid body tumors. Report of 2 cases.

    abstract::Ultrasonography may be useful as a screening test for carotid body tumors before arteriography. Such tumors are seen as solid, well-circumscribed, weakly echogenic masses located at the carotid bifurcation. ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/132.2.409

    authors: Gooding GA

    更新日期:1979-08-01 00:00:00

  • Virtual autopsy: preliminary experience in high-velocity gunshot wound victims.

    abstract:PURPOSE:To retrospectively assess virtual autopsy performed with multidetector computed tomography (CT) for the forensic evaluation of gunshot wound victims. MATERIALS AND METHODS:The institutional review board approved this HIPAA-compliant study and did not require informed consent of the next of kin. Thirteen consec...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2402050972

    authors: Levy AD,Abbott RM,Mallak CT,Getz JM,Harcke HT,Champion HR,Pearse LA

    更新日期:2006-08-01 00:00:00

  • Exercise radionuclide ventriculography: practical considerations and sensitivity of coronary artery disease detection.

    abstract::Eighty-nine patients were evaluated for coronary artery disease (CAD) with exercise radionuclide ventriculography (ERV) and contrast coronary angiography. In 70 patients with documented lesions the ERV was abnormal in 65 for a sensitivity of 93%. In patients with normal coronary arteries, the ERV was abnormal in none ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/132.3.697

    authors: Brady TJ,Thrall JH,Clare JM,Rogers WL,Lo K,Pitt B

    更新日期:1979-09-01 00:00:00

  • Comparison of vertebral and peripheral mineral losses in disuse osteoporosis in monkeys.

    abstract::Vertebral mineral loss (measured by computed tomography) was compared to peripheral mineral loss (measured by photon absorptiometry) in immobilized monkeys. Significantly greater loss was found in axial cancallous bone than in peripheral cortical bone. The accuracy of vertebral CT measurements in one animal was determ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.134.2.6766220

    authors: Cann CE,Genant HK,Young DR

    更新日期:1980-02-01 00:00:00

  • White matter anisotropy in childhood medulloblastoma survivors: association with neurotoxicity risk factors.

    abstract:PURPOSE:To prospectively evaluate the relationships between change in white matter (WM) anisotropy and (a) patient age at craniospinal irradiation (CSI), (b) CSI dose, and (c) time of magnetic resonance (MR) imaging since CSI and to determine the effect of these neurotoxicity risk factors on WM anisotropy in posttreatm...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2362041066

    authors: Khong PL,Leung LH,Chan GC,Kwong DL,Wong WH,Cao G,Ooi GC

    更新日期:2005-08-01 00:00:00

  • Adult respiratory distress syndrome in children.

    abstract::Clinical, radiological, and pathologic data for nine children with adult respiratory distress syndrome (ARDS) were reviewed. The children ranged in age from 7 months to 15 years (mean age, 7.4 yrs). Underlying diseases and precipitating events included sepsis, pneumonia, near drowning, aspiration pneumonia, central ne...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.157.1.4034980

    authors: Effmann EL,Merten DF,Kirks DR,Pratt PC,Spock A

    更新日期:1985-10-01 00:00:00

  • Quantification of articular cartilage in the knee with pulsed saturation transfer subtraction and fat-suppressed MR imaging: optimization and validation.

    abstract:PURPOSE:To assess the reproducibility and accuracy of volumetric quantifications of articular cartilage in the knee determined with three-dimensional (3D) magnetic resonance (MR) imaging combined with pulsed saturation transfer subtraction (STS) or T1-weighted fat suppression (FS) imaging. MATERIALS AND METHODS:Eight ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.192.2.8029420

    authors: Peterfy CG,van Dijke CF,Janzen DL,Glüer CC,Namba R,Majumdar S,Lang P,Genant HK

    更新日期:1994-08-01 00:00:00

  • Diffusion-tensor MR imaging and tractography: exploring brain microstructure and connectivity.

    abstract::Diffusion magnetic resonance (MR) imaging is evolving into a potent tool in the examination of the central nervous system. Although it is often used for the detection of acute ischemia, evaluation of directionality in a diffusion measurement can be useful in white matter, which demonstrates strong diffusion anisotropy...

    journal_title:Radiology

    pub_type: 杂志文章,评审

    doi:10.1148/radiol.2452060445

    authors: Nucifora PG,Verma R,Lee SK,Melhem ER

    更新日期:2007-11-01 00:00:00

  • Atypical small hemangiomas of the liver: "bright dot" sign at two-phase spiral CT.

    abstract:PURPOSE:To determine if a tiny enhancing dot is characteristic of small hemangiomas with low attenuation during the hepatic arterial phase (HAP) and portal venous phase (PVP) of two-phase spiral computed tomography (CT). MATERIALS AND METHODS:Among 249 consecutive patients with 377 hemangiomas who underwent two-phase ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.208.2.9680590

    authors: Jang HJ,Choi BI,Kim TK,Yun EJ,Kim KW,Han JK,Han MC

    更新日期:1998-08-01 00:00:00

  • Imaging the postoperative neck.

    abstract::The variety of cervical lymphadenectomy and neck reconstructive procedures that have been developed in the past 20 years has led to confusion about nomenclature in the literature. In addition, some of these operations result in gross alteration of the normal anatomy. The authors review the present classification syste...

    journal_title:Radiology

    pub_type: 杂志文章,评审

    doi:10.1148/radiology.187.3.8497600

    authors: Som PM,Urken ML,Biller H,Lidov M

    更新日期:1993-06-01 00:00:00

  • Work in progress: abnormal parathyroid glands. Comparison of nonselective arterial digital arteriography, selective parathyroid arteriography, and venous digital arteriography as methods of detection.

    abstract::Nonselective arterial digital arteriography (NSADA), selective parathyroid arteriography (SPA), and venous digital arteriography (VDA) were compared as methods of detecting parathyroid enlargement in 14 patients with primary hyperparathyroidism undergoing preoperative localization following unsuccessful neck surgery. ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.148.1.6134300

    authors: Krudy AG,Doppman JL,Miller DL,Marx SJ,Norton JA,Santora AC 2nd,Lasker RD,Downs RW,Spiegel AM,Aurbach GD

    更新日期:1983-07-01 00:00:00

  • Practical and technical considerations in establishing an intraoperative radiation therapy program in the community practice.

    abstract::The Radiation Oncology Center in Sacramento, California, has developed a procedure for establishing an intraoperative radiation therapy facility in a community practice. The logistics pertaining to personnel, equipment, physical measurements, and quality assurance are presented. Particular emphasis is given to the mos...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.168.1.3380970

    authors: Wolkov HB,Chenery SG,Asche DR,McPeek EC,Hanks GE,Holmes TW

    更新日期:1988-07-01 00:00:00

  • MR imaging of the dilated biliary tract.

    abstract::The magnetic resonance (MR) examinations of 18 patients with dilated bile ducts were reviewed retrospectively to determine the capability of MR to demonstrate biliary dilatation, assess MR appearance of the dilated biliary tract using spin-echo techniques, and define the optimal MR imaging parameters (repetition time ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.158.2.3941860

    authors: Dooms GC,Fisher MR,Higgins CB,Hricak H,Goldberg HI,Margulis AR

    更新日期:1986-02-01 00:00:00

  • MR imaging of blood vessels using three-dimensional reconstruction: methodology.

    abstract::Multisection, dual-echo magnetic resonance (MR) transaxial images of blood vessels contain both anatomic and qualitative information about flow. Even so, the images are produced as a series of two-dimensional tomographic sections from which full visualization of connected structures is difficult. A computer algorithm ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.157.3.4059560

    authors: Hale JD,Valk PE,Watts JC,Kaufman L,Crooks LE,Higgins CB,Deconinck F

    更新日期:1985-12-01 00:00:00

  • Pulmonary nodules: estimation of malignancy at thin-section helical CT--effect of computer-aided diagnosis on performance of radiologists.

    abstract:PURPOSE:To evaluate the effect of a computer-aided diagnosis (CAD) system on the diagnostic performance of radiologists for the estimation of the malignancy of pulmonary nodules on thin-section helical computed tomographic (CT) scans. MATERIALS AND METHODS:The institutional review board approved use of the CT database...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2383050167

    authors: Awai K,Murao K,Ozawa A,Nakayama Y,Nakaura T,Liu D,Kawanaka K,Funama Y,Morishita S,Yamashita Y

    更新日期:2006-04-01 00:00:00

  • Dutch iliac stent trial: long-term results in patients randomized for primary or selective stent placement.

    abstract:PURPOSE:To determine long-term results of the prospective Dutch Iliac Stent Trial. MATERIALS AND METHODS:The study protocol was approved by local institutional review boards. All patients gave written informed consent. Two hundred seventy-nine patients (201 men, 78 women; mean age, 58 years) with iliac artery disease ...

    journal_title:Radiology

    pub_type: 杂志文章,随机对照试验

    doi:10.1148/radiol.2382041053

    authors: Klein WM,van der Graaf Y,Seegers J,Spithoven JH,Buskens E,van Baal JG,Buth J,Moll FL,Overtoom TT,van Sambeek MR,Mali WP

    更新日期:2006-02-01 00:00:00

  • Detection of fat in a renal cell carcinoma mimicking angiomyolipoma.

    abstract::The authors report a case of renal cell carcinoma in which computed tomography (CT) revealed fatty tissue and calcification within a solid renal mass. The detection of a mass with negative CT numbers at CT and increased echogenicity at ultrasound led to the incorrect diagnosis of angiomyolipoma (renal hamartoma). The ...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.188.2.8327690

    authors: Strotzer M,Lehner KB,Becker K

    更新日期:1993-08-01 00:00:00

  • Thyroid Imaging Reporting and Data System (TI-RADS): A User's Guide.

    abstract::In 2017, the Thyroid Imaging Reporting and Data System (TI-RADS) Committee of the American College of Radiology (ACR) published a white paper that presented a new risk-stratification system for classifying thyroid nodules on the basis of their appearance at ultrasonography (US). In ACR TI-RADS, points in five feature ...

    journal_title:Radiology

    pub_type: 杂志文章,评审

    doi:10.1148/radiol.2017171240

    authors: Tessler FN,Middleton WD,Grant EG

    更新日期:2018-04-01 00:00:00

  • The air-contrast stomach examination: an overview.

    abstract::An overview of current air-contrast stomach examination techniques is offered: (a) Hampton's technique which does not use distension; (b) the Japanese method which utilizes the relaxant drug Buscopan, good distension, and much patient rotation; (c) a third method in which no drugs are used but more distension is obtai...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/117.3.743

    authors: Miller RE

    更新日期:1975-12-01 00:00:00

  • Pelvimetry by digital radiography: a low-dose examination.

    abstract::Digital radiographs obtained with a CT scanner were evaluated as an alternative to conventional pelvimetry. Dosimetry studies demonstrated an absorbed radiation dose of 22 mrad for each digital radiograph, approximately 3% of the dose for comparable standard radiographs. Distance measurements were accurately made on d...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.143.3.7079500

    authors: Federle MP,Cohen HA,Rosenwein MF,Brant-Zawadzki MN,Cann CE

    更新日期:1982-06-01 00:00:00

  • The accordion sign at CT: a nonspecific finding in patients with colonic edema.

    abstract:PURPOSE:To determine whether the "accordion sign" is a specific computed tomographic (CT) sign of Clostridium difficile colitis. MATERIALS AND METHODS:Fifty-seven patients with CT evidence of severe colitis, as judged by colonic wall thickening, an abnormal haustral pattern, the target sign, and stranding of the peric...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.211.3.r99jn32743

    authors: Macari M,Balthazar EJ,Megibow AJ

    更新日期:1999-06-01 00:00:00

  • Factors associated with academic radiology research productivity.

    abstract:PURPOSE:To determine factors associated with research productivity among all university radiology departments in the United States. MATERIALS AND METHODS:As an observational study, this was exempt from institutional review board approval. All 47,299 radiology articles from 1996 through 2003 that were indexed in the Na...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2373041508

    authors: Itagaki MW,Pile-Spellman J

    更新日期:2005-12-01 00:00:00

  • Fluoroscopic removal of ingested alkaline batteries.

    abstract::The ingestion of small, flat disc batteries by infants and children may be life threatening if the battery leaks or disintegrates resulting in perforation of the gastrointestinal tract. The fluoroscopic removal of six ingested batteries from the stomach using an orogastric tube and magnet and a Foley catheter is descr...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.150.2.6691120

    authors: Jaffe RB,Corneli HM

    更新日期:1984-02-01 00:00:00

  • Focal-spot measurement: comparison of slit, pinhole, and star resolution pattern techniques.

    abstract::The National Electrical Manufacturers Association (NEMA) has established a new standard for x-ray tube focal-spot measurements using a focal-spot camera with a slit instead of a pinhole. Measurements from the slit focal-spot camera are compared and star resolution pattern techniques for 50 focal spots. In addition, me...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.165.1.3628780

    authors: Everson JD,Gray JE

    更新日期:1987-10-01 00:00:00

  • Locally recurrent breast cancer: pulse dose rate brachytherapy for repeat irradiation following lumpectomy-- a second chance to preserve the breast.

    abstract:PURPOSE:To perform and assess the effectiveness of local excision of recurrent tumor followed by postoperative pulse dose rate (PDR) brachytherapy. MATERIALS AND METHODS:From 1994 to 2000, 17 patients who had small recurrent breast carcinomas after initially undergoing breast-conserving therapy (BCT), which included p...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2253011913

    authors: Resch A,Fellner C,Mock U,Handl-Zeller L,Biber E,Seitz W,Pötter R

    更新日期:2002-12-01 00:00:00

  • Gallbladder carcinoma: evaluation with MR imaging.

    abstract::Magnetic resonance (MR) images of 19 patients with histologically proved gallbladder carcinoma were retrospectively reviewed to determine the appearance of the primary tumor, and to assess the ability of MR imaging to demonstrate the various modes of tumor spread beyond the gallbladder. The primary tumor, as well as t...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiology.174.1.2294540

    authors: Sagoh T,Itoh K,Togashi K,Shibata T,Minami S,Noma S,Yamashita K,Nishimura K,Asato R,Mori K

    更新日期:1990-01-01 00:00:00

  • T2 relaxation time of cartilage at MR imaging: comparison with severity of knee osteoarthritis.

    abstract:PURPOSE:To evaluate differences in T2 values in femoral and tibial cartilage at magnetic resonance (MR) imaging in patients with varying degrees of osteoarthritis (OA) compared with healthy subjects and to develop a mapping and display method based on calculation of T2 z scores for visual grading and assessment of cart...

    journal_title:Radiology

    pub_type: 杂志文章

    doi:10.1148/radiol.2322030976

    authors: Dunn TC,Lu Y,Jin H,Ries MD,Majumdar S

    更新日期:2004-08-01 00:00:00