Is an ROC-type response truly always better than a binary response in observer performance studies?

Abstract:

RATIONALE AND OBJECTIVES:The aim of this study was to assess similarities and differences between methods of performance comparisons under binary (yes or no) and receiver-operating characteristic (ROC)-type pseudocontinuous (0-100) rating data ascertained during an observer performance study of interpretation of full-field digital mammography (FFDM) versus FFDM plus digital breast tomosynthesis. MATERIALS AND METHODS:Rating data consisted of ROC-type pseudocontinuous and binary ratings generated by eight radiologists evaluating 77 digital mammographic examinations. Overall performance levels were summarized with a conventionally used probability of correct discrimination or, equivalently, the area under the ROC curve (AUC), which under a binary scale is related to Youden's index. Magnitudes of differences in the reader-averaged empirical AUCs between FFDM alone and FFDM plus digital breast tomosynthesis were compared in the context of fixed-reader and random-reader variability of the estimates. RESULTS:The absolute differences between modes using the empirical AUCs were larger on average for the binary scale (0.12 vs 0.07) and for the majority of individual readers (six of eight). Standardized differences were consistent with this finding (2.32 vs 1.63 on average). Reader-averaged differences in AUCs standardized by fixed-reader and random-reader variances were also smaller under the binary rating paradigm. The discrepancy between AUC differences depended on the location of the reader-specific binary operating points. CONCLUSIONS:The human observer's operating point should be a primary consideration in designing an observer performance study. Although in general, the ROC-type rating paradigm provides more detailed information on the characteristics of different modes, it does not reflect the actual operating point adopted by human observers. There are application-driven scenarios in which analysis based on binary responses may provide statistical advantages.

journal_name

Acad Radiol

journal_title

Academic radiology

authors

Gur D,Bandos AI,Rockette HE,Zuley ML,Hakim CM,Chough DM,Ganott MA,Sumkin JH

doi

10.1016/j.acra.2009.12.012

subject

Has Abstract

pub_date

2010-05-01 00:00:00

pages

639-45

issue

5

eissn

1076-6332

issn

1878-4046

pii

S1076-6332(10)00007-3

journal_volume

17

pub_type

杂志文章
  • Evaluation of the quality of radiology requisitions for intensive care unit patients.

    abstract:RATIONALE AND OBJECTIVES:The study aim is to evaluate the quality of radiology requisitions for plain film radiographs on intensive care unit (ICU) patients. MATERIALS AND METHODS:Radiology requisitions for 58 patients in ICU units at our children's hospital and the corresponding original orders for the study, written...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2005.10.017

    authors: Cohen MD,Curtin S,Lee R

    更新日期:2006-02-01 00:00:00

  • Estimation of volumes of distribution and intratumoral ethanol concentrations by computed tomography scanning after percutaneous ethanol injection.

    abstract:RATIONALE AND OBJECTIVES:We developed a technique for estimating the volumes of distribution and intratumoral ethanol concentrations using computed tomography (CT) scanning in patients undergoing percutaneous ethanol injection (PEI) treatment of malignant hepatic tumors. METHODS:A phantom containing anhydrous ethanol ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/S1076-6332(96)80333-3

    authors: Alexander DG,Unger EC,Seeger SJ,Karmann S,Krupinski EA

    更新日期:1996-01-01 00:00:00

  • Evaluation of hardware-related geometrical distortion in structural MRI at 7 Tesla for image-guided applications in neurosurgery.

    abstract:RATIONALE AND OBJECTIVES:Geometrical distortion is a well-known problem in structural magnetic resonance imaging (MRI), leading to pixel shifts with variations up to several millimeters. Because the main factors of geometrical distortion are proportional to B(0), MRI spatial encoding distortions tend to increase with h...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2011.02.011

    authors: Dammann P,Kraff O,Wrede KH,Özkan N,Orzada S,Mueller OM,Sandalcioglu IE,Sure U,Gizewski ER,Ladd ME,Gasser T

    更新日期:2011-07-01 00:00:00

  • Clinical evaluation of wavelet-compressed digitized screen-film mammography.

    abstract:RATIONALE AND OBJECTIVES:The authors compared diagnostic accuracy and callback rates with conventional screen-film mammograms and wavelet-compressed digitized images. MATERIALS AND METHODS:Sixty sets of mammograms (four views per case) were digitized at a spatial resolution of 100 microm. The images were wavelet compr...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(00)80004-5

    authors: Powell KA,Mallasch PG,Obuchowski NA,Kerczewski RJ,Ganobcik SN,Cardenosa G,Chilcote W

    更新日期:2000-05-01 00:00:00

  • Lung nodule CAD software as a second reader: a multicenter study.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this multicenter, multireader study was to evaluate the performance of computed tomography (CT) lung nodule computer-aided detection (CAD) software as a second reader. METHODS AND MATERIALS:The study involved 109 patients from four sites. The data were collected from a variety o...

    journal_title:Academic radiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.acra.2007.09.027

    authors: White CS,Pugatch R,Koonce T,Rust SW,Dharaiya E

    更新日期:2008-03-01 00:00:00

  • Simulation of mammographic lesions.

    abstract:RATIONALE AND OBJECTIVES:This study presents a method for generating breast masses and microcalcifications in mammography via simulation. This simulation method allows for the creation of large image datasets with particular lesions, which may serve as a useful tool for perception studies measuring imaging system perfo...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2006.03.015

    authors: Saunders R,Samei E,Baker J,Delong D

    更新日期:2006-07-01 00:00:00

  • Effects of Patient Size and Radiation Dose on Iodine Quantification in Dual-Source Dual-Energy CT.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this study was to investigate the potential effects of patient size and radiation dose on the accuracy of iodine quantification using dual-source dual-energy computed tomography (CT). MATERIALS AND METHODS:Three phantoms representing different patient sizes were constructed, con...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.12.027

    authors: Jiang X,Yang X,Hintenlang DE,White RD

    更新日期:2021-01-01 00:00:00

  • Added Value of Bone Subtraction in Dual-energy Digital Radiography in the Detection of Pneumothorax: Impact of Reader Expertise and Medical Specialty.

    abstract:RATIONALE AND OBJECTIVES:This study aimed to determine the value of dual-energy thoracic radiography in the diagnosis of pneumothorax considering the reader's experience. MATERIALS AND METHODS:Forty patients with a suspected pneumothorax, imaged with dual-energy chest radiographs, were divided into two groups: those w...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2017.06.015

    authors: Urbaneja A,Dodin G,Hossu G,Bakour O,Kechidi R,Gondim Teixeira P,Blum A

    更新日期:2018-01-01 00:00:00

  • Can coronary artery anomalies be detected on CT calcium scoring studies?

    abstract:PURPOSE:To determine whether coronary artery anomalies can be detected on noncontrast computed tomography (CT) coronary artery calcium scoring (CCS) studies. MATERIALS AND METHODS:A total of 126 patients (mean age 62 years; 35 women) underwent noncontrast CCS and contrast enhanced coronary CT angiography (cCTA). Thirt...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2012.12.006

    authors: Maddux PT,Schoepf UJ,Zwerner PL,Krazinski AW,Berghaus TM,Morris PB,Thilo C

    更新日期:2013-05-01 00:00:00

  • Computer-aided diagnosis of soft-tissue tumors using sonographic morphologic and texture features.

    abstract:RATIONALE AND OBJECTIVES:The aim of this study was to develop a computer-aided diagnosis (CAD) system in assessing the sonographic morphologic and texture features of soft-tissue tumors. MATERIALS AND METHODS:The retrospective study involved 114 pathology proven cases including 73 benign and 41 malignant soft-tissue t...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2009.07.024

    authors: Chen CY,Chiou HJ,Chou SY,Chiou SY,Wang HK,Chou YH,Chiang HK

    更新日期:2009-12-01 00:00:00

  • Assessment of intraaxial and extraaxial brain lesions with digitized computed tomographic images versus film: ROC analysis.

    abstract:RATIONALE AND OBJECTIVES:The authors evaluated the diagnostic accuracy of viewing computed tomographic (CT) scans as film versus soft-copy images at a workstation. METHODS:Receiver operating characteristic analysis of the interpretation of 202 CT scans (103 were normal, 99 were abnormal) by five neuroradiologists was ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(97)80004-9

    authors: el-Saden SM,Hademenos GJ,Zhu W,Sayre JW,Glenn B,Steidler J,Kode L,King B,Quinones D,Valentino DJ,Bentson JR

    更新日期:1997-02-01 00:00:00

  • Parents of Pediatric Radiology Patients Prefer Timely Reporting and Discussing Results with Referring Providers.

    abstract:RATIONALE AND OBJECTIVES:With the introduction of new communication channels, such as encrypted messaging and online electronic medical record patient portals, there are ever-increasing ways for patients and their families to access their medical information. While patient preferences regarding how they receive the res...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.07.010

    authors: Edwards EA,Cote A,Phelps AS,Naeger DM

    更新日期:2020-05-01 00:00:00

  • Grand rounds and a visiting professorship program in a department of radiology: how we do it.

    abstract::We discuss the benefits of maintaining an active, rigorous, and highly structured grand rounds (GR) program in an academic radiology department. These benefits include education for faculty (continuing medical education), fellows, and residents and a venue for building collaboration and camaraderie within the departme...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2012.08.003

    authors: Tridandapani S,Mullins ME,Meltzer CC

    更新日期:2012-11-01 00:00:00

  • Microvascular versus macrovascular cerebral vasomotor reactivity in patients with severe internal carotid artery stenosis or occlusion.

    abstract:RATIONALE AND OBJECTIVES:In patients with severe internal carotid artery steno-occlusive lesions (ISOL), impaired cerebrovascular reactivity (CVR) is predictive of future ischemic stroke (IS) or transient ischemic attack (TIA). Therefore, the evaluation of CVR in ISOL patients may be a means to evaluate the risk for IS...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2013.10.010

    authors: Zirak P,Delgado-Mederos R,Dinia L,Martí-Fàbregas J,Durduran T

    更新日期:2014-02-01 00:00:00

  • Anatomic Eponyms in Neuroradiology: Brain, Cerebral Vasculature, and Calvarium.

    abstract::Medical eponyms are ubiquitous, numerous, and at times controversial. They are often useful for succinctly conveying complex concepts, and familiarity with eponyms is important for proper usage and appropriate communication. In this historical review, we identify 18 anatomic eponyms used to describe structures of the ...

    journal_title:Academic radiology

    pub_type: 传,历史文章,杂志文章,评审

    doi:10.1016/j.acra.2016.01.008

    authors: Bunch PM,Zamani AA

    更新日期:2016-06-01 00:00:00

  • Optimal ECG trigger point in electron-beam CT studies: three methods for minimizing motion artifacts.

    abstract:RATIONALE AND OBJECTIVES:The authors hypothesized that electrocardiographic triggering near end systole could minimize motion artifacts in electron-beam computed tomography (CT) of the coronary artery. MATERIALS AND METHODS:The study included 2,660 patients who underwent coronary artery calcium scanning with electron-...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/S1076-6332(03)80722-5

    authors: Mao S,Budoff MJ,Bin L,Liu SC

    更新日期:2001-11-01 00:00:00

  • Learning radiology a survey investigating radiology resident use of textbooks, journals, and the internet.

    abstract:RATIONALE AND OBJECTIVES:We surveyed radiology residents to understand which information sources residents use to learn radiology. MATERIALS AND METHODS:A 15-question survey on learning resources was given to radiology residents at one institution. The survey queried residents about their preferences for sources when ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2007.06.002

    authors: Kitchin DR,Applegate KE

    更新日期:2007-09-01 00:00:00

  • CT detection of serious internal and skeletal injuries in patients with pelvic fractures.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this study was to determine the percentage of patients with known pelvic fractures who have additional findings of intraabdominal injury, as diagnosed at abdominal computed tomography (CT), and to determine if patients with specific types or patterns of fractures are more likely ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/S1076-6332(99)80209-8

    authors: Killeen KL,DeMeo JH

    更新日期:1999-04-01 00:00:00

  • Correction of helical CT attenuation values with wide beam collimation: in vitro test with urinary calculi.

    abstract:RATIONALE AND OBJECTIVES:Urinary calculi are now commonly detected with helical computed tomography (CT), and it has been proposed that stone composition can be determined from CT attenuation values. However, typical scans are made with a beam collimation of 5 mm or more, resulting in volume averaging and reduction in ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/S1076-6332(03)80619-0

    authors: Williams JC Jr,Saw KC,Monga AG,Chua GT,Lingeman JE,McAteer JA

    更新日期:2001-06-01 00:00:00

  • Iron oxide nanoparticles as magnetic resonance contrast agent for tumor imaging via folate receptor-targeted delivery.

    abstract:RATIONALE AND OBJECTIVE:Targeted delivery is a highly desirable strategy for diagnostic imaging because of enhanced efficacy and reduced dosage/toxicity. Receptor-targeting was used to deliver contrast-producing superparamagnetic iron oxide (IO) nanoparticle to receptor-expressing tumors for in vivo magnetic resonance ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2004.04.018

    authors: Choi H,Choi SR,Zhou R,Kung HF,Chen IW

    更新日期:2004-09-01 00:00:00

  • Sonographic evaluation of orthotopic bladder tumors in mice treated with TNP-470, an angiogenic inhibitor.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this study was to determine the feasibility of using transabdominal ultrasonography (US) to monitor tumor growth and response to therapy in a mouse model of orthotopic bladder carcinoma. MATERIALS AND METHODS:Human bladder carcinoma cell suspensions were injected into the bladde...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(01)90038-8

    authors: Rooks V,Beecken WD,Iordanescu I,Taylor GA

    更新日期:2001-02-01 00:00:00

  • Combining Washout and Noncontrast Data From Adrenal Protocol CT: Improving Diagnostic Performance.

    abstract:RATIONALE AND OBJECTIVES:To determine if combination of washout and noncontrast data from delayed adrenal computed tomography (CT) improves diagnostic performance, and demonstration of an optimizing analytical framework. MATERIALS AND METHODS:This retrospective study consisted of 97 adrenal lesions, in 96 patients, wi...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2017.12.005

    authors: Ng CS,Altinmakas E,Wei W,Ghosh P,Li X,Grubbs EG,Perrier NA,Prieto VG,Lee JE,Hobbs BP

    更新日期:2018-07-01 00:00:00

  • LSU Health Sciences Center New Orleans Department of Radiology: effects of Hurricane Katrina.

    abstract::This case study chronicles the impact of Hurricane Katrina on the Department of Radiology at the Louisiana State University School of Medicine in New Orleans and the department's subsequent efforts to recover and re-dedicate itself to providing quality patient care and resident education. Hurricane Katrina damaged the...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2009.01.016

    authors: Duggal A,Letourneau JG,Bok LR

    更新日期:2009-05-01 00:00:00

  • Evaluation of analgesic effect of equimolar mixture of oxygen and nitrous oxide inhalation during percutaneous biopsy of focal liver lesions: a double-blind randomized study.

    abstract:RATIONALE AND OBJECTIVES:Percutaneous liver biopsy for tumors performed under local anesthesia is still a painful procedure. The aim of this study was to evaluate the patient's reaction and the analgesic efficacy and safety of an equimolar mixture of oxygen and nitrous oxide (EMONO) inhalation. MATERIALS AND METHODS:A...

    journal_title:Academic radiology

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.acra.2011.01.025

    authors: Meskine N,Vullierme MP,Zappa M,d'Assignies G,Sibert A,Vilgrain V

    更新日期:2011-07-01 00:00:00

  • MDCT for computerized volumetry of pneumothoraces in pediatric patients.

    abstract:RATIONALE AND OBJECTIVES:Our purpose in this study was to develop an automated computer-aided volumetry (CAV) scheme for quantifying pneumothorax in multidetector computed tomography (MDCT) images for pediatric patients and to investigate the imaging parameters that may affect its accuracy. MATERIALS AND METHODS:Fifty...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2010.11.008

    authors: Cai W,Lee EY,Vij A,Mahmood SA,Yoshida H

    更新日期:2011-03-01 00:00:00

  • Continuous arterial spin-labeling perfusion magnetic resonance imaging of the human testis.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this study was to determine if continuous arterial spin-labeling perfusion magnetic resonance imaging could be used to detect testicular perfusion in human subjects. MATERIALS AND METHODS:Continuous arterial spin-labeling magnetic resonance perfusion imaging was performed in sev...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(03)00576-2

    authors: Pretorius ES,Roberts DA

    更新日期:2004-01-01 00:00:00

  • Providing Feedback: Practical Skills and Strategies.

    abstract::Feedback is an essential component of education. It is designed to influence, reinforce, and change behaviors, concepts, and attitudes in learners. Although providing constructive feedback can be challenging, it is a learnable skill. The negative consequences of destructive feedback or lack of feedback all together ar...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2016.11.023

    authors: Sarkany D,Deitte L

    更新日期:2017-06-01 00:00:00

  • Long residence time of ultrasound microbubbles targeted to integrin in murine tumor model.

    abstract:RATIONALE AND OBJECTIVES:The aim of this study was to evaluate the intratumoral residence time of microbubbles (MBs) targeted to alpha(v)beta(3) integrin expressed in the endothelial cells of mice during the process of tumor angiogenesis. MATERIALS AND METHODS:For the preparation of MBs, decafluorobutane gas was sonic...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2009.07.017

    authors: Jun HY,Park SH,Kim HS,Yoon KH

    更新日期:2010-01-01 00:00:00

  • Using a human visual system model to optimize soft-copy mammography display: influence of veiling glare.

    abstract:RATIONALE AND OBJECTIVES:This project evaluated human observer performance and that of a human visual system model (JNDmetrix) to assess whether the veiling glare of a digital display influences observer performance during soft-copy interpretation of mammographic images for the detection of masses. MATERIALS AND METHO...

    journal_title:Academic radiology

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.acra.2005.11.006

    authors: Krupinski EA,Lubin J,Roehrig H,Johnson J,Nafziger J

    更新日期:2006-03-01 00:00:00

  • The value of dynamic MRI studies in parotid tumors.

    abstract:RATIONALE AND OBJECTIVES:To evaluate the ability of dynamic contrast-enhanced magnetic resonance imaging (MRI) to differentiate several tumor entities of the parotid gland in a prospective clinical trial. MATERIALS AND METHODS:A total of 112 patients with parotid tumors were examined with dynamic contrast-enhanced 1.5...

    journal_title:Academic radiology

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.acra.2007.03.004

    authors: Alibek S,Zenk J,Bozzato A,Lell M,Grunewald M,Anders K,Rabe C,Iro H,Bautz W,Greess H

    更新日期:2007-06-01 00:00:00