Multi-reader ROC studies with split-plot designs: a comparison of statistical methods.

Abstract:

RATIONALE AND OBJECTIVES:Multireader imaging trials often use a factorial design, in which study patients undergo testing with all imaging modalities and readers interpret the results of all tests for all patients. A drawback of this design is the large number of interpretations required of each reader. Split-plot designs have been proposed as an alternative, in which one or a subset of readers interprets all images of a sample of patients, while other readers interpret the images of other samples of patients. In this paper, the authors compare three methods of analysis for the split-plot design. MATERIALS AND METHODS:Three statistical methods are presented: the Obuchowski-Rockette method modified for the split-plot design, a newly proposed marginal-mean analysis-of-variance approach, and an extension of the three-sample U-statistic method. A simulation study using the Roe-Metz model was performed to compare the type I error rate, power, and confidence interval coverage of the three test statistics. RESULTS:The type I error rates for all three methods are close to the nominal level but tend to be slightly conservative. The statistical power is nearly identical for the three methods. The coverage of 95% confidence intervals falls close to the nominal coverage for small and large sample sizes. CONCLUSIONS:The split-plot multireader, multicase study design can be statistically efficient compared to the factorial design, reducing the number of interpretations required per reader. Three methods of analysis, shown to have nominal type I error rates, similar power, and nominal confidence interval coverage, are available for this study design.

journal_name

Acad Radiol

journal_title

Academic radiology

authors

Obuchowski NA,Gallas BD,Hillis SL

doi

10.1016/j.acra.2012.09.012

subject

Has Abstract

pub_date

2012-12-01 00:00:00

pages

1508-17

issue

12

eissn

1076-6332

issn

1878-4046

pii

S1076-6332(12)00483-7

journal_volume

19

pub_type

杂志文章
  • Augmented Radiologist Workflow Improves Report Value and Saves Time: A Potential Model for Implementation of Artificial Intelligence.

    abstract:RATIONALE AND OBJECTIVES:Our primary aim was to improve radiology reports by increasing concordance of target lesion measurements with oncology records using radiology preprocessors (RP). Faster notification of incidental actionable findings to referring clinicians and clinical radiologist exam interpretation time savi...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.09.014

    authors: Do HM,Spear LG,Nikpanah M,Mirmomen SM,Machado LB,Toscano AP,Turkbey B,Bagheri MH,Gulley JL,Folio LR

    更新日期:2020-01-01 00:00:00

  • CT angiography with gadolinium-based contrast media.

    abstract:RATIONALE AND OBJECTIVES:To evaluate the potential use of gadolinium (Gd)-based contrast media, especially that of Gadovist, a 1-molar Gd medium, in computed tomography (CT) and compare our findings with standard iodinated contrast media. MATERIAL AND METHODS:Using a live rabbit and an acrylic CT body phantom for comp...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2006.03.019

    authors: Bonvento MJ,Moore WH,Button TM,Weinmann HJ,Yakupov R,Dilmanian FA

    更新日期:2006-08-01 00:00:00

  • Quantitative Multivolume Proton-Magnetic Resonance Imaging in Lung Transplant Recipients: Comparison With Computed Tomography and Spirometry.

    abstract:RATIONALE AND OBJECTIVES:Acute and chronic graft rejection remains the major problem in clinical surveillance of lung-transplanted patients and early detection of complications is of capital importance to allow the optimal therapeutic option. The aim of this study was to investigate the role of quantitative non contras...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2020.05.026

    authors: Pennati F,Salito C,Borzani I,Carrafiello G,Morlacchi LC,Vaira V,Nosotti M,Palleschi A,Aliverti A

    更新日期:2020-07-08 00:00:00

  • Characteristics of regions suspicious for pulmonary nodules at chest radiography.

    abstract:RATIONALE AND OBJECTIVES:This study was performed to determine physical characteristics of areas on chest radiographs that are suspicious but not definitive for the presence of a pulmonary nodule and the characteristics of areas that contain an obvious nodule. MATERIALS AND METHODS:Two groups of patients were identifi...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(98)80297-3

    authors: Drayer JA,Vittitoe NF,Vargas-Voracek R,Baydush AH,Ravin CE,Floyd CE Jr

    更新日期:1998-09-01 00:00:00

  • Evaluation of meniscal tears: fast spin-echo versus conventional spin-echo magnetic resonance imaging.

    abstract:RATIONALE AND OBJECTIVES:We compared the performance of fast spin-echo (FSE) with conventional spin-echo (CSE) magnetic resonance (MR) imaging sequences in the detection of meniscal tears. METHODS:Seventy-three patients underwent MR examination of the knee for suspected internal derangement. Each patient was scanned w...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(05)80166-7

    authors: Anderson MW,Raghavan N,Seidenwurm DJ,Greenspan A,Drake C

    更新日期:1995-03-01 00:00:00

  • High-resolution magnetic resonance imaging and conventional magnetic resonance imaging on a standard field-strength magnetic resonance system compared to arthroscopy in patients with suspected meniscal tears.

    abstract:RATIONALE AND OBJECTIVES:We sought to evaluate the diagnostic performance of high-resolution magnetic resonance imaging (MRI) and conventional MRI of the knee on a standard-field-strength MRI system compared to arthroscopic findings in patients with suspected meniscal tears. MATERIALS AND METHODS:Forty-two patients (2...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2008.02.007

    authors: Nemec SF,Marlovits S,Trattnig S,Matzek W,Mayerhoefer ME,Krestan CR

    更新日期:2008-07-01 00:00:00

  • Evaluation of the effect of computer-aided classification of benign and malignant lesions on reader performance in automated three-dimensional breast ultrasound.

    abstract:RATIONALE AND OBJECTIVES:To investigate the effect of a newly developed computer-aided diagnosis (CAD) system on reader interpretation of breast lesions in automated three-dimensional (3D) breast ultrasound. MATERIALS AND METHODS:A CAD system was developed to differentiate malignant lesions from benign lesions includi...

    journal_title:Academic radiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.acra.2013.07.013

    authors: Tan T,Platel B,Twellmann T,van Schie G,Mus R,Grivegnée A,Mann RM,Karssemeijer N

    更新日期:2013-11-01 00:00:00

  • Assessment of the renal corticomedullary (23)Na gradient using isotropic data sets.

    abstract:RATIONALE AND OBJECTIVES:(23)Na magnetic resonance imaging is a promising technique for the noninvasive imaging of renal function. Past investigations of the renal corticomedullary [(23)Na] gradient have relied on imaging only in the coronal plane and on cumbersome calculations of [(23)Na], which require the use of ext...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2012.10.009

    authors: Haneder S,Konstandin S,Morelli JN,Schad LR,Schoenberg SO,Michaely HJ

    更新日期:2013-04-01 00:00:00

  • Thin-section CT evidence of bronchial thickening in children with stable asthma: bronchoconstriction or airway remodeling?

    abstract:RATIONALE AND OBJECTIVES:The authors performed this study to determine if bronchial wall thickening is present in children with moderate to severe asthma during periods free from clinical bronchoconstriction. MATERIALS AND METHODS:The authors obtained low (radiation) dose thin-section computed tomographic (CT) scans i...

    journal_title:Academic radiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/S1076-6332(03)80535-4

    authors: Ketai L,Coutsias C,Williamson S,Coutsias V

    更新日期:2001-03-01 00:00:00

  • Diagnostic accuracy using diffusion tensor imaging in the diagnosis of ALS: a meta-analysis.

    abstract:RATIONALE AND OBJECTIVES:A number of studies have reported decreases in fractional anistropy (FA) in amyotrophic lateral sclerosis using diffusion tensor imaging (DTI). The purpose of this study was to perform a meta-analysis in order to estimate the diagnostic test accuracy measures of DTI for the diagnosis of amyotro...

    journal_title:Academic radiology

    pub_type: 杂志文章,meta分析

    doi:10.1016/j.acra.2012.04.012

    authors: Foerster BR,Dwamena BA,Petrou M,Carlos RC,Callaghan BC,Pomper MG

    更新日期:2012-09-01 00:00:00

  • LSU Health Sciences Center New Orleans Department of Radiology: effects of Hurricane Katrina.

    abstract::This case study chronicles the impact of Hurricane Katrina on the Department of Radiology at the Louisiana State University School of Medicine in New Orleans and the department's subsequent efforts to recover and re-dedicate itself to providing quality patient care and resident education. Hurricane Katrina damaged the...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2009.01.016

    authors: Duggal A,Letourneau JG,Bok LR

    更新日期:2009-05-01 00:00:00

  • Comparison of digital radiographs with film radiographs for the classification of pneumoconiosis.

    abstract:RATIONALE AND OBJECTIVES:Digital chest imaging has replaced film chest radiographs in many centers, but the International Labour Organization classification system, which is the most widely used system for recognition and classification of dust-related abnormalities, is predicated on film chest radiographs. The purpose...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2008.12.020

    authors: Franzblau A,Kazerooni EA,Sen A,Goodsitt MM,Lee SY,Rosenman KD,Lockey JE,Meyer CA,Gillespie BW,Petsonk EL,Wang ML

    更新日期:2009-06-01 00:00:00

  • Social Media Tips to Enhance Medical Education.

    abstract::In this article, we describe how social media can supplement traditional education, articulate the advantages and disadvantages of various social media platforms for both teachers and learners, discuss best practices to maintain confidentiality of protected health information, and provide tips for implementing social ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2016.12.023

    authors: Shah V,Kotsenas AL

    更新日期:2017-06-01 00:00:00

  • The effects of magnetically labeled rat spleen-originated endothelial progenitor cells on growth of glioma in vivo an experimental study.

    abstract:RATIONALE AND OBJECTIVES:The aim of this study was to investigate the effects of exogenous endothelial progenitor cells (EPCs) on the growth and invasiveness of glioma in vivo to provide an experimental basis for the value and safety of using magnetically labeled EPCs as target vectors to detect early infiltration of g...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2011.02.017

    authors: Fang J,Wang S,Chen J,Zhang Y,Zhang B,Liang H,Zhang W

    更新日期:2011-07-01 00:00:00

  • Modified Subtraction Coronary CT Angiography Method for Patients Unable to Perform Long Breath-Holds: A Preliminary Study.

    abstract:RATIONALE AND OBJECTIVES:Severe calcifications of the coronary arteries are still a major challenge in coronary computed tomography (CT) angiography (CCTA). Subtraction CCTA using a 320-detector row CT scanner has recently been introduced for patients with severe calcifications. However, the conventional subtraction CC...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2016.04.012

    authors: Yoshioka K,Tanaka R,Nagata K,Sasaki T,Takeda K,Ueda T,Sugawara T,Ueyama Y,Chiba T,Sasaki A,Kikuchi K

    更新日期:2016-09-01 00:00:00

  • A model for a PC-based, universal-format, multimedia digitization system: moving beyond the scanner.

    abstract:RATIONALE AND OBJECTIVES:Digitizing images for use in case presentations based on hardcopy films, slides, photographs, negatives, books, and videos can present a challenging task. Scanners and digital cameras have become standard tools of the trade. Unfortunately, use of these devices to digitize multiple images in man...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(03)00010-2

    authors: McEachen JC,Cusack TJ,McEachen JC

    更新日期:2003-08-01 00:00:00

  • Usefulness of 1H MR spectroscopy in the evaluation of myocardial metabolism in patients with dilated idiopathic cardiomyopathy: pilot study.

    abstract:RATIONALE AND OBJECTIVES:Primary dilated cardiomyopathy is a disease of unknown etiology, and it leads to severe heart failure. Abnormalities of the cardiac metabolism can play an important role in prognosis and influence the symptomatology in this group. The aim of this study was to assess cardiac metabolism using pro...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(03)00104-1

    authors: Walecki J,Michalak MJ,Michalak E,Bilinska ZT,Ruzyllo W

    更新日期:2003-10-01 00:00:00

  • Healthy aging: an automatic analysis of global and regional morphological alterations of human brain.

    abstract:RATIONALE AND OBJECTIVES:Morphologic changes of the human brain during healthy aging provide useful reference knowledge for age-related brain disorders. The aim of this study was to explore age-related global and regional morphological changes of healthy adult brains. MATERIALS AND METHODS:T1-weighted magnetic resonan...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2012.03.006

    authors: Long X,Liao W,Jiang C,Liang D,Qiu B,Zhang L

    更新日期:2012-07-01 00:00:00

  • Clinical Outcomes of Patients with Porcelain Gallbladder Diagnosed on CT.

    abstract:PURPOSE:To evaluate diagnostic accuracy and clinical outcomes of patients with porcelain gallbladder (GB) diagnosed on CT. MATERIALS AND METHODS:In this IRB-approved, HIPAA-compliant study, consecutive patients with porcelain gallbladder reported on CT between December 1, 2000 and August 31, 2017 in a tertiary academi...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2020.07.006

    authors: Appel E,Dommaraju S,Siewert B,Gonzalez RS,Camacho A,Park Y,Brook OR

    更新日期:2020-08-04 00:00:00

  • Visual and Quantitative Evaluation of Emphysema: A Case-Control Study of 1111 Participants in the Pilot Swedish CArdioPulmonary BioImage Study (SCAPIS).

    abstract:RATIONALE AND OBJECTIVES:Emphysema is a hallmark of chronic obstructive pulmonary disease. The primary aim of this study was to investigate inter- and intraobserver agreement of visual assessment of mild emphysema in low-dose multidetector computed tomography of subjects in the pilot SCAPIS in order to certify consiste...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.06.019

    authors: Vikgren J,Khalil M,Cederlund K,Sörensen K,Boijsen M,Brandberg J,Lampa E,Sköld MC,Wollmer P,Lindberg E,Engvall JE,Bergström G,Torén K,Johnsson ÅA

    更新日期:2020-05-01 00:00:00

  • Automatic determination of differential coronary artery motion minima for cardiac computed tomography optimal phase selection.

    abstract:RATIONALE AND OBJECTIVES:Selecting the optimal phase for coronary artery evaluation can be challenging, especially at higher heart rates, given that the optimal phase may differ for each of the coronary arteries. This study aimed to evaluate a novel vessel-specific algorithm which automatically outputs the minimum moti...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2015.01.013

    authors: Lessick J,Klass O,Wuchenauer S,Walker MJ,Schmitt H,Peters J,Weese J,Brunner H,Vembar M,Grass M,Aronson D,Hoffmann MH

    更新日期:2015-06-01 00:00:00

  • CT-Radiomic Approach to Predict G1/2 Nonfunctional Pancreatic Neuroendocrine Tumor.

    abstract:RATIONALE AND OBJECTIVES:Tumor grading of nonfunctional pancreatic neuroendocrine tumors (NF-pNETs) determines the choice of clinical treatment and management. The pathological grade of pancreatic neuroendocrine tumors is usually assessed on postoperative specimens. The goal of our study is to establish a tumor grade (...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2020.01.002

    authors: Zhao Z,Bian Y,Jiang H,Fang X,Li J,Cao K,Ma C,Wang L,Zheng J,Yue X,Zhang H,Wang X,Madabhushi A,Xu J,Jin G,Lu J

    更新日期:2020-12-01 00:00:00

  • Endpoint localization in guide wire tracking during endovascular interventions.

    abstract:RATIONALE AND OBJECTIVES:A method is presented to track guide wires during endovascular interventions under X-ray fluoroscopy. Accurate guide wire tracking can be used to improve guide wire visualization in the low quality fluoroscopic images, and to estimate the position of the guide wire in world coordinates for navi...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(03)00539-7

    authors: Baert SA,van Walsum T,Niessen WJ

    更新日期:2003-12-01 00:00:00

  • Use of 3D Printed Models to Create Molds for Shaping Implants for Surgical Repair of Orbital Fractures.

    abstract:RATIONALE AND OBJECTIVES:Surgical repair of an isolated orbital fracture requires anatomically accurate implant shape and placement. We describe a three-dimensional (3D) printing technique to customize the shape of commercially available absorbable implants. MATERIALS AND METHODS:We reviewed our early experience with ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.06.023

    authors: Weadock WJ,Heisel CJ,Kahana A,Kim J

    更新日期:2020-04-01 00:00:00

  • Continuous quality improvement for radiologists.

    abstract::Continuous quality improvement methods provide a relatively new way to improve quality in healthcare by using a systems approach with key features of customer-mindedness, data collection, experimentation, and teamwork. CQI is not meant to solve simple problems but is increasingly applied in our complex healthcare orga...

    journal_title:Academic radiology

    pub_type: 杂志文章,评审

    doi:10.1016/s1076-6332(03)00670-6

    authors: Applegate KE

    更新日期:2004-02-01 00:00:00

  • Inflammatory Activity in Atelectatic and Normally Aerated Regions During Early Acute Lung Injury.

    abstract:RATIONALE AND OBJECTIVES:Pulmonary atelectasis presumably promotes and facilitates lung injury. However, data are limited on its direct and remote relation to inflammation. We aimed to assess regional 2-deoxy-2-[18F]-fluoro-D-glucose (18F-FDG) kinetics representative of inflammation in atelectatic and normally aerated ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2019.12.022

    authors: Hinoshita T,Ribeiro GM,Winkler T,de Prost N,Tucci MR,Costa ELV,Wellman TJ,Hashimoto S,Zeng C,Carvalho AR,Melo MFV

    更新日期:2020-12-01 00:00:00

  • Tracheobronchial metal stents: effects of covering a bronchial ostium in pigs.

    abstract:RATIONALE AND OBJECTIVES:The purpose of this study was to examine the effects of placing a metal stent across a bronchial orifice. MATERIALS AND METHODS:Nine pigs were used as test subjects, because the right upper lobe bronchus comes directly off the trachea in these animals. One of three types of metal stents was pl...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/s1076-6332(99)80253-0

    authors: Bjarnason H,Cahill B,Kløw NE,Han YM,Urness M,Günther R,Gomes MR,Rosenberg MS,Dietz CA Jr,Hunter DW

    更新日期:1999-10-01 00:00:00

  • Optimization of shaded surface display for CT angiography.

    abstract:RATIONALE AND OBJECTIVES:The authors performed this study to determine the optimum threshold for performing computed tomographic (CT) angiography with shaded surface display (SSD). MATERIALS AND METHODS:A dedicated phantom was developed with an 8-mm luminal diameter. Each of 19 vessels had stenoses ranging from 0% to ...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/S1076-6332(03)80641-4

    authors: Addis KA,Hopper KD,Iyriboz TA,Kasales CJ,Liu Y,Wise SW

    更新日期:2001-10-01 00:00:00

  • A Breast Radiology Department-operated, Proactive Same-day Program Identifies Pathogenic Breast Cancer Mutations in Unaffected Women.

    abstract:RATIONALE AND OBJECTIVES:Pathogenic mutations in some genes elevate women's breast cancer risk, necessitating risk-reduction strategies. Unfortunately, women are underscreened for cancer risk, and when identified as potentially high risk, women seldom pursue genetic counseling or testing. To improve cancer risk managem...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2020.12.005

    authors: Loving VA,Luiten RC,Siettmann JM,Mina LA

    更新日期:2020-12-15 00:00:00

  • 3 Tesla 23Na Magnetic Resonance Imaging During Acute Kidney Injury.

    abstract:RATIONALE AND OBJECTIVES:Sodium and proton magnetic resonance imaging (23Na/1H-MRI) have shown that muscle and skin can store Na+ without water. In chronic renal failure and in heart failure, Na+ mobilization occurs, but is variable depending on age, dialysis vintage, and other features. Na+ storage depots have not bee...

    journal_title:Academic radiology

    pub_type: 杂志文章

    doi:10.1016/j.acra.2017.03.012

    authors: Hammon M,Grossmann S,Linz P,Seuss H,Hammon R,Rosenhauer D,Janka R,Cavallaro A,Luft FC,Titze J,Uder M,Dahlmann A

    更新日期:2017-09-01 00:00:00