Establishment of Best Practices for Evidence for Prediction: A Review.

Abstract:

Importance:Great interest exists in identifying methods to predict neuropsychiatric disease states and treatment outcomes from high-dimensional data, including neuroimaging and genomics data. The goal of this review is to highlight several potential problems that can arise in studies that aim to establish prediction. Observations:A number of neuroimaging studies have claimed to establish prediction while establishing only correlation, which is an inappropriate use of the statistical meaning of prediction. Statistical associations do not necessarily imply the ability to make predictions in a generalized manner; establishing evidence for prediction thus requires testing of the model on data separate from those used to estimate the model's parameters. This article discusses various measures of predictive performance and the limitations of some commonly used measures, with a focus on the importance of using multiple measures when assessing performance. For classification, the area under the receiver operating characteristic curve is an appropriate measure; for regression analysis, correlation should be avoided, and median absolute error is preferred. Conclusions and Relevance:To ensure accurate estimates of predictive validity, the recommended best practices for predictive modeling include the following: (1) in-sample model fit indices should not be reported as evidence for predictive accuracy, (2) the cross-validation procedure should encompass all operations applied to the data, (3) prediction analyses should not be performed with samples smaller than several hundred observations, (4) multiple measures of prediction accuracy should be examined and reported, (5) the coefficient of determination should be computed using the sums of squares formulation and not the correlation coefficient, and (6) k-fold cross-validation rather than leave-one-out cross-validation should be used.

journal_name

JAMA Psychiatry

journal_title

JAMA psychiatry

authors

Poldrack RA,Huckins G,Varoquaux G

doi

10.1001/jamapsychiatry.2019.3671

subject

Has Abstract

pub_date

2020-05-01 00:00:00

pages

534-540

issue

5

eissn

2168-622X

issn

2168-6238

pii

2756204

journal_volume

77

pub_type

杂志文章
  • Repetitive Transcranial Magnetic Stimulation for the Acute Treatment of Major Depressive Episodes: A Systematic Review With Network Meta-analysis.

    abstract:Importance:Although several strategies of repetitive transcranial magnetic stimulation (rTMS) have been investigated as treatment of major depressive disorder (MDD), their comparative efficacy and acceptability is unknown. Objective:To establish the relative efficacy and acceptability of the different modalities of rT...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,meta分析,评审

    doi:10.1001/jamapsychiatry.2016.3644

    authors: Brunoni AR,Chaimani A,Moffa AH,Razza LB,Gattaz WF,Daskalakis ZJ,Carvalho AF

    更新日期:2017-02-01 00:00:00

  • Altered Markers of Cortical γ-Aminobutyric Acid Neuronal Activity in Schizophrenia: Role of the NARP Gene.

    abstract:IMPORTANCE:In schizophrenia, working memory deficits appear to reflect abnormalities in the generation of gamma oscillations in the dorsolateral prefrontal cortex. The generation of gamma oscillations requires the phasic excitation of inhibitory parvalbumin-containing interneurons. Thus, gamma oscillations depend, in p...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2015.0533

    authors: Kimoto S,Zaki MM,Bazmi HH,Lewis DA

    更新日期:2015-08-01 00:00:00

  • Prescription Opioid Use and Risk for Major Depressive Disorder and Anxiety and Stress-Related Disorders: A Multivariable Mendelian Randomization Analysis.

    abstract:Importance:Growing evidence suggests that prescription opioid use affects depression and anxiety disorders; however, observational studies are subject to confounding, making causal inference and determining the direction of these associations difficult. Objective:To investigate the potential bidirectional associations...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.3554

    authors: Rosoff DB,Smith GD,Lohoff FW

    更新日期:2020-11-11 00:00:00

  • Latent Trajectories of Common Mental Health Disorder Risk Across 3 Decades of Adulthood in a Population-Based Cohort.

    abstract:Importance:Epidemiologic evidence indicates that most of the general population will experience a mental health disorder at some point in their lives. However, few prospective population-based studies have estimated trajectories of risk for mental disorders from young through middle adulthood to estimate the proportion...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2016.1921

    authors: Paksarian D,Cui L,Angst J,Ajdacic-Gross V,Rössler W,Merikangas KR

    更新日期:2016-10-01 00:00:00

  • Mental Health in the Coronavirus Disease 2019 Emergency-The Italian Response.

    abstract:Importance:This article briefly reports the experience of mental health services and the lessons learned during the coronavirus disease 2019 (COVID-19) crisis. In particular, this report offers opportunities to build on experience gained in managing the COVID-19 emergency in the Departments of Mental Health and Addicti...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.1276

    authors: de Girolamo G,Cerveri G,Clerici M,Monzani E,Spinogatti F,Starace F,Tura G,Vita A

    更新日期:2020-09-01 00:00:00

  • Effect of Aspirin vs Placebo on the Prevention of Depression in Older People: A Randomized Clinical Trial.

    abstract:Importance:Depression is associated with increased inflammation, which may precede its onset, especially in older people. Some preclinical data suggest potential antidepressant effects of aspirin, supported by limited observational data suggesting lower rates of depression in individuals treated with aspirin. There cur...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.1214

    authors: Berk M,Woods RL,Nelson MR,Shah RC,Reid CM,Storey E,Fitzgerald S,Lockery JE,Wolfe R,Mohebbi M,Dodd S,Murray AM,Stocks N,Fitzgerald PB,Mazza C,Agustini B,McNeil JJ

    更新日期:2020-10-01 00:00:00

  • Neural correlates of recall of life events in conversion disorder.

    abstract:IMPORTANCE:Freud argued that in conversion disorder (CD) the affect attached to stressful memories is "repressed" and "converted" into physical symptoms, although this has never been subject to scientific study to our knowledge. OBJECTIVE:To examine the neural correlates of recall of life events judged to be of causal...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2013.2842

    authors: Aybek S,Nicholson TR,Zelaya F,O'Daly OG,Craig TJ,David AS,Kanaan RA

    更新日期:2014-01-01 00:00:00

  • Association Between Maternal Smoking During Pregnancy and Severe Mental Illness in Offspring.

    abstract:Importance:Several recent population-based studies have linked exposure to maternal smoking during pregnancy to increased risk of severe mental illness in offspring (eg, bipolar disorder, schizophrenia). It is not yet clear, however, whether this association results from causal teratogenic effects or from confounding i...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2017.0456

    authors: Quinn PD,Rickert ME,Weibull CE,Johansson ALV,Lichtenstein P,Almqvist C,Larsson H,Iliadou AN,D'Onofrio BM

    更新日期:2017-06-01 00:00:00

  • Incidence and Determinants of Mental Health Service Use After Bariatric Surgery.

    abstract:Importance:Obesity is associated with an increased prevalence of psychiatric disorders. The association of bariatric surgery with mental health outcomes is poorly understood. Objective:To investigate the association of bariatric surgery with the incidence of outpatient, emergency department (ED), and inpatient mental ...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2019.2741

    authors: Morgan DJR,Ho KM,Platell C

    更新日期:2020-01-01 00:00:00

  • Self-injury Mortality in the United States in the Early 21st Century: A Comparison With Proximally Ranked Diseases.

    abstract:Importance:Fatal self-injury in the United States associated with deliberate behaviors is seriously underestimated owing to misclassification of poisoning suicides and mischaracterization of most drug poisoning deaths as "accidents" on death certificates. Objective:To compare national trends and patterns of self-injur...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2016.1870

    authors: Rockett IR,Lilly CL,Jia H,Larkin GL,Miller TR,Nelson LS,Nolte KB,Putnam SL,Smith GS,Caine ED

    更新日期:2016-10-01 00:00:00

  • Effect of Early Adult Patterns of Physical Activity and Television Viewing on Midlife Cognitive Function.

    abstract:IMPORTANCE:Sedentary behaviors and physical inactivity are not only increasing worldwide but also are critical risk factors for adverse health outcomes. Yet, few studies have examined the effects of sedentary behavior on cognition or the long-term role of either behavior in early to middle adulthood. OBJECTIVE:To inve...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2015.2468

    authors: Hoang TD,Reis J,Zhu N,Jacobs DR Jr,Launer LJ,Whitmer RA,Sidney S,Yaffe K

    更新日期:2016-01-01 00:00:00

  • Identification of pathways for bipolar disorder: a meta-analysis.

    abstract:IMPORTANCE:Genome-wide investigations provide systematic information regarding the neurobiology of psychiatric disorders. OBJECTIVE:To identify biological pathways that contribute to risk for bipolar disorder (BP) using genes with consistent evidence for association in multiple genome-wide association studies (GWAS). ...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,meta分析

    doi:10.1001/jamapsychiatry.2014.176

    authors: Nurnberger JI Jr,Koller DL,Jung J,Edenberg HJ,Foroud T,Guella I,Vawter MP,Kelsoe JR,Psychiatric Genomics Consortium Bipolar Group.

    更新日期:2014-06-01 00:00:00

  • A multisite, double-blind, placebo-controlled clinical trial to evaluate the safety and efficacy of vigabatrin for treating cocaine dependence.

    abstract:IMPORTANCE:Cocaine dependence is a significant public health problem, yet no validated pharmacological treatment exists. The potent γ-aminobutyric acid (GABA)ergic medication vigabatrin has previously been shown to be effective in a double-blind single-site study conducted in Mexico. OBJECTIVE:To evaluate the safety a...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,多中心研究,随机对照试验

    doi:10.1001/jamapsychiatry.2013.872

    authors: Somoza EC,Winship D,Gorodetzky CW,Lewis D,Ciraulo DA,Galloway GP,Segal SD,Sheehan M,Roache JD,Bickel WK,Jasinski D,Watson DW,Miller SR,Somoza P,Winhusen T

    更新日期:2013-06-01 00:00:00

  • Posttraumatic stress disorder and incidence of type 2 diabetes mellitus in a sample of women: a 22-year longitudinal study.

    abstract:IMPORTANCE:Posttraumatic stress disorder (PTSD) is a common, debilitating mental disorder that has been associated with type 2 diabetes mellitus (T2D) and its risk factors, including obesity, in cross-sectional studies. If PTSD increases risk of incident T2D, enhanced surveillance in high-risk populations may be warran...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2014.2632

    authors: Roberts AL,Agnew-Blais JC,Spiegelman D,Kubzansky LD,Mason SM,Galea S,Hu FB,Rich-Edwards JW,Koenen KC

    更新日期:2015-03-01 00:00:00

  • Epidemiology of DSM-5 Alcohol Use Disorder: Results From the National Epidemiologic Survey on Alcohol and Related Conditions III.

    abstract:IMPORTANCE:National epidemiologic information from recently collected data on the new DSM-5 classification of alcohol use disorder (AUD) using a reliable, valid, and uniform data source is needed. OBJECTIVE:To present nationally representative findings on the prevalence, correlates, psychiatric comorbidity, associated...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2015.0584

    authors: Grant BF,Goldstein RB,Saha TD,Chou SP,Jung J,Zhang H,Pickering RP,Ruan WJ,Smith SM,Huang B,Hasin DS

    更新日期:2015-08-01 00:00:00

  • Causes of Death After Nonfatal Opioid Overdose.

    abstract:Importance:A recent increase in patients presenting with nonfatal opioid overdoses has focused clinical attention on characterizing their risks of premature mortality. Objective:To describe all-cause mortality rates, selected cause-specific mortality rates, and standardized mortality rate ratios (SMRs) of adults durin...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2018.1471

    authors: Olfson M,Crystal S,Wall M,Wang S,Liu SM,Blanco C

    更新日期:2018-08-01 00:00:00

  • Associations Between Prenatal Cannabis Exposure and Childhood Outcomes: Results From the ABCD Study.

    abstract:Importance:In light of increasing cannabis use among pregnant women, the US Surgeon General recently issued an advisory against the use of marijuana during pregnancy. Objective:To evaluate whether cannabis use during pregnancy is associated with adverse outcomes among offspring. Design, Setting, and Participants:In t...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.2902

    authors: Paul SE,Hatoum AS,Fine JD,Johnson EC,Hansen I,Karcher NR,Moreau AL,Bondy E,Qu Y,Carter EB,Rogers CE,Agrawal A,Barch DM,Bogdan R

    更新日期:2021-01-01 00:00:00

  • A Telephone-Based Program to Provide Symptom Monitoring Alone vs Symptom Monitoring Plus Care Management for Late-Life Depression and Anxiety: A Randomized Clinical Trial.

    abstract:IMPORTANCE:Mental health (MH) conditions are undertreated in late life. It is important to identify treatment strategies that address variability in treatment content and delivery and take individual-specific symptoms into account, particularly among low-income, community-dwelling older adults. OBJECTIVE:To evaluate p...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,随机对照试验

    doi:10.1001/jamapsychiatry.2015.2157

    authors: Mavandadi S,Benson A,DiFilippo S,Streim JE,Oslin D

    更新日期:2015-12-01 00:00:00

  • Depression with atypical features and increase in obesity, body mass index, waist circumference, and fat mass: a prospective, population-based study.

    abstract:IMPORTANCE:Depression and obesity are 2 prevalent disorders that have been repeatedly shown to be associated. However, the mechanisms and temporal sequence underlying this association are poorly understood. OBJECTIVE:To determine whether the subtypes of major depressive disorder (MDD; melancholic, atypical, combined, ...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2014.411

    authors: Lasserre AM,Glaus J,Vandeleur CL,Marques-Vidal P,Vaucher J,Bastardot F,Waeber G,Vollenweider P,Preisig M

    更新日期:2014-08-01 00:00:00

  • Metabolic Effects of Antipsychotics on Adiposity and Insulin Sensitivity in Youths: A Randomized Clinical Trial.

    abstract:Importance:Antipsychotic medications are commonly used to treat nonpsychotic disruptive behavioral disorders in youths. Objective:To characterize the metabolic effects of first exposure to antipsychotics in youths using criterion standard assessments of body composition and insulin sensitivity. Design, Setting, and P...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,随机对照试验

    doi:10.1001/jamapsychiatry.2018.1088

    authors: Nicol GE,Yingling MD,Flavin KS,Schweiger JA,Patterson BW,Schechtman KB,Newcomer JW

    更新日期:2018-08-01 00:00:00

  • Comparison of 2 family therapies for adolescent anorexia nervosa: a randomized parallel trial.

    abstract:IMPORTANCE:Anorexia nervosa (AN) is a serious disorder with high rates of morbidity and mortality. Family-based treatment (FBT) is an evidence-based therapy for adolescent AN, but less than half of those who receive this approach recover. Hence, it is important to identify other approaches to prevent the development of...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,多中心研究,随机对照试验

    doi:10.1001/jamapsychiatry.2014.1025

    authors: Agras WS,Lock J,Brandt H,Bryson SW,Dodge E,Halmi KA,Jo B,Johnson C,Kaye W,Wilfley D,Woodside B

    更新日期:2014-11-01 00:00:00

  • Activation, self-management, engagement, and retention in behavioral health care: a randomized clinical trial of the DECIDE intervention.

    abstract:IMPORTANCE:Given minority patients' unequal access to quality care, patient activation and self-management strategies have been suggested as a promising approach to improving mental health care. OBJECTIVE:To determine whether the DECIDE (Decide the problem; Explore the questions; Closed or open-ended questions; Identi...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,多中心研究,随机对照试验

    doi:10.1001/jamapsychiatry.2013.4519

    authors: Alegría M,Carson N,Flores M,Li X,Shi P,Lessios AS,Polo A,Allen M,Fierro M,Interian A,Jimenez A,La Roche M,Lee C,Lewis-Fernández R,Livas-Stein G,Safar L,Schuman C,Storey J,Shrout PE

    更新日期:2014-05-01 00:00:00

  • Prediction of functional outcome in individuals at clinical high risk for psychosis.

    abstract:IMPORTANCE:A major public health concern associated with schizophrenia and psychotic disorders is the long-term disability that involves impaired cognition, lack of social support, and an inability to function independently in the community. A critical goal of early detection and intervention studies in psychosis is th...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2013.1909

    authors: Carrión RE,McLaughlin D,Goldberg TE,Auther AM,Olsen RH,Olvet DM,Correll CU,Cornblatt BA

    更新日期:2013-11-01 00:00:00

  • Associations Between Cannabis Use and Physical Health Problems in Early Midlife: A Longitudinal Comparison of Persistent Cannabis vs Tobacco Users.

    abstract:IMPORTANCE:After major policy changes in the United States, policymakers, health care professionals, and the general public seek information about whether recreational cannabis use is associated with physical health problems later in life. OBJECTIVE:To test associations between cannabis use over 20 years and a variety...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2016.0637

    authors: Meier MH,Caspi A,Cerdá M,Hancox RJ,Harrington H,Houts R,Poulton R,Ramrakha S,Thomson WM,Moffitt TE

    更新日期:2016-07-01 00:00:00

  • Genetic and environmental influences on the familial transmission of externalizing disorders in adoptive and twin offspring.

    abstract:IMPORTANCE:Twin-family studies have shown that parent-child resemblance on substance use disorders and antisocial behavior can be accounted for by the transmission of a general liability to a spectrum of externalizing disorders. Most studies, however, include only biological parents and offspring, which confound geneti...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2013.258

    authors: Hicks BM,Foster KT,Iacono WG,McGue M

    更新日期:2013-10-01 00:00:00

  • Interventions to Reduce Compulsory Psychiatric Admissions: A Systematic Review and Meta-analysis.

    abstract:IMPORTANCE:Compulsory admissions, defined as admissions against the will of the patient (according to local judicial procedures), have a strong effect on psychiatric patients. In several Western countries, the rate of such admissions is tending to rise. Its reduction is urgently needed. OBJECTIVE:To establish which in...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,meta分析,评审

    doi:10.1001/jamapsychiatry.2016.0501

    authors: de Jong MH,Kamperman AM,Oorschot M,Priebe S,Bramer W,van de Sande R,Van Gool AR,Mulder CL

    更新日期:2016-07-01 00:00:00

  • Use of Machine Learning to Determine Deviance in Neuroanatomical Maturity Associated With Future Psychosis in Youths at Clinically High Risk.

    abstract:Importance:Altered neurodevelopmental trajectories are thought to reflect heterogeneity in the pathophysiologic characteristics of schizophrenia, but whether neural indicators of these trajectories are associated with future psychosis is unclear. Objective:To investigate distinct neuroanatomical markers that can diffe...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,多中心研究

    doi:10.1001/jamapsychiatry.2018.1543

    authors: Chung Y,Addington J,Bearden CE,Cadenhead K,Cornblatt B,Mathalon DH,McGlashan T,Perkins D,Seidman LJ,Tsuang M,Walker E,Woods SW,McEwen S,van Erp TGM,Cannon TD,North American Prodrome Longitudinal Study (NAPLS) Consortium and

    更新日期:2018-09-01 00:00:00

  • Efficacy and Safety of Transcranial Direct Current Stimulation as an Add-on Treatment for Bipolar Depression: A Randomized Clinical Trial.

    abstract:Importance:More effective, tolerable interventions for bipolar depression treatment are needed. Transcranial direct current stimulation (tDCS) is a novel therapeutic modality with few severe adverse events that showed promising results for unipolar depression. Objective:To determine the efficacy and safety of tDCS as ...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章,随机对照试验

    doi:10.1001/jamapsychiatry.2017.4040

    authors: Sampaio-Junior B,Tortella G,Borrione L,Moffa AH,Machado-Vieira R,Cretaz E,Fernandes da Silva A,Fraguas R,Aparício LV,Klein I,Lafer B,Goerigk S,Benseñor IM,Lotufo PA,Gattaz WF,Brunoni AR

    更新日期:2018-02-01 00:00:00

  • Cortical Proteins Associated With Cognitive Resilience in Community-Dwelling Older Persons.

    abstract:Importance:Identifying genes and proteins for cognitive resilience (ie, targets that may be associated with slowing or preventing cognitive decline regardless of the presence, number, or combination of common neuropathologic conditions) provides a complementary approach to developing novel therapeutics for the treatmen...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.1807

    authors: Yu L,Tasaki S,Schneider JA,Arfanakis K,Duong DM,Wingo AP,Wingo TS,Kearns N,Thatcher GRJ,Seyfried NT,Levey AI,De Jager PL,Bennett DA

    更新日期:2020-07-01 00:00:00

  • Religious Service Attendance and Deaths Related to Drugs, Alcohol, and Suicide Among US Health Care Professionals.

    abstract:Importance:The increase in deaths related to drugs, alcohol, and suicide (referred to as deaths from despair) has been identified as a public health crisis. The antecedents associated with these deaths have, however, seldom been investigated empirically. Objective:To prospectively examine the association between relig...

    journal_title:JAMA psychiatry

    pub_type: 杂志文章

    doi:10.1001/jamapsychiatry.2020.0175

    authors: Chen Y,Koh HK,Kawachi I,Botticelli M,VanderWeele TJ

    更新日期:2020-07-01 00:00:00