Reference effect measures for quantifying, comparing and visualizing variation from random and fixed effects in non-normal multilevel models, with applications to site variation in medical procedure use and outcomes.

Abstract:

BACKGROUND:Multilevel models for non-normal outcomes are widely used in medical and health sciences research. While methods for interpreting fixed effects are well-developed, methods to quantify and interpret random cluster variation and compare it with other sources of variation are less established. Random cluster variation, sometimes referred to as general contextual effects (GCE), may be the main focus of a study; therefore, easily interpretable methods are needed to quantify GCE. We propose a Reference Effect Measure (REM) approach to 1) quantify GCE and compare it to individual subject and cluster covariate effects, and 2) quantify relative magnitudes of GCE and variation from sets of measured factors. METHODS:To illustrate REM, we consider a two-level mixed logistic model with patients clustered within hospitals and a random intercept for hospitals. We compare patients at hospitals at given percentiles of the estimated random effect distribution to patients at a median or 'reference' hospital. These estimates are then compared numerically and graphically to individual fixed effects to quantify GCE in the context of effects of other measured variables (aim 1). We then extend this approach by comparing variation from the random effect distribution to variation from sets of fixed effects to understand their magnitudes relative to overall outcome variation (aim 2). RESULTS:Using an example of initiation of rhythm control treatment in atrial fibrillation (AF) patients within the Veterans Affairs (VA), we use REM to demonstrate that random variation across hospitals (GCE) in initiation of treatment is substantially greater than that due to most individual patient factors, and explains at least as much variation in treatment initiation as do all patient factors combined. These results are contrasted with a relatively small GCE compared with patient factors in 1 year mortality following hospitalization for AF patients. CONCLUSIONS:REM provides a means of quantifying random effect variation (GCE) with multilevel data and can be used to explore drivers of outcome variation. This method is easily interpretable and can be presented visually. REM offers a simple, interpretable approach for evaluating questions of growing importance in the study of health care systems.

journal_name

BMC Med Res Methodol

authors

Glorioso TJ,Grunwald GK,Ho PM,Maddox TM

doi

10.1186/s12874-018-0517-7

subject

Has Abstract

pub_date

2018-07-06 00:00:00

pages

74

issue

1

issn

1471-2288

pii

10.1186/s12874-018-0517-7

journal_volume

18

pub_type

杂志文章
  • Outcome-sensitive multiple imputation: a simulation study.

    abstract:BACKGROUND:Multiple imputation is frequently used to deal with missing data in healthcare research. Although it is known that the outcome should be included in the imputation model when imputing missing covariate values, it is not known whether it should be imputed. Similarly no clear recommendations exist on: the util...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0281-5

    authors: Kontopantelis E,White IR,Sperrin M,Buchan I

    更新日期:2017-01-09 00:00:00

  • Variable selection in social-environmental data: sparse regression and tree ensemble machine learning approaches.

    abstract:BACKGROUND:Social-environmental data obtained from the US Census is an important resource for understanding health disparities, but rarely is the full dataset utilized for analysis. A barrier to incorporating the full data is a lack of solid recommendations for variable selection, with researchers often hand-selecting ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01183-9

    authors: Handorf E,Yin Y,Slifker M,Lynch S

    更新日期:2020-12-10 00:00:00

  • Accounting for misclassification bias of binary outcomes due to underscreening: a sensitivity analysis.

    abstract:BACKGROUND:Diagnostic tests are performed in a subset of the population who are at higher risk, resulting in undiagnosed cases among those who do not receive the test. This poses a challenge for estimating the prevalence of the disease in the study population, and also for studying the risk factors for the disease. ME...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0447-9

    authors: Zhang N,Cheng S,Ambroggio L,Florin TA,Macaluso M

    更新日期:2017-12-12 00:00:00

  • Implementation of clinical research trials using web-based and mobile devices: challenges and solutions.

    abstract:BACKGROUND:With the increasing implementation of web-based, mobile health interventions in clinical trials, it is crucial for researchers to address the security and privacy concerns of patient information according to high ethical standards. The full process of meeting these standards is often made more complicated du...

    journal_title:BMC medical research methodology

    pub_type: 社论

    doi:10.1186/s12874-017-0324-6

    authors: Eagleson R,Altamirano-Diaz L,McInnis A,Welisch E,De Jesus S,Prapavessis H,Rombeek M,Seabrook JA,Park T,Norozi K

    更新日期:2017-03-17 00:00:00

  • Crowding in the emergency department in the absence of boarding - a transition regression model to predict departures and waiting time.

    abstract:BACKGROUND:Crowding in the emergency department (ED) is associated with increased mortality, increased treatment cost, and reduced quality of care. Crowding arises when demand exceed resources in the ED and a first sign may be increasing waiting time. We aimed to quantify predictors for departure from the ED, and relat...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0710-3

    authors: Eiset AH,Kirkegaard H,Erlandsen M

    更新日期:2019-03-29 00:00:00

  • Cross-cultural validation of the Functional Vision Questionnaire for Children and Young People (FVQ_CYP) with visual impairment in the Dutch population: challenges and opportunities.

    abstract:BACKGROUND:To assess cross-cultural validity between Dutch and English versions of the FVQ_CYP, a patient-reported outcome measure developed in the United Kingdom (UK) for children and adolescents with (severe) visual impairment or blindness (VI for brevity) to measure functional vision. METHODS:The 36-item FVQ_CYP wa...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0875-9

    authors: Elsman EBM,Tadić V,Peeters CFW,van Rens GHMB,Rahi JS,van Nispen RMA

    更新日期:2019-12-03 00:00:00

  • Analytical methods used in estimating the prevalence of HIV/AIDS from demographic and cross-sectional surveys with missing data: a systematic review.

    abstract:BACKGROUND:Sero- prevalence studies often have a problem of missing data. Few studies report the proportion of missing data and even fewer describe the methods used to adjust the results for missing data. The objective of this review was to determine the analytical methods used for analysis in HIV surveys with missing ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-00944-w

    authors: Mosha NR,Aluko OS,Todd J,Machekano R,Young T

    更新日期:2020-03-14 00:00:00

  • Joint modelling of time-to-event and multivariate longitudinal outcomes: recent developments and issues.

    abstract:BACKGROUND:Available methods for the joint modelling of longitudinal and time-to-event outcomes have typically only allowed for a single longitudinal outcome and a solitary event time. In practice, clinical studies are likely to record multiple longitudinal outcomes. Incorporating all sources of data will improve the p...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0212-5

    authors: Hickey GL,Philipson P,Jorgensen A,Kolamunnage-Dona R

    更新日期:2016-09-07 00:00:00

  • Mechanisms and pathways to impact in public health research: a preliminary analysis of research funded by the National Institute for Health Research (NIHR).

    abstract:BACKGROUND:The mechanisms and pathways to impacts from public health research in the UK have not been widely studied. Through the lens of one funder (NIHR), our aims are to map the diversity of public health research, in terms of funding mechanisms, disciplinary contributions, and public health impacts, identify exampl...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-0905-7

    authors: Boulding H,Kamenetzky A,Ghiga I,Ioppolo B,Herrera F,Parks S,Manville C,Guthrie S,Hinrichs-Krapels S

    更新日期:2020-02-19 00:00:00

  • Bikeability and methodological issues using the active commuting route environment scale (ACRES) in a metropolitan setting.

    abstract:BACKGROUND:Route environments can positively influence people's active commuting and thereby contribute to public health. The Active Commuting Route Environment Scale (ACRES) was developed to study active commuters' perceptions of their route environments. However, bicycle commuters represent a small portion of the pop...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-6

    authors: Wahlgren L,Schantz P

    更新日期:2011-01-17 00:00:00

  • Estimating the loss of lifetime function using flexible parametric relative survival models.

    abstract:BACKGROUND:Within cancer care, dynamic evaluations of the loss in expectation of life provides useful information to patients as well as physicians. The loss of lifetime function yields the conditional loss in expectation of life given survival up to a specific time point. Due to the inevitable censoring in time-to-eve...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0661-8

    authors: Jakobsen LH,Andersson TM,Biccler JL,El-Galaly TC,Bøgsted M

    更新日期:2019-01-28 00:00:00

  • Evaluating treatments in health care: the instability of a one-legged stool.

    abstract:BACKGROUND:Both scientists and the public routinely refer to randomized controlled trials (RCTs) as being the 'gold standard' of scientific evidence. Although there is no question that placebo-controlled RCTs play a significant role in the evaluation of new pharmaceutical treatments, especially when it is important to ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-65

    authors: Kaplan BJ,Giesbrecht G,Shannon S,McLeod K

    更新日期:2011-05-11 00:00:00

  • Recruiting and motivating black subjects to complete a lengthy survey in a large cohort study: an exploration of different strategies.

    abstract:BACKGROUND:The effectiveness of multiple innovative recruitment strategies for enrolling Black/African American participants to the Adventist Health Study-2 (AHS-2) is described. The study's focus is diet and breast, prostate and colon cancer. METHODS:Promotions centered on trust, relationship building and incentives ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-14-46

    authors: Herring P,Butler T,Hall S,Bennett H,Montgomery SB,Fraser G

    更新日期:2014-04-03 00:00:00

  • Dynamic risk prediction for diabetes using biomarker change measurements.

    abstract:BACKGROUND:Dynamic risk models, which incorporate disease-free survival and repeated measurements over time, might yield more accurate predictions of future health status compared to static models. The objective of this study was to develop and apply a dynamic prediction model to estimate the risk of developing type 2 ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0812-y

    authors: Parast L,Mathews M,Friedberg MW

    更新日期:2019-08-14 00:00:00

  • Norm values and psychometric properties for the German health regulatory focus scale - results of a representative survey.

    abstract:BACKGROUND:The health regulatory focus is an application of Higgins' regulatory focus theory to a health-specific context. It explains individual differences in health motivation, strategies, and behavior. Previous research found the Health Regulatory Focus Scale (HRFS) to be a reliable and valid measure for the constr...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-00927-x

    authors: Schmalbach B,Zenger M,Brähler E,Petrowski K

    更新日期:2020-03-02 00:00:00

  • Error in statistical tests of error in statistical tests.

    abstract:BACKGROUND:A recent paper found that terminal digits of statistical values in Nature deviated significantly from an equiprobable distribution, indicating errors or inconsistencies in rounding. This finding, as well as the discovery that a large percentage of p values were inconsistent with reported test statistics, led...

    journal_title:BMC medical research methodology

    pub_type: 评论,杂志文章

    doi:10.1186/1471-2288-6-45

    authors: Jeng M

    更新日期:2006-09-13 00:00:00

  • Dichotomisation using a distributional approach when the outcome is skewed.

    abstract:BACKGROUND:Dichotomisation of continuous outcomes has been rightly criticised by statisticians because of the loss of information incurred. However to communicate a comparison of risks, dichotomised outcomes may be necessary. Peacock et al. developed a distributional approach to the dichotomisation of normally distribu...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0028-8

    authors: Sauzet O,Ofuya M,Peacock JL

    更新日期:2015-04-24 00:00:00

  • Psychometric properties of a short version of the Job Anxiety Scale.

    abstract:BACKGROUND:Occupational stress and specifically job anxiety are crucial factors in determining health outcomes, job satisfaction as well as performance. In order to assess this phenomenon, the Job Anxiety Scale is one of the instruments available. It consists of 70 items that are clustered in 14 subscales and five dime...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,收录出版

    doi:10.1186/s12874-020-00974-4

    authors: Schmalbach B,Kalkbrenner A,Bassler M,Hinz A,Petrowski K

    更新日期:2020-04-21 00:00:00

  • The use of fasting vs. non-fasting triglyceride concentration for estimating the prevalence of high LDL-cholesterol and metabolic syndrome in population surveys.

    abstract:BACKGROUND:For practical reasons it is not easy to obtain fasting samples in large population health surveys. Non-fasting triglyceride (Tg) values are difficult to interpret. The authors compared the accuracy of statistically corrected non-fasting Tg values with true fasting values and estimated the misclassification o...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-63

    authors: Sundvall J,Leiviskä J,Laatikainen T,Peltonen M,Salomaa V,Vanhala M,Korpi-Hyövälti E,Lauronen J,Alfthan G

    更新日期:2011-05-10 00:00:00

  • Effects of the search technique on the measurement of the change in quality of randomized controlled trials over time in the field of brain injury.

    abstract:BACKGROUND:To determine if the search technique that is used to sample randomized controlled trial (RCT) manuscripts from a field of medical science can influence the measurement of the change in quality over time in that field. METHODS:RCT manuscripts in the field of brain injury were identified using two readily-ava...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-5-7

    authors: Borsody MK,Yamada C

    更新日期:2005-02-07 00:00:00

  • Misclassification of incident conditions using claims data: impact of varying the period used to exclude pre-existing disease.

    abstract:BACKGROUND:Estimating the incidence of medical conditions using claims data often requires constructing a prevalence period that predates an event of interest, for instance the diagnosis of cancer, to exclude those with pre-existing conditions from the incidence risk set. Those conditions missed during the prevalence p...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-32

    authors: Griffiths RI,O'Malley CD,Herbert RJ,Danese MD

    更新日期:2013-03-06 00:00:00

  • Incorporating nonlinearity into mediation analyses.

    abstract:BACKGROUND:Mediation is an important issue considered in the behavioral, medical, and social sciences. It addresses situations where the effect of a predictor variable X on an outcome variable Y is explained to some extent by an intervening, mediator variable M. Methods for addressing mediation have been available for ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0296-6

    authors: Knafl GJ,Knafl KA,Grey M,Dixon J,Deatrick JA,Gallo AM

    更新日期:2017-03-21 00:00:00

  • Network-meta analysis made easy: detection of inconsistency using factorial analysis-of-variance models.

    abstract:BACKGROUND:Network meta-analysis can be used to combine results from several randomized trials involving more than two treatments. Potential inconsistency among different types of trial (designs) differing in the set of treatments tested is a major challenge, and application of procedures for detecting and locating inc...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-14-61

    authors: Piepho HP

    更新日期:2014-05-10 00:00:00

  • The use of relative incidence ratios in self-controlled case series studies: an overview.

    abstract:BACKGROUND:The self-controlled case series (SCCS) is a useful design for investigating associations between outcomes and transient exposures. The SCCS design controls for all fixed covariates, but effect modification can still occur. This can be evaluated by including interaction terms in the model which, when exponent...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0225-0

    authors: Hawken S,Potter BK,Little J,Benchimol EI,Mahmud S,Ducharme R,Wilson K

    更新日期:2016-09-23 00:00:00

  • Validity of information on atopic disease and other illness in young children reported by parents in a prospective birth cohort study.

    abstract:BACKGROUND:The longitudinal birth cohort study is the preferred design for studies of childhood health, particularly atopic disease. Still, prospective data collection depends on recollection of the medical history since the previous visit representing a potential recall-bias. We aimed to ascertain the quality of infor...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-160

    authors: Vissing NH,Jensen SM,Bisgaard H

    更新日期:2012-10-22 00:00:00

  • The need to balance merits and limitations from different disciplines when considering the stepped wedge cluster randomized trial design.

    abstract:BACKGROUND:Various papers have addressed pros and cons of the stepped wedge cluster randomized trial design (SWD). However, some issues have not or only limitedly been addressed. Our aim was to provide a comprehensive overview of all merits and limitations of the SWD to assist researchers, reviewers and medical ethics ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0090-2

    authors: de Hoop E,van der Tweel I,van der Graaf R,Moons KG,van Delden JJ,Reitsma JB,Koffijberg H

    更新日期:2015-10-30 00:00:00

  • Random allocation software for parallel group randomized trials.

    abstract:BACKGROUND:Typically, randomization software should allow users to exert control over the different aspects of randomization including block design, provision of unique identifiers and control over the format and type of program output. While some of these characteristics have been addressed by available software, none...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-4-26

    authors: Saghaei M

    更新日期:2004-11-09 00:00:00

  • High-performing physicians are more likely to participate in a research study: findings from a quality improvement study.

    abstract:BACKGROUND:Participants in voluntary research present a different demographic profile than those who choose not to participate, affecting the generalizability of many studies. Efforts to evaluate these differences have faced challenges, as little information is available from non-participants. Leveraging data from a re...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0809-6

    authors: Dahrouge S,Armstrong CD,Hogg W,Singh J,Liddy C

    更新日期:2019-08-07 00:00:00

  • Comparison of sample characteristics in two pregnancy cohorts: community-based versus population-based recruitment methods.

    abstract:BACKGROUND:One of the biggest challenges for population health studies is the recruitment of participants. Questions that investigators have asked are "who volunteers for studies?" and "does recruitment method influence characteristics of the samples?" The purpose of this paper was to compare sample characteristics of ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-149

    authors: Leung BM,McDonald SW,Kaplan BJ,Giesbrecht GF,Tough SC

    更新日期:2013-12-06 00:00:00

  • Self-reported measures in health research for people with intellectual disabilities: an inclusive pilot study on suitability and reliability.

    abstract:BACKGROUND:The lack of suitable and reliable scales to measure self-reported health and health behaviour among people with intellectual disabilities (ID) is an important methodological challenge in health research. This study, which was undertaken together with co-researchers with ID, explores possibilities for self-re...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0539-1

    authors: Vlot-van Anrooij K,Tobi H,Hilgenkamp TIM,Leusink GL,Naaldenberg J

    更新日期:2018-07-16 00:00:00