The reliability of peer assessments. A meta-analysis.

Abstract:

:A meta-analysis of studies examining the interrater reliability of the standard practice of peer assessments of quality of care was conducted. Using the Medline, Health Planning and Administration, and SCISEARCH databases, the English-language literature from 1966 through 1991 was searched for studies of chance corrected agreement among peer reviewers. The weighted mean kappa of 21 independent findings from 13 studies was .31. Comparison of this result with widely used standards suggests that the interrater reliability of peer assessment is quite limited and needs improvement. Research needs to be directed at modifying the peer review process to improve its reliability or at identifying indexes of quality with sufficient validity and reliability that they can be employed without subsequent peer review.

journal_name

Eval Health Prof

authors

Goldman RL

doi

10.1177/016327879401700101

subject

Has Abstract

pub_date

1994-03-01 00:00:00

pages

3-21

issue

1

eissn

0163-2787

issn

1552-3918

journal_volume

17

pub_type

杂志文章,meta分析
  • Hidden bias in the use of archival data.

    abstract::Nonresponses in archival data may violate the missing-at-random assumption in ways difficult to detect. Standard methods of comparing sociodemographics of respondents and nonrespondents are inappropriate when the units of analysis are not also the individuals who maintain the archival record. Under these circumstances...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878701000202

    authors: Dworkin RJ

    更新日期:1987-06-01 00:00:00

  • A comparison of empirical ranking methods of frequency and severity ratings of clinical presentations.

    abstract::This study compares five methods of ranking Likert ratings on frequency and severity scales. Data were drawn from an international online survey conducted as part of a practice analysis with 91 diplomates of the American Chiropractic Board of Radiology. A total of 129 clinical presentations had been rated on two scale...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278711425041

    authors: Smith SD,Beran TN

    更新日期:2012-12-01 00:00:00

  • Surveying clinicians by web: current issues in design and administration.

    abstract::The versatility, speed, and reduced costs with which web surveys can be conducted with clinicians are often offset by low response rates. Drawing on best practices and general recommendations in the literature, we provide an evidence-based overview of methods for conducting online surveys with providers. We highlight ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278713496630

    authors: Dykema J,Jones NR,Piché T,Stevenson J

    更新日期:2013-09-01 00:00:00

  • Can survey data be used to estimate physician practice costs?

    abstract::Health policy makers rely on survey estimates of physician practice costs to set reimbursement rates. The Health Care Financing Administration has just funded a multimillion dollar effort to collect survey data that will be used to revise payments under the Medicare Fee Schedule. However, the ability of large-scale ph...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327879601900102

    authors: Berk ML,Mueller CD,Thran S

    更新日期:1996-03-01 00:00:00

  • Evaluation of a hospice program: effects on terminally ill patients and their families.

    abstract::Hospice care in the United States has grown rapidly since its introduction here from England in the mid-1970s. Surprisingly little evaluation of services has been made public, particularly with regard not merely to demographics but also the actual effect of hospice caregiving on patients and families. This article des...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878100400305

    authors: Silver S

    更新日期:1981-09-01 00:00:00

  • Type II translation: transporting prevention interventions from research to real-world settings.

    abstract::This article summarizes research on Type II translation of prevention interventions aimed at enhancing the adoption of effective programs and practices in communities. The primary goal of Type II translation is to institutionalize evidence-based programs, products, and services. First, the authors describe theoretical...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278706290408

    authors: Rohrbach LA,Grana R,Sussman S,Valente TW

    更新日期:2006-09-01 00:00:00

  • Rethinking how to measure organizational culture in the hospital setting. The Hospital Culture Scale.

    abstract::Like all organizations, health care delivery systems must be concerned with understanding the implicit beliefs, values, and assumptions extant within the organization that ubiquitously motivate and shape the behavior of participating members. The Hospital Culture Scale (HCS) was designed as a way to assess the unique ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/016327879501800205

    authors: Klingle RS,Burgoon M,Afifi W,Callister M

    更新日期:1995-06-01 00:00:00

  • Selective reporting of adjusted estimates in observational epidemiology studies: reasons and implications for meta-analyses.

    abstract::For meta-analyses of observational epidemiology studies, unadjusted and adjusted study estimates are often extracted. However, there is evidence of selective reporting of adjusted study estimates. We investigate adjustment reporting bias, examining the reasons why some studies do not contribute an adjusted estimate to...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/0163278708324438

    authors: Peters J,Mengersen K

    更新日期:2008-12-01 00:00:00

  • Context effects. Powerful therapies or methodological bias?

    abstract::This article provides an analysis of the way in which placebo effects could be investigated and taken into account, supported by a description of some of the major articles that have been published on this topic. Based on conceptual, theoretical, and methodological issues, the authors highlight some of the reasons for...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/0163278703026002003

    authors: Di Blasi Z,Kleijnen J

    更新日期:2003-06-01 00:00:00

  • Measuring patient perceptions of the patient-doctor interaction. Development of the PDIS.

    abstract::Patient satisfaction has been related to patient compliance, doctor-patient information exchange, and continuity of care. However, the construct of patient satisfaction is not well defined; its measurement has typically consisted of the anecdotal generation of questions that may not tap issues of concern to patients. ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878400700106

    authors: Smith JK,Falvo D,McKillip J,Pitz G

    更新日期:1984-03-01 00:00:00

  • Periodic competency review. An assessment and policy model.

    abstract::Periodic review of professional competence or performance is on the public agenda as a response to rising malpractice suits, iatrogenic morbidity, and mortality resulting directly from professional incompetence, and the inadequacy of current licensing and discipline systems. This article reviews several continuing com...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878701000306

    authors: Mandelbaum L

    更新日期:1987-09-01 00:00:00

  • Mail versus internet surveys: determinants of method of response preferences among health professionals.

    abstract::The authors evaluated determinants of response to Internet-based surveys in a sample (n = 5600) of Texas healthcare professionals. Participants were given the option of responding by mail or over the Web (response, 66%). Internet respondents were younger (p < .001), had worked fewer years in healthcare (p < .001), and...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278707300634

    authors: Lusk C,Delclos GL,Burau K,Drawhorn DD,Aday LA

    更新日期:2007-06-01 00:00:00

  • Factors shaping physicians' willingness to accommodate medication requests.

    abstract::Medical decisions, including physicians' prescribing behaviors, are shaped by a complex interplay of clinical and nonclinical factors. We aim to determine how physician, patient, and relationship characteristics influence physicians' decisions to accommodate brand-name prescription drug requests. We applied multivaria...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278712468756

    authors: Arney J,Street RL Jr,Naik AD

    更新日期:2014-09-01 00:00:00

  • Recommendations for care related to follow-up of abnormal cancer screening tests: accuracy of patient report.

    abstract::When using patient self-report of processes of care as part of measuring quality performance, validity and reliability are important considerations. In this study, the congruence of patient report of recommendations of screening follow-up care was compared with record audit data. Survey data were collected from a rand...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278705278273

    authors: Puleo E,Zapka JG,Goins KV,Yood MU,Mouchawar J,Manos M,Somkin C,Taplin S

    更新日期:2005-09-01 00:00:00

  • Outpatient utilization patterns and quality outcomes after first acute episode of mental health hospitalization. Is some better than none, and is more service associated with better outcomes?

    abstract::Access to outpatient services within the first 30 days after an inpatient mental health episode may influence relapse risk. A retrospective cohort of 3,755 adult Medicaid mental health inpatients discharged from their first managed care acute episode of care from July 1, 1996, through May 20, 1998, were studied. Resul...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/01632780022034714

    authors: Huff ED

    更新日期:2000-12-01 00:00:00

  • The opinions of injecting drug user (IDUs) HIV patients and health professionals on access to antiretroviral treatment and health services in Valencia, Spain.

    abstract::The benefits of HIV treatment (Highly Active Antiretroviral Therapy [HAART]) have been less apparent in injecting drug users (IDUs), most probably as a result of poor adherence to treatment. We explored factors related to HIV treatment adherence as reported by 23 IDU-HIV patients and nine health professionals from hea...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278711401743

    authors: Garcia de la Hera M,Davo MC,Ballester-Añón R,Vioque J

    更新日期:2011-09-01 00:00:00

  • When is knowledge ripe for primary care? An exploratory study on the meaning of evidence.

    abstract::The objectives of this study were to explore the meaning of scientific evidence as it is understood by primary care physicians. Individual interviews were conducted with actors chosen for their roles in the production and use of knowledge: 22 family physicians, 13 specialist physicians, and 6 researchers. Two situatio...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278707311870

    authors: Beaulieu MD,Proulx M,Jobin G,Kugler M,Gossard F,Denis JL,Larouche D

    更新日期:2008-03-01 00:00:00

  • Are positive alternative medical therapy trials credible?: Evidence from four high-impact medical journals.

    abstract::Forty-five complementary and alternative medicine (CAM) efficacy randomized controlled trials (RCTs) from high-impact medical journals (NEJM, JAMA, Annals of Internal Medicine, and Archives of Internal Medicine) were reviewed based on their meeting three validity criteria (the existence of a placebo control, moderate ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278709346810

    authors: Barker Bausell R

    更新日期:2009-12-01 00:00:00

  • Chronic fatigue syndrome: sociodemographic subtypes in a community-based sample.

    abstract::Most chronic fatigue syndrome (CFS) studies are based on information about patients from primary or tertiary care settings. These patients might not be typical of patients in the general population. This investigation involved examinations of individuals with CFS from a community-based study. A random sample of 18,675...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/01632780022034598

    authors: Jason LA,Taylor RR,Kennedy CL,Jordan K,Song S,Johnson DE,Torres SR

    更新日期:2000-09-01 00:00:00

  • Utilization of behavioral knowledge from short-term workshops.

    abstract::Workshops on new methods in the health professions have often been evaluated for participant reaction and understanding of the material. But behavior change on the job following workshops has seldom been assessed. Rehabilitation professionals in seven states were surveyed to determine use of behavior management proced...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327877900200404

    authors: Marr JN,Greenwood R

    更新日期:1979-12-01 00:00:00

  • Translation of evidence-based tobacco use prevention programming in Croatia.

    abstract::Croatia faces a major threat of smoking-related health burdens given the monumental changes this newly formed country has undergone in the past 15 years because of its postcommunist democratic and economic transitions and the destructive 1991-1995 war with Serbia. This article provides information on Croatia's general...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278708320151

    authors: Skara S,Kovacic L,Civljak M,Voncina L

    更新日期:2008-09-01 00:00:00

  • Common metrics to assess the efficiency of clinical research.

    abstract::The Clinical and Translational Science Award (CTSA) program was designed by the National Institutes of Health (NIH) to develop processes and infrastructure for clinical and translational research throughout the United States. The CTSA initiative now funds 61 institutions. In 2012, the National Center for Advancing Tra...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278713499586

    authors: Rubio DM

    更新日期:2013-12-01 00:00:00

  • From the general to the specific. Using meta-analytic reports in clinical decision making.

    abstract::The purpose of this article is to describe how health practitioners can interpret the results from published meta-analytic reports of intervention effectiveness and efficacy studies, and then communicate those results, in a manner that helps patients make critical clinical and life decisions. The problem of using rese...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/01632780122034939

    authors: Tickle-Degnen L

    更新日期:2001-09-01 00:00:00

  • Recruitment strategies. Pharmacists' participation in an evaluation project to dispense emergency contraception.

    abstract::The objective of this article was to describe the effectiveness of a multifocus recruitment strategy to a pilot project allowing direct provision of emergency contraception (EC) in a community pharmacy through collaborative agreements between pharmacists and physicians. The project recruited pharmacies through direct ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278703261202

    authors: Cockerill R,Cohen M,Dunn S,Brown T

    更新日期:2004-03-01 00:00:00

  • Follow-up for medical care among drug users with hepatitis C.

    abstract::Prevalence of hepatitis C (HCV) in injection drug users (IDUs) is high and once HCV has been detected, follow-up medical care is essential. Six hundred and one current and former IDUs who tested positive for HCV antibodies received referrals for medical care. Twenty-four percent (147) of participants returned to be in...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278706296003

    authors: Reynolds GL,Fisher DG,Jaffe A,Edwards J

    更新日期:2006-12-01 00:00:00

  • Longitudinal outcomes of a comprehensive, incentivized worksite wellness program.

    abstract::This study evaluated health benefits of long-term participation in an employer-based wellness program, focusing on selected chronic disease risk factors. A repeated measures longitudinal time-series study was conducted of 8 years of existing prospectively collected annual data, including clinical measures of weight, b...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278710379222

    authors: Neville BH,Merrill RM,Kumpfer KL

    更新日期:2011-03-01 00:00:00

  • A residential continuum for the chronically mentally ill: a Markov probability model.

    abstract::The movement of a group of chronically mentally ill clients among various facilities for residential care was described with a Markov probability model. The results of a goodness-of-fit test indicated that the model accurately captured the flow of clients among the state psychiatric hospital, the in-patient units, the...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878100400108

    authors: Drachman D

    更新日期:1981-03-01 00:00:00

  • Convergence between cluster analysis and the Angoff method for setting minimum passing scores on credentialing examinations.

    abstract::Cluster analysis can be a useful statistical technique for setting minimum passing scores on high-stakes examinations by grouping examinees into homogenous clusters based on their responses to test items. It has been most useful for supplementing data or validating minimum passing scores determined from expert judgmen...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278707307904

    authors: Hess B,Subhiyah RG,Giordano C

    更新日期:2007-12-01 00:00:00

  • Assessing costs and potential returns of evidence-based programs for seniors.

    abstract::The authors describe the customary tools used by health services researchers to conduct economic evaluations of health interventions. Recognizing the inherent challenges of these tools for utilization in contemporary public health practice, we recommend a practical cost-benefit analysis (PCBA) to allow public health p...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278710393955

    authors: Miller TR,Dickerson JB,Smith ML,Ory MG

    更新日期:2011-06-01 00:00:00

  • Scope, Influence, and Interdisciplinary Collaboration: The Publication Portfolio of the NIH Clinical and Translational Science Awards (CTSA) Program From 2006 Through 2017.

    abstract::The Clinical and Translational Science Awards (CTSA) program sponsors an array of innovative, collaborative research. This study uses complementary bibliometric approaches to assess the scope, influence, and interdisciplinary collaboration of publications supported by single CTSA hubs and those supported by multiple h...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278719839435

    authors: Llewellyn N,Carter DR,DiazGranados D,Pelfrey C,Rollins L,Nehl EJ

    更新日期:2020-09-01 00:00:00