Correcting performance-rating errors in oral examinations.

Abstract:

:Although oral examinations are widely used for making decisions regarding an individual's level of competence, they are frequently of limited reliability. A significant part of the error in oral performance ratings is due to the tendency for some evaluators to be lenient and others to be stringent in their assignment of ratings. This article describes and evaluates a simple method to identify and correct for errors of leniency and stringency. The method, which is based on a regression model recommended by Wilson (1988), extends and simplifies the procedures recommended by Cason and Cason (1984, 1985). The method provides an estimate of each individual's performance that has been corrected for errors of leniency and stringency. In addition, it produces for each rater an index of leniency or stringency and several other statistics useful in evaluating the properties of rating data. The regression method is applied to performance ratings from three separate administrations of an oral examination in a medical specialty. The results indicate modest but significant levels of leniency and stringency error; correcting for such errors would change the pass/fail decisions for about 6% of the examinees. Limitations of the procedure, as well as the need for additional research, are discussed.

journal_name

Eval Health Prof

authors

Raymond MR,Webb LC,Houston WM

doi

10.1177/016327879101400107

subject

Has Abstract

pub_date

1991-03-01 00:00:00

pages

100-22

issue

1

eissn

0163-2787

issn

1552-3918

journal_volume

14

pub_type

杂志文章
  • Integrating performance measure data into the Joint Commission accreditation process.

    abstract::This article describes the Joint Commission's implementation plans, experience, and results to date of incorporating performance measurement data into the accreditation process. These plans have evolved in response to changes in the health care environment, feedback from accredited organizations, and both technical an...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/016327879902200301

    authors: Braun BI,Koss RG,Loeb JM

    更新日期:1999-09-01 00:00:00

  • Translation and Psychometric Evaluation of the Mahoney Pain Scale Among Iranian Elderly With Dementia: A Methodological Study.

    abstract::Valid and reliable measurement tools are necessary for pain assessment among the elderly with dementia. This study aimed to translate the Mahoney Pain Scale (MPS) into Persian and evaluate its psychometric properties among Iranian elderly with dementia. In this methodological study, after translating and assessing the...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278720961818

    authors: Zare M,Tagharrobi Z,Sharifi K,Sooki Z,Abolhasani J

    更新日期:2020-09-29 00:00:00

  • Using focus groups to assess primary care patients' satisfaction.

    abstract::The present report describes the use of patient focus groups by a primary health care facility. We review our rationale for using focus groups and the process we used to prepare for and conduct them. We then highlight the results and lessons learned through this experience. Focus groups can be an excellent method for ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/01632780022034480

    authors: Schwarz M,Landis SE,Rowe JE,Janes CL,Pullman N

    更新日期:2000-03-01 00:00:00

  • Adherence to practice guidelines, clinical outcomes, and costs among Medicaid enrollees with severe mental illnesses.

    abstract::The treatment of Medicaid enrollees diagnosed with depression or schizophrenia was examined to determine whether adherence to treatment guideline was associated with health care financing strategy, clinical outcomes, and cost-effectiveness. Individuals in a fee-for-service condition were significantly more likely to r...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278708328744

    authors: Stiles PG,Boothroyd RA,Dhont K,Beiler PF,Green AE

    更新日期:2009-03-01 00:00:00

  • Clinical guidelines and practice: a commentary on the complexity of tinnitus management.

    abstract::Subjective tinnitus is an enigmatic and chronic condition that is predominantly managed as symptomatic. Little high-level evidence exists for the efficacy and specificity of the various tinnitus management strategies currently used, and this is reflected in documents that aim to guide clinicians. As a consequence, the...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278710390355

    authors: Hoare DJ,Hall DA

    更新日期:2011-12-01 00:00:00

  • Development of 7-Item Perceived Meaning of Activity in Housing (PMA-H-7) to Assess Opportunities for Meaningful Activities in the Supported Housing Context for People With Psychiatric Disabilities.

    abstract::People with psychiatric disabilities need to be able to express their views about their support, for example, the provision of meaningful activity in supported accommodation. Brief scales may be necessary to avoid strain and discomfort for this group. The aim was to develop a brief version of Perceived Meaning in Acti...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278719845036

    authors: Eklund M,Brunt D

    更新日期:2020-12-01 00:00:00

  • Experiencing satisfaction and stress as an AIDS care provider. The AIDS caregiver scale.

    abstract::Health care workers (n = 342) completed items examining their emotional experiences working with HIV-positive patients. Factor analysis produced two subscales: Personal Satisfaction and Emotional Stress. A subset of these care workers (n = 251) claimed increased stress and depression after 6 months. In a second study,...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327879301600303

    authors: Ferrari JR,McCown W,Pantano J

    更新日期:1993-09-01 00:00:00

  • Are positive alternative medical therapy trials credible?: Evidence from four high-impact medical journals.

    abstract::Forty-five complementary and alternative medicine (CAM) efficacy randomized controlled trials (RCTs) from high-impact medical journals (NEJM, JAMA, Annals of Internal Medicine, and Archives of Internal Medicine) were reviewed based on their meeting three validity criteria (the existence of a placebo control, moderate ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278709346810

    authors: Barker Bausell R

    更新日期:2009-12-01 00:00:00

  • Factors that influence health risk behaviors among tanning salon patrons.

    abstract::A principal components analysis identified the immediate convenience of indoor tanning (i.e., relaxation, perceived attractiveness of tans) as a major factor that motivates people to visit commercial tanning salons frequently and continually. Long-term considerations, such as price, socializing, and perceived health, ...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327879702000401

    authors: Beasley TM,Kittel BS

    更新日期:1997-12-01 00:00:00

  • Serendipity and pseudoscience. A look at health-related program evaluation.

    abstract::This article presents a discussion of the pitfalls of sloppy science as it is applied to the evaluation of human service programs. The differences are examined between use of the scientific method and trial-and-error approaches to assessment. The major requirements for sound research are presented, as are the pitfalls...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327879301600401

    authors: Holmes D,Teresi J,Ory M

    更新日期:1993-12-01 00:00:00

  • Student Perceptions of Collaboration Skills in an Interprofessional Context: Development and Initial Validation of the Self-Assessed Collaboration Skills Instrument.

    abstract::An integral component of interprofessional education (IPE) is the development of a collaboration-ready health-care workforce. While collaboration is a fundamental element of IPE, there is no existing measure of collaboration skills that is not context specific. This article describes the development and initial valida...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278717752438

    authors: Hinyard L,Toomey E,Eliot K,Breitbach A

    更新日期:2019-12-01 00:00:00

  • Physicians and DRGs. Hospital management alternatives.

    abstract::Rising health care costs in general and hospital costs in particular have placed significant financial strain on federal programs such as Medicare. It was with this in mind that the Reagan administration in 1983 initiated Medicare's new prospective payment system based on diagnostic-related groups (DRGs) to reduce the...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878801100405

    authors: Patel K

    更新日期:1988-12-01 00:00:00

  • Translation of evidence-based tobacco use prevention programming in Croatia.

    abstract::Croatia faces a major threat of smoking-related health burdens given the monumental changes this newly formed country has undergone in the past 15 years because of its postcommunist democratic and economic transitions and the destructive 1991-1995 war with Serbia. This article provides information on Croatia's general...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278708320151

    authors: Skara S,Kovacic L,Civljak M,Voncina L

    更新日期:2008-09-01 00:00:00

  • Data sources for health care quality evaluation.

    abstract::The central purpose of this article is to review the major approaches to evaluation of health care quality in terms of its use of different data sources. The areas of application, advantages, and disadvantages of seven data sources are presented, including direct observation, clinical records, record abstracts, clinic...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878300600302

    authors: Baker F

    更新日期:1983-09-01 00:00:00

  • A measure of perceived burden among caregivers.

    abstract::This report describes an easily administered scale for measuring perceived burden among caregivers of family members with dementia. During home interviews conducted in 1991, 82 caregivers rated their family member with dementia on several items related to functional ability, the type of care provided, and associated s...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:

    authors: Macera CA,Eaker ED,Jannarone RJ,Davis DR,Stoskopf CH

    更新日期:1993-06-01 00:00:00

  • Assessment of communication barriers in community pharmacies.

    abstract::This study identified previously reported facilitators and barriers to pharmacist-client communication and then evaluated their impact on the observed communication behaviors of pharmacists. Pharmacists (n = 100) completed a seven-page questionnaire collecting information on 11 variables that had been organized accord...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278703258104

    authors: Paluck EC,Green LW,Frankish CJ,Fielding DW,Haverkamp B

    更新日期:2003-12-01 00:00:00

  • Patient characteristics of provider survey respondents: no evidence of nonresponse bias.

    abstract::Surveys of physicians are an important tool to assess opinions and self-reported behaviors of this policy-relevant population. However, this population is notoriously difficult to survey and plagued with low and falling response rates. In order to evaluate the potential import of response rate, we examine the presence...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278711435542

    authors: Ziegenfuss JY,Shah ND,Fan J,Houten HK,Deming JR,Smith SA,Beebe TJ

    更新日期:2012-12-01 00:00:00

  • Advocacy through convenient definition of societal objectives. Cost-benefit analyses in health care.

    abstract::In modern democracies, societal objectives are pluralistic, varied, and often mutually conflicting. Consequently, cost-benefit analysts have considerable scope for defining societal objectives in a manner most suitable for advocating their clients' programs. Three different therapies for a hypothetical disease are con...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878200500401

    authors: Joglekar P

    更新日期:1982-12-01 00:00:00

  • Validity of the International Physical Activity Questionnaire-Short Form for Application in Asian Countries: A Study in Vietnam.

    abstract::This is the first study in the Asia-Pacific region to examine the criterion validity of the self-reported International Physical Activity Questionnaire-Short form (IPAQ-SF) using accelerometers, in terms of achieving the World Health Organization's (WHO) recommended physical activity guidelines for health benefits. Vi...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278718819708

    authors: Tran VD,Do VV,Pham NM,Nguyen CT,Xuong NT,Jancey J,Lee AH

    更新日期:2020-06-01 00:00:00

  • Health care workers in the Dominican Republic: self-perceived role in smoking cessation.

    abstract::A Dominican Republic (DR)-based multi-community trial of smoking cessation viewed health care workers (HCWs) as potential interventionists. Effectively engaging them requires a clear understanding of their attitudes and practices regarding smoking. A Rapid Assessment Procedure, conducted among HCWs in six economically...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278709333152

    authors: Dozier AM,Ossip DJ,Diaz S,Sierra-Torres E,Quiñones de Monegro Z,Armstrong L,Chin NP,McIntosh S

    更新日期:2009-06-01 00:00:00

  • A research agenda for licensing and certification testing validation studies.

    abstract::The set of articles appearing in this issue of Evaluation & the Health Professions provides an in-depth examination of the validation process for licensing and certification test score decisions. The winter 1990 issue of Educational Measurement: Issues and Practice also contains an excellent set of articles about lice...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/016327879401700208

    authors: Haladyna TM

    更新日期:1994-06-01 00:00:00

  • Needs assessment and hospice planning in a rural setting.

    abstract::A needs assessment is a research and planning activity designed to determine a community's service needs and utilization patterns. One of the most practical ways to plan services is to gather information accurately about a defined population and use that information to revise existing programs or develop new ones. Unf...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878400700206

    authors: Byrne CM

    更新日期:1984-06-01 00:00:00

  • Methodological issues for health-related surveys of multicultural older women.

    abstract::Given concerns about survey nonresponse bias as well as the need to plan resources for participant recruitment, this study tracked each step of the recruitment process (location, response, consent, and completion) of sociodemographically diverse older women for a survey concerning mammography experience. Younger, less...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327879401700408

    authors: Zapka JG,Chasan-Taber L,Bigelow C,Hurley T

    更新日期:1994-12-01 00:00:00

  • Improving Our Ability to Evaluate Underlying Mechanisms of Behavioral Onset and Other Event Occurrence Outcomes: A Discrete-Time Survival Mediation Model.

    abstract::The purpose of this article is to introduce and describe a statistical model that researchers can use to evaluate underlying mechanisms of behavioral onset and other event occurrence outcomes. Specifically, the article develops a framework for estimating mediation effects with outcomes measured in discrete-time epochs...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278713512124

    authors: Fairchild AJ,Abara WE,Gottschall AC,Tein JY,Prinz RJ

    更新日期:2015-09-01 00:00:00

  • Factors shaping physicians' willingness to accommodate medication requests.

    abstract::Medical decisions, including physicians' prescribing behaviors, are shaped by a complex interplay of clinical and nonclinical factors. We aim to determine how physician, patient, and relationship characteristics influence physicians' decisions to accommodate brand-name prescription drug requests. We applied multivaria...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278712468756

    authors: Arney J,Street RL Jr,Naik AD

    更新日期:2014-09-01 00:00:00

  • Linking Existing Instruments to Develop an Activity of Daily Living Item Bank.

    abstract::This study examined dimensionality and item-level psychometric properties of an item bank measuring activities of daily living (ADL) across inpatient rehabilitation facilities and community living centers. Common person equating method was used in the retrospective veterans data set. This study examined dimensionality...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278716676873

    authors: Li CY,Romero S,Bonilha HS,Simpson KN,Simpson AN,Hong I,Velozo CA

    更新日期:2018-03-01 00:00:00

  • The reliability of peer assessments. A meta-analysis.

    abstract::A meta-analysis of studies examining the interrater reliability of the standard practice of peer assessments of quality of care was conducted. Using the Medline, Health Planning and Administration, and SCISEARCH databases, the English-language literature from 1966 through 1991 was searched for studies of chance correc...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,meta分析

    doi:10.1177/016327879401700101

    authors: Goldman RL

    更新日期:1994-03-01 00:00:00

  • Selective reporting of adjusted estimates in observational epidemiology studies: reasons and implications for meta-analyses.

    abstract::For meta-analyses of observational epidemiology studies, unadjusted and adjusted study estimates are often extracted. However, there is evidence of selective reporting of adjusted study estimates. We investigate adjustment reporting bias, examining the reasons why some studies do not contribute an adjusted estimate to...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章,评审

    doi:10.1177/0163278708324438

    authors: Peters J,Mengersen K

    更新日期:2008-12-01 00:00:00

  • Involving people with severe mental illness in evaluation and performance improvement.

    abstract::This article argues for the increased involvement of people with severe mental illness and consumers of other health and mental health services in evaluation and performance improvement in the organizations from which they receive services. Among other reasons, consumers can bring a different perspective to the select...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/0163278702025003003

    authors: Linhorst DM,Eckert A

    更新日期:2002-09-01 00:00:00

  • Evaluation of a hospice program: effects on terminally ill patients and their families.

    abstract::Hospice care in the United States has grown rapidly since its introduction here from England in the mid-1970s. Surprisingly little evaluation of services has been made public, particularly with regard not merely to demographics but also the actual effect of hospice caregiving on patients and families. This article des...

    journal_title:Evaluation & the health professions

    pub_type: 杂志文章

    doi:10.1177/016327878100400305

    authors: Silver S

    更新日期:1981-09-01 00:00:00