Abstract:
:Although oral examinations are widely used for making decisions regarding an individual's level of competence, they are frequently of limited reliability. A significant part of the error in oral performance ratings is due to the tendency for some evaluators to be lenient and others to be stringent in their assignment of ratings. This article describes and evaluates a simple method to identify and correct for errors of leniency and stringency. The method, which is based on a regression model recommended by Wilson (1988), extends and simplifies the procedures recommended by Cason and Cason (1984, 1985). The method provides an estimate of each individual's performance that has been corrected for errors of leniency and stringency. In addition, it produces for each rater an index of leniency or stringency and several other statistics useful in evaluating the properties of rating data. The regression method is applied to performance ratings from three separate administrations of an oral examination in a medical specialty. The results indicate modest but significant levels of leniency and stringency error; correcting for such errors would change the pass/fail decisions for about 6% of the examinees. Limitations of the procedure, as well as the need for additional research, are discussed.
journal_name
Eval Health Profjournal_title
Evaluation & the health professionsauthors
Raymond MR,Webb LC,Houston WMdoi
10.1177/016327879101400107subject
Has Abstractpub_date
1991-03-01 00:00:00pages
100-22issue
1eissn
0163-2787issn
1552-3918journal_volume
14pub_type
杂志文章abstract::This article describes the Joint Commission's implementation plans, experience, and results to date of incorporating performance measurement data into the accreditation process. These plans have evolved in response to changes in the health care environment, feedback from accredited organizations, and both technical an...
journal_title:Evaluation & the health professions
pub_type: 杂志文章,评审
doi:10.1177/016327879902200301
更新日期:1999-09-01 00:00:00
abstract::Valid and reliable measurement tools are necessary for pain assessment among the elderly with dementia. This study aimed to translate the Mahoney Pain Scale (MPS) into Persian and evaluate its psychometric properties among Iranian elderly with dementia. In this methodological study, after translating and assessing the...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278720961818
更新日期:2020-09-29 00:00:00
abstract::The present report describes the use of patient focus groups by a primary health care facility. We review our rationale for using focus groups and the process we used to prepare for and conduct them. We then highlight the results and lessons learned through this experience. Focus groups can be an excellent method for ...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/01632780022034480
更新日期:2000-03-01 00:00:00
abstract::The treatment of Medicaid enrollees diagnosed with depression or schizophrenia was examined to determine whether adherence to treatment guideline was associated with health care financing strategy, clinical outcomes, and cost-effectiveness. Individuals in a fee-for-service condition were significantly more likely to r...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278708328744
更新日期:2009-03-01 00:00:00
abstract::Subjective tinnitus is an enigmatic and chronic condition that is predominantly managed as symptomatic. Little high-level evidence exists for the efficacy and specificity of the various tinnitus management strategies currently used, and this is reflected in documents that aim to guide clinicians. As a consequence, the...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278710390355
更新日期:2011-12-01 00:00:00
abstract::People with psychiatric disabilities need to be able to express their views about their support, for example, the provision of meaningful activity in supported accommodation. Brief scales may be necessary to avoid strain and discomfort for this group. The aim was to develop a brief version of Perceived Meaning in Acti...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278719845036
更新日期:2020-12-01 00:00:00
abstract::Health care workers (n = 342) completed items examining their emotional experiences working with HIV-positive patients. Factor analysis produced two subscales: Personal Satisfaction and Emotional Stress. A subset of these care workers (n = 251) claimed increased stress and depression after 6 months. In a second study,...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327879301600303
更新日期:1993-09-01 00:00:00
abstract::Forty-five complementary and alternative medicine (CAM) efficacy randomized controlled trials (RCTs) from high-impact medical journals (NEJM, JAMA, Annals of Internal Medicine, and Archives of Internal Medicine) were reviewed based on their meeting three validity criteria (the existence of a placebo control, moderate ...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278709346810
更新日期:2009-12-01 00:00:00
abstract::A principal components analysis identified the immediate convenience of indoor tanning (i.e., relaxation, perceived attractiveness of tans) as a major factor that motivates people to visit commercial tanning salons frequently and continually. Long-term considerations, such as price, socializing, and perceived health, ...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327879702000401
更新日期:1997-12-01 00:00:00
abstract::This article presents a discussion of the pitfalls of sloppy science as it is applied to the evaluation of human service programs. The differences are examined between use of the scientific method and trial-and-error approaches to assessment. The major requirements for sound research are presented, as are the pitfalls...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327879301600401
更新日期:1993-12-01 00:00:00
abstract::An integral component of interprofessional education (IPE) is the development of a collaboration-ready health-care workforce. While collaboration is a fundamental element of IPE, there is no existing measure of collaboration skills that is not context specific. This article describes the development and initial valida...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278717752438
更新日期:2019-12-01 00:00:00
abstract::Rising health care costs in general and hospital costs in particular have placed significant financial strain on federal programs such as Medicare. It was with this in mind that the Reagan administration in 1983 initiated Medicare's new prospective payment system based on diagnostic-related groups (DRGs) to reduce the...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327878801100405
更新日期:1988-12-01 00:00:00
abstract::Croatia faces a major threat of smoking-related health burdens given the monumental changes this newly formed country has undergone in the past 15 years because of its postcommunist democratic and economic transitions and the destructive 1991-1995 war with Serbia. This article provides information on Croatia's general...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278708320151
更新日期:2008-09-01 00:00:00
abstract::The central purpose of this article is to review the major approaches to evaluation of health care quality in terms of its use of different data sources. The areas of application, advantages, and disadvantages of seven data sources are presented, including direct observation, clinical records, record abstracts, clinic...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327878300600302
更新日期:1983-09-01 00:00:00
abstract::This report describes an easily administered scale for measuring perceived burden among caregivers of family members with dementia. During home interviews conducted in 1991, 82 caregivers rated their family member with dementia on several items related to functional ability, the type of care provided, and associated s...
journal_title:Evaluation & the health professions
pub_type: 杂志文章,评审
doi:
更新日期:1993-06-01 00:00:00
abstract::This study identified previously reported facilitators and barriers to pharmacist-client communication and then evaluated their impact on the observed communication behaviors of pharmacists. Pharmacists (n = 100) completed a seven-page questionnaire collecting information on 11 variables that had been organized accord...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278703258104
更新日期:2003-12-01 00:00:00
abstract::Surveys of physicians are an important tool to assess opinions and self-reported behaviors of this policy-relevant population. However, this population is notoriously difficult to survey and plagued with low and falling response rates. In order to evaluate the potential import of response rate, we examine the presence...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278711435542
更新日期:2012-12-01 00:00:00
abstract::In modern democracies, societal objectives are pluralistic, varied, and often mutually conflicting. Consequently, cost-benefit analysts have considerable scope for defining societal objectives in a manner most suitable for advocating their clients' programs. Three different therapies for a hypothetical disease are con...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327878200500401
更新日期:1982-12-01 00:00:00
abstract::This is the first study in the Asia-Pacific region to examine the criterion validity of the self-reported International Physical Activity Questionnaire-Short form (IPAQ-SF) using accelerometers, in terms of achieving the World Health Organization's (WHO) recommended physical activity guidelines for health benefits. Vi...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278718819708
更新日期:2020-06-01 00:00:00
abstract::A Dominican Republic (DR)-based multi-community trial of smoking cessation viewed health care workers (HCWs) as potential interventionists. Effectively engaging them requires a clear understanding of their attitudes and practices regarding smoking. A Rapid Assessment Procedure, conducted among HCWs in six economically...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278709333152
更新日期:2009-06-01 00:00:00
abstract::The set of articles appearing in this issue of Evaluation & the Health Professions provides an in-depth examination of the validation process for licensing and certification test score decisions. The winter 1990 issue of Educational Measurement: Issues and Practice also contains an excellent set of articles about lice...
journal_title:Evaluation & the health professions
pub_type: 杂志文章,评审
doi:10.1177/016327879401700208
更新日期:1994-06-01 00:00:00
abstract::A needs assessment is a research and planning activity designed to determine a community's service needs and utilization patterns. One of the most practical ways to plan services is to gather information accurately about a defined population and use that information to revise existing programs or develop new ones. Unf...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327878400700206
更新日期:1984-06-01 00:00:00
abstract::Given concerns about survey nonresponse bias as well as the need to plan resources for participant recruitment, this study tracked each step of the recruitment process (location, response, consent, and completion) of sociodemographically diverse older women for a survey concerning mammography experience. Younger, less...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327879401700408
更新日期:1994-12-01 00:00:00
abstract::The purpose of this article is to introduce and describe a statistical model that researchers can use to evaluate underlying mechanisms of behavioral onset and other event occurrence outcomes. Specifically, the article develops a framework for estimating mediation effects with outcomes measured in discrete-time epochs...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278713512124
更新日期:2015-09-01 00:00:00
abstract::Medical decisions, including physicians' prescribing behaviors, are shaped by a complex interplay of clinical and nonclinical factors. We aim to determine how physician, patient, and relationship characteristics influence physicians' decisions to accommodate brand-name prescription drug requests. We applied multivaria...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278712468756
更新日期:2014-09-01 00:00:00
abstract::This study examined dimensionality and item-level psychometric properties of an item bank measuring activities of daily living (ADL) across inpatient rehabilitation facilities and community living centers. Common person equating method was used in the retrospective veterans data set. This study examined dimensionality...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278716676873
更新日期:2018-03-01 00:00:00
abstract::A meta-analysis of studies examining the interrater reliability of the standard practice of peer assessments of quality of care was conducted. Using the Medline, Health Planning and Administration, and SCISEARCH databases, the English-language literature from 1966 through 1991 was searched for studies of chance correc...
journal_title:Evaluation & the health professions
pub_type: 杂志文章,meta分析
doi:10.1177/016327879401700101
更新日期:1994-03-01 00:00:00
abstract::For meta-analyses of observational epidemiology studies, unadjusted and adjusted study estimates are often extracted. However, there is evidence of selective reporting of adjusted study estimates. We investigate adjustment reporting bias, examining the reasons why some studies do not contribute an adjusted estimate to...
journal_title:Evaluation & the health professions
pub_type: 杂志文章,评审
doi:10.1177/0163278708324438
更新日期:2008-12-01 00:00:00
abstract::This article argues for the increased involvement of people with severe mental illness and consumers of other health and mental health services in evaluation and performance improvement in the organizations from which they receive services. Among other reasons, consumers can bring a different perspective to the select...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/0163278702025003003
更新日期:2002-09-01 00:00:00
abstract::Hospice care in the United States has grown rapidly since its introduction here from England in the mid-1970s. Surprisingly little evaluation of services has been made public, particularly with regard not merely to demographics but also the actual effect of hospice caregiving on patients and families. This article des...
journal_title:Evaluation & the health professions
pub_type: 杂志文章
doi:10.1177/016327878100400305
更新日期:1981-09-01 00:00:00