Quantifying error in OSCE standard setting for varying cohort sizes: A resampling approach to measuring assessment quality.

Abstract:

BACKGROUND:The use of the borderline regression method (BRM) is a widely accepted standard setting method for OSCEs. However, it is unclear whether this method is appropriate for use with small cohorts (e.g. specialist post-graduate examinations). AIMS AND METHODS:This work uses an innovative application of resampling methods applied to four pre-existing OSCE data sets (number of stations between 17 and 21) from two institutions to investigate how the robustness of the BRM changes as the cohort size varies. Using a variety of metrics, the 'quality' of an OSCE is evaluated for cohorts of approximately n = 300 down to n = 15. Estimates of the standard error in station-level and overall pass marks, R(2) coefficient, and Cronbach's alpha are all calculated as cohort size varies. RESULTS AND CONCLUSION:For larger cohorts (n > 200), the standard error in the overall pass mark is small (less than 0.5%), and for individual stations is of the order of 1-2%. These errors grow as the sample size reduces, with cohorts of less than 50 candidates showing unacceptably large standard error. Alpha and R(2) also become unstable for small cohorts. The resampling methodology is shown to be robust and has the potential to be more widely applied in standard setting and medical assessment quality assurance and research.

journal_name

Med Teach

journal_title

Medical teacher

authors

Homer M,Pell G,Fuller R,Patterson J

doi

10.3109/0142159X.2015.1029898

subject

Has Abstract

pub_date

2016-01-01 00:00:00

pages

181-8

issue

2

eissn

0142-159X

issn

1466-187X

journal_volume

38

pub_type

杂志文章
  • Evaluation of a computer program ('disect') to consolidate anatomy knowledge: a randomised-controlled trial.

    abstract:BACKGROUND:The teaching of anatomy to medical undergraduates continues to develop. Medical imaging can accurately demonstrate anatomy. 'disect' is a computer program which manipulates and reconstructs real CT images in 3-D. AIM:To implement and assess a novel computer-based imaging resource. METHODS:Third-year underg...

    journal_title:Medical teacher

    pub_type: 杂志文章,随机对照试验

    doi:10.3109/01421590903144110

    authors: Tam MD,Hart AR,Williams SM,Holland R,Heylings D,Leinster S

    更新日期:2010-01-01 00:00:00

  • Pharmacases.de - a student-centered e-learning project of clinical pharmacology.

    abstract:AIM:The aim of the project Pharmacases.de was to develop an innovative concept for creating high-quality e-learning content which integrates and promotes the theoretical and cooperative skills of final-year medical students and is easily adoptable by cooperating institutes and hospitals. METHODS AND RESULTS:A peer-tea...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.759642

    authors: Zollner B,Sucha M,Berg C,Muß N,Amann P,Amann-Neher B,Oestreicher D,Engelhardt S,Sarikas A

    更新日期:2013-01-01 00:00:00

  • The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12.

    abstract:BACKGROUND:Portfolios in post-graduate healthcare education are used to support reflective practice, deliver summative assessment, aid knowledge management processes and are seen as a key connection between learning at organisational and individual levels. This systematic review draws together the evidence on the effec...

    journal_title:Medical teacher

    pub_type: 杂志文章,评审

    doi:10.1080/01421590902883056

    authors: Tochel C,Haig A,Hesketh A,Cadzow A,Beggs K,Colthart I,Peacock H

    更新日期:2009-04-01 00:00:00

  • Assessing competencies using milestones along the way.

    abstract::This paper presents perspectives and controversies surrounding the use of milestones to assess competency in outcomes-based medical education. Global perspectives (Canada, Europe, and the United States) and developments supporting their rationales are discussed. In Canada, there is a significant movement away from con...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2014.993954

    authors: Tekian A,Hodges BD,Roberts TE,Schuwirth L,Norcini J

    更新日期:2015-04-01 00:00:00

  • A comprehensive checklist for reporting the use of OSCEs.

    abstract:BACKGROUND:The Objective Structured Clinical Examination (OSCE) has experienced an explosion of use which has rarely been accompanied by systematic investigations on its validity, reliability and feasibility. A systematic review of OSCE was undertaken as part of Best Evidence Medical Education at the Centre for Evidenc...

    journal_title:Medical teacher

    pub_type: 杂志文章,评审

    doi:10.1080/01421590802578277

    authors: Patricio M,Juliao M,Fareleira F,Young M,Norman G,Vaz Carneiro A

    更新日期:2009-02-01 00:00:00

  • Preparing teachers for competency-based medical education: Fundamental teaching activities.

    abstract:INTRODUCTION:The move to competency-based medical education has created new challenges for medical teachers, including the need to reflect on and further develop their own competencies as teachers. Guidance is needed to ensure comprehensive and coherent programs of faculty development to meet the needs of teachers. ME...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/0142159X.2017.1394998

    authors: Walsh A,Koppula S,Antao V,Bethune C,Cameron S,Cavett T,Clavet D,Dove M

    更新日期:2018-01-01 00:00:00

  • Medical school accreditation in China: a Sino-Australian collaboration.

    abstract::In 2008, China established a medical school accreditation process based on international standards and guidelines. Twenty schools had been accredited by 2013 and it is intended to accredit all 137 schools by 2020. To achieve this ambitious aim, Chinese medical educators have entered into collaboration with their Austr...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2014.917286

    authors: Geffen L,Cheng B,Field M,Zhao S,Walters T,Yang L

    更新日期:2014-11-01 00:00:00

  • Connectivism: A knowledge learning theory for the digital age?

    abstract:BACKGROUND:The emergence of the internet, particularly Web 2.0 has provided access to the views and opinions of a wide range of individuals opening up opportunities for new forms of communication and knowledge formation. Previous ways of navigating and filtering available information are likely to prove ineffective in ...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2016.1173661

    authors: Goldie JG

    更新日期:2016-10-01 00:00:00

  • 'They've all got to learn'. Medical students' learning from patients in ambulatory (outpatient and general practice) consultations.

    abstract:BACKGROUND:The dynamics of effective teaching consultations need to be better understood. AIM:Find from medical students, patients and doctors how to optimize learning in ambulatory consultations. METHODS:Patients and students independently gave semi-structured exit interviews after 25 ambulatory teaching consultatio...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590802464445

    authors: Ashley P,Rhodes N,Sari-Kouzel H,Mukherjee A,Dornan T

    更新日期:2009-02-01 00:00:00

  • Performance of paediatric Senior House Officers following changes in recruitment.

    abstract:BACKGROUND:Recruitment to Paediatric Senior House Officer (SHO) posts has traditionally been undertaken by each individual hospital trust. AIMS:This study looks at whether shortlisting and interview scores from a Central Paediatric SHO recruitment process are a reliable predicator of performance. METHODS:A questionna...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590701509670

    authors: Bindal T,Wall D,Goodyear HM

    更新日期:2007-06-01 00:00:00

  • Effects of interactive instructional techniques in a web-based peripheral nervous system component for human anatomy.

    abstract:BACKGROUND:This study investigated the effects of interactive instructional techniques in a web-based peripheral nervous system (PNS) component of a first year medical school human anatomy course. METHOD:Existing data from 9 years of instruction involving 856 students were used to determine (1) the effect of web-based...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590701753518

    authors: Allen EB,Walls RT,Reilly FD

    更新日期:2008-02-01 00:00:00

  • How to develop a competency-based examination blueprint for longitudinal standardized patient clinical skills assessments.

    abstract:BACKGROUND:Objective Structured Clinical Exams (OSCEs) with standardized patients (SPs) are commonly used in medical education to assess learners' clinical skills. However, assessments are often discrete rather than intentionally developmentally sequenced. AIMS:We developed an examination blueprint to optimize assessm...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.809408

    authors: Mookherjee S,Chang A,Boscardin CK,Hauer KE

    更新日期:2013-11-01 00:00:00

  • The elusive content of the medical-school curriculum: a method to the madness.

    abstract::A major problem for curriculum and course planners is coping simultaneously with the expanding knowledge base and having less time to teach. A widely used solution is to include huge amounts of information in the curriculum. A better solution is to identify a manageable core of relevant knowledge. One way is to begin ...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590500237598

    authors: D'Eon M,Crawford R

    更新日期:2005-12-01 00:00:00

  • The effects of utilizing a near-patient e-learning tool on medical student learning.

    abstract:CONTEXT:This study aimed to develop a near-patient, e-learning tool and explore student views on how utilization of such a tool influenced their learning. METHODS:Third year medical students from Monash University in Melbourne, Australia were invited to trial a novel, near-patient, e-learning tool in two separate pilo...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2014.955842

    authors: Selzer R,Tallentire VR,Foley F

    更新日期:2015-01-01 00:00:00

  • Health inequities: the need for action by schools of medicine.

    abstract:BACKGROUND:It is well recognised that marked inequalities in mortality and morbidity exist between populations particularly those in lower socio-economic groups, including Indigenous and some ethnic minorities. Academic medicine has not yet articulated a clear stance on reducing health inequity within communities. AIM...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590801948042

    authors: Sanson-Fisher RW,Williams N,Outram S

    更新日期:2008-01-01 00:00:00

  • Using medical knowledge sources on handheld computers--a qualitative study among junior doctors.

    abstract:BACKGROUND:The emergence of mobile computing could have an impact on how junior doctors learn. To exploit this opportunity it is essential to understand their information seeking process. AIM:To explore junior doctors' experiences of using medical knowledge sources on handheld computers. METHOD:Interviews with five S...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590701507294

    authors: Axelson C,Wårdh I,Strender LE,Nilsson G

    更新日期:2007-09-01 00:00:00

  • Creating a test blueprint for a progress testing program: A paired-comparisons approach.

    abstract:CONTEXT:Creating a new testing program requires the development of a test blueprint that will determine how the items on each test form are distributed across possible content areas and practice domains. To achieve validity, categories of a blueprint are typically based on the judgments of content experts. How experts ...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/0142159X.2017.1403015

    authors: von Bergmann H,Childs RA

    更新日期:2018-03-01 00:00:00

  • Psychometrics of the scale of attitudes toward physician-pharmacist collaboration: a study with medical students.

    abstract:BACKGROUND:Despite the emphasis placed on interdisciplinary education and interprofessional collaboration between physicians and pharmacologists, no psychometrically sound instrument is available to measure attitudes toward collaborative relationships. AIM:This study was designed to examine psychometrics of an instrum...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2012.714877

    authors: Hojat M,Spandorfer J,Isenberg GA,Vergare MJ,Fassihi R,Gonnella JS

    更新日期:2012-01-01 00:00:00

  • The year in a training practice: what has lasting value? Grounded theoretical categories and dimensions from a pilot study.

    abstract::Experienced General Practitioners were asked about lasting value from GP Registrar training. Grounded theory was used to construct eight categories of training behaviours. In each category, dimensions spanned between learner-preferred behaviours and those the learners perceived as less helpful. Learners valued teachin...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590020031138

    authors: Peile EB,Easton GP,Johnson N

    更新日期:2001-03-01 00:00:00

  • What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research.

    abstract::Workplace-based assessment is more commonly given a lukewarm than a warm welcome by its prospective users. In this article, we summarise the workplace-based assessment literature as well as our own experiences with workplace-based assessment to derive lessons that can facilitate acceptance of workplace-based assessmen...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.798403

    authors: Driessen E,Scheele F

    更新日期:2013-07-01 00:00:00

  • Bridging the global health training gap: Design and evaluation of a new clinical global health course at Harvard Medical School.

    abstract:BACKGROUND:Medical student and resident interest in global health has been growing rapidly. Meanwhile, educational opportunities for trainees remain limited, and many trainees participate in global health experiences abroad without adequate preparation. Medical institutions are attempting to respond to this training ga...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2011.577122

    authors: Nelson BD,Saltzman A,Lee PT

    更新日期:2012-01-01 00:00:00

  • Clinical teaching as part of continuing professional development: Does teaching enhance clinical performance?

    abstract:INTRODUCTION:Physicians identify teaching as a factor that enhances performance, although existing data to support this relationship is limited. PURPOSE:To determine whether there were differences in clinical performance scores as assessed through multisource feedback (MSF) data based on clinical teaching. METHODS:MS...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2015.1112895

    authors: Lockyer JM,Hodgson CS,Lee T,Faremo S,Fisher B,Dafoe W,Yiu V,Violato C

    更新日期:2016-08-01 00:00:00

  • Teaching doctors to treat doctors: medical student peer counselling.

    abstract::Physicians' emotional problems need to be recognized and treated. Intervention and prevention in this problem area have been attempted at the Medical College of Wisconsin through a programme of peer counselling designed to teach student physicians how to recognize and treat emotional difficulties faced by their peers....

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/01421598009072181

    authors: Spiro JH,Roenneburg M,Maly BJ

    更新日期:1980-01-01 00:00:00

  • Student perceptions about the occurrence of critical incidents in tutorial groups.

    abstract::The study reported here examines student perceptions about the occurrence of critical incidents in tutorial groups across years of medical training. The study investigates the following research questions. (1) Which factors underlie the occurrence of critical incidents in the tutorial group? (2) How do students rate t...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/0142159002005596

    authors: de Grave WS,Dolmans DH,van Der Vleuten CP

    更新日期:2001-01-01 00:00:00

  • Self-perceived attitudes and skills of cultural competence: a comparison of family medicine and internal medicine residents.

    abstract::This study surveyed resident perceptions of competent cross-cultural doctor-patient communication as a step toward developing an integrative primary care cross-cultural curriculum. Respondents were 57 first-, second- and third-year residents in family medicine (FA) and internal medicine (IMA) who completed a questionn...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/0142159031000100454

    authors: Shapiro J,Hollingshead J,Morrison E

    更新日期:2003-05-01 00:00:00

  • Experience of establishment of multiple mini structure interview as part of student admission policy at Faculty of Medicine, King Abdulaziz University, 2011-2012.

    abstract::Faculty of Medicine (FOM), King Abdulaziz University (KAU), requested for international recognition by the Laison Committee of Canadian Medical Education (LCME) during the period 2008-2010. Selection of medical students was a must standard in LCME. After obtaining a written permission from higher administration at KAU...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.765543

    authors: El Says F,Ayuob N,Fahmy AR,El Fayez F,Hasanian M,El Deek B

    更新日期:2013-01-01 00:00:00

  • German undergraduate medical students' attitudes and needs regarding medical errors and patient safety--a national survey in Germany.

    abstract:INTRODUCTION:Errors in medicine and patient safety are topics with growing scientific and public attention. In undergraduate medical education, these issues are little investigated so far. The aim of this study was to collect data regarding attitudes and needs of medical students. METHODS:In a sample of 269 German med...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2014.891008

    authors: Kiesewetter J,Kager M,Lux R,Zwissler B,Fischer MR,Dietz I

    更新日期:2014-06-01 00:00:00

  • "I have the right to a private life": medical students' views about professionalism in a digital world.

    abstract:BACKGROUND:Social media site use is ubiquitous, particularly Facebook. Postings on social media can have an impact on the perceived professionalism of students and practitioners. AIMS:In this study, we explored the attitudes and understanding of undergraduate medical students towards professionalism, with a specific f...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.802301

    authors: Ross S,Lai K,Walton JM,Kirwan P,White JS

    更新日期:2013-10-01 00:00:00

  • Twelve tips for making the best use of feedback.

    abstract:BACKGROUND:Feedback is generally regarded as crucial for learning. We focus on feedback provided through instruments developed to inform self-assessment and support learners to improve performance. These instruments are being used commonly in medical education, but they are ineffective if the feedback is not well recei...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.3109/0142159X.2013.769676

    authors: van der Leeuw RM,Slootweg IA

    更新日期:2013-05-01 00:00:00

  • An innovative model for teaching complex clinical procedures: integration of standardised patients into ward round training for final year students.

    abstract:OBJECTIVES:Ward rounds are an essential activity for doctors in hospital settings and represent complex tasks requiring not only medical knowledge but also communication skills, clinical technical skills, patient management skills and team-work skills. However, although the need for ward round training is emphasized in...

    journal_title:Medical teacher

    pub_type: 杂志文章

    doi:10.1080/01421590701299264

    authors: Nikendei C,Kraus B,Lauber H,Schrauth M,Weyrich P,Zipfel S,Jünger J,Briem S

    更新日期:2007-03-01 00:00:00