Validity Evidence for a Residency Admissions Standardized Assessment Letter for Pediatrics.

Abstract:

:Construct: This study aims to provide validity evidence for the standardized Assessment Letter for Pediatrics as a measure of competencies expected of a 1st-year pediatrics resident as part of a pediatric residency admissions process. BACKGROUND:The Narrative Letter of Recommendation is a frequently used tool in the residency admissions process even though it has poor interrater reliability, lacks pertinent content, and does not correlate with residency performance. A newer tool, the Standardized Letter, has shown validity evidence for content and interrater reliability in other specialties. We sought to develop and provide validity evidence for the standardized Assessment Letter for Pediatrics. APPROACH:All 2012 and 2013 applicants invited to interview at the University of Geneva Pediatrics Residency Program provided 2 standardized Assessment Letters. Content for the letter was based on CanMEDS roles and ratings of 6 desired competencies and an overall assessment. Validity evidence was gathered for internal structure (Cronbach's alpha and generalizability), response process (interrater reliability with intraclass correlation), relations to other variables (Pearson's correlation coefficient), and consequences (logistic regression to predict admission). RESULTS:One hundred fourteen faculty completed 142 standardized Assessment Letters for 71 applicants. Average overall assessment was 3.0 of 4 (SD = 0.59). Cronbach's alpha was 0.93. The G-coefficient was 0.59. The decision study projected that four Assessment Letters are needed to attain a G-coefficient of 0.73. Applicant variance (28.5%) indicated high applicant differentiation. The Assessment Letter intraclass coefficient was 0.51, 95% confidence interval (CI) [0.43, 0.59]. Assessment Letter scores were correlated with the structured interview (r = .28), 95% CI [0.05, 0.51]; global rating (r = .36), 95% CI [0.13, 0.58]; and admissions decision (r = .25), 95% CI [0.02, 0.46]. Assessment Letter scores did not predict the admissions decision (odds ratio = 1.67, p = .37) after controlling for the unique contribution of the structured interview and global rating scores. CONCLUSION:Validity evidence supports use of the Assessment Letter for Pediatrics; future studies should refine items to improve predictive validity and explore how to best integrate the Assessment Letter into the residency admissions process.

journal_name

Teach Learn Med

authors

Bajwa NM,Yudkowsky R,Belli D,Vu NV,Park YS

doi

10.1080/10401334.2017.1367297

subject

Has Abstract

pub_date

2018-04-01 00:00:00

pages

173-183

issue

2

eissn

1040-1334

issn

1532-8015

journal_volume

30

pub_type

杂志文章
  • Mindfulness Meditation and Interprofessional Cardiopulmonary Resuscitation: A Mixed-Methods Pilot Study.

    abstract:PROBLEM:Mindfulness training includes mindfulness meditation, which has been shown to improve both attention and self-awareness. Medical providers in the intensive care unit often deal with difficult situations with strong emotions, life-and-death decisions, and both interpersonal and interprofessional conflicts. The e...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2018.1462186

    authors: Kelm DJ,Ridgeway JL,Gas BL,Mohan M,Cook DA,Nelson DR,Benzo RP

    更新日期:2018-10-01 00:00:00

  • A Proposed Shared Vision for Leadership Development for all Medical Students: A Call from a Coalition of Diverse Medical Schools.

    abstract::Issue: Despite clear relevance, need, descriptive literature, and student interest, few schools offer required curriculum to develop leadership skills. This paper outlines a proposed shared vision for leadership development drawn from a coalition of diverse medical schools. We advocate that leadership development is a...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2020.1754835

    authors: Mangrulkar RS,Tsai A,Cox SM,Halaas GW,Nelson EA,Nesse RE,Silvestri RC,Radabaugh CL,Skochelak S,Beck Dallaghan GL,Steiner B

    更新日期:2020-10-01 00:00:00

  • Ambulatory morning report: an underutilized educational modality.

    abstract:BACKGROUND:Many medicine residency training programs include a lecture-based preclinic conference series as part of the ambulatory educational curriculum when more effective teaching formats might be available. Our institution has replaced this lecture-based teaching format with an ambulatory morning report modeled aft...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/S15328015TLM1301_9

    authors: Demopoulos B,Pelzman F,Wenderoth S

    更新日期:2001-01-01 00:00:00

  • Attending Emergency Physicians' Perceptions of a Programmatic Workplace-Based Assessment System: The McMaster Modular Assessment Program (McMAP).

    abstract::Construct: The McMaster Modular Assessment Program (McMAP) is a programmatic workplace-based assessment (WBA) system that provides emergency medicine trainees with competency judgments through frequent task-specific and global daily assessments. Background: The longevity of McMAP relative to other programmatic WBA sys...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2019.1574581

    authors: Acai A,Li SA,Sherbino J,Chan TM

    更新日期:2019-08-01 00:00:00

  • e-Professionalism: a new frontier in medical education.

    abstract:BACKGROUND:This article, prepared by the Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee, discusses the evolving challenges facing medical educators posed by social media and a new form of professionalism that has been termed e-professionalism. SUMMARY:E-professionalism...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2013.770741

    authors: Kaczmarczyk JM,Chuang A,Dugoff L,Abbott JF,Cullimore AJ,Dalrymple J,Davis KR,Hueppchen NA,Katz NT,Nuthalapaty FS,Pradhan A,Wolf A,Casey PM

    更新日期:2013-01-01 00:00:00

  • A novel advance directives course provides a transformative learning experience for medical students.

    abstract:BACKGROUND:Inadequate physician training may be a barrier to physician-patient discussions of advance directives (ADs). DESCRIPTION:The purpose of this study was to determine the effects of an ADs course, which includes completing a directive, on medical students' perceived knowledge of and preparedness for discussing...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401331003656678

    authors: Mueller PS,Litin SC,Hook CC,Creagan ET,Cha SS,Beckman TJ

    更新日期:2010-04-01 00:00:00

  • Medical students' experience with psychiatric diagnoses in a family medicine clerkship.

    abstract:BACKGROUND:Many medical schools require a family medicine clerkship, yet little is known about the quantity and diversity of the diagnoses encountered by the students. PURPOSE:This study examines patients encountered with psychiatric diagnoses using quantitative data collected by students in a family practice clerkshi...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/S15328015TLM1303_6

    authors: O'Hara BS,Saywell RM Jr,Smidley JA,Burba JL,Thakker N,Bogdewic SP,Zollinger TW

    更新日期:2001-07-01 00:00:00

  • Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study.

    abstract:UNLABELLED:CONSTRUCT: Authentic standard setting methods will demonstrate high convergent validity evidence of their outcomes, that is, cutoff scores and pass/fail decisions, with most other methods when compared with each other. BACKGROUND:The objective structured clinical examination (OSCE) was established for valid...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2015.1044749

    authors: Yousuf N,Violato C,Zuberi RW

    更新日期:2015-01-01 00:00:00

  • Code status discussions: agreement between internal medicine residents and hospitalized patients.

    abstract:BACKGROUND:Medical residents frequently participate in code status discussions, yet their competency in this role has not been evaluated. PURPOSE:The objective is to determine the quality of code status discussions from the perspective of both resident and patient. METHODS:We conducted a cross-sectional survey of con...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2010.512537

    authors: Loertscher LL,Beckman TJ,Cha SS,Reed DA

    更新日期:2010-10-01 00:00:00

  • Designing a National Longitudinal Faculty Development Curriculum Focused on Educational Scholarship: Process, Outcomes, and Lessons Learned.

    abstract:PROBLEM:Clinical educators at U.S. academic health centers are frequently disadvantaged in the academic promotion system, lacking needed faculty development, mentoring, and networking support. INTERVENTION:In 2006, we implemented the national Educational Scholars Program to offer faculty development in educational sch...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2017.1282370

    authors: Chandran L,Gusic ME,Lane JL,Baldwin CD

    更新日期:2017-07-01 00:00:00

  • Grading Practices and Distributions Across Internal Medicine Clerkships.

    abstract:THEORY:Clerkship evaluation and grading practices vary widely between U.S. medical schools. Grade inflation continues to exist, and grade distribution is likely to be different among U.S. medical schools. HYPOTHESES:Increasing the number of available grades curtails "grade inflation." METHOD:A national survey of all ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2016.1164605

    authors: Fazio SB,Torre DM,DeFer TM

    更新日期:2016-07-01 00:00:00

  • Teaching geriatric assessment in home visits: the family physician/geriatrician attachment.

    abstract:BACKGROUND:Geriatric clinical clerkships in Israel teach mostly about the hospitalized elder patient with almost no ambulatory experience. Meanwhile, primary care physicians provide most of the health care to the elderly in the community. This article describes an innovation in the curriculum of the 5th-year family med...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/S15328015TLM1502_09

    authors: Tandeter H,Peleg R,Menahem S,Biderman A,Fried VA

    更新日期:2003-04-01 00:00:00

  • Building Collaboration: A Scoping Review of Cultural Competency and Safety Education and Training for Healthcare Students and Professionals in Canada.

    abstract::Phenomenon: This scoping literature review summarizes current Canadian health science education and training aimed to lessen health gaps between Aboriginal and non-Aboriginal peoples. APPROACH:Keyword searches of peer-reviewed and gray literature databases, websites, and resources recommended by local Aboriginal comm...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章,评审

    doi:10.1080/10401334.2016.1234960

    authors: Guerra O,Kurtz D

    更新日期:2017-04-01 00:00:00

  • The Development and Impact of a Social Media and Professionalism Course for Medical Students.

    abstract:PROBLEM:Inappropriate social media behavior can have detrimental effects on students' future opportunities, but medical students are given little opportunity to reflect upon ways of integrating their social media identities with their newly forming professional identities. INTERVENTION:In 2012, a required educational ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2016.1275971

    authors: Gomes AW,Butera G,Chretien KC,Kind T

    更新日期:2017-07-01 00:00:00

  • Is medicine altruistic? A query from the medical school admissions office.

    abstract:BACKGROUND:Applicants to medical school often state that they are motivated by a desire to help others. Admissions officers must evaluate these claims, but assessment of altruism is difficult and imprecise. SUMMARY:A purely utilitarian moral philosophy denies the possibility of altruism. However, the Enlightenment phi...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/s15328015tlm1801_10

    authors: Bardes CL

    更新日期:2006-01-01 00:00:00

  • Pimping: perspectives of 4th year medical students.

    abstract:BACKGROUND:A well-known phenomenon among U.S. medical students known as pimping, or the pedagogical device of questioning students in the clinical setting, receives virtually no attention in medical literature. PURPOSE:Identifying 4th-year medical students' relevant knowledge and attitudes about pimping may help educa...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/s15328015tlm1702_14

    authors: Wear D,Kokinova M,Keck-McNulty C,Aultman J

    更新日期:2005-04-01 00:00:00

  • Development of a communication curriculum for emergency medicine residents.

    abstract:BACKGROUND:The Accreditation Council for Graduate Medical Education requires residency training programs to develop methods to teach and assess communication skills in residents to ensure competence as a practitioner. In response, we piloted a communication curriculum for emergency medicine residents. We describe the c...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401330903228604

    authors: Sullivan C,Ellison SR,Quaintance J,Arnold L,Godrey P

    更新日期:2009-10-01 00:00:00

  • A novel student-led approach to multiple-choice question generation and online database creation, with targeted clinician input.

    abstract:PROBLEM:Multiple-choice questions (MCQs) are the main method of assessing medical student knowledge. As a result there is a high demand from medical students for formative MCQs. However, teaching staff rarely have the time or incentive to develop high-quality formative questions, focusing instead on material for high-s...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2015.1011651

    authors: Harris BH,Walsh JL,Tayyaba S,Harris DA,Wilson DJ,Smith PE

    更新日期:2015-01-01 00:00:00

  • How good is good? Students and assessors' perceptions of qualitative markers of performance.

    abstract:BACKGROUND:Qualitative markers of performance are routinely used for medical student assessment, though the extent to which such markers can be readily translated to actionable pieces of information remains uncertain. PURPOSE:To explore (a) the perceived value to be indicated by descriptor phrases commonly used for de...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2012.741545

    authors: Ma HK,Min C,Neville A,Eva K

    更新日期:2013-01-01 00:00:00

  • Effect of bedside physical diagnosis training on third-year medical students' physical exam skills.

    abstract:BACKGROUND:Graduating medical students, when surveyed, noted a deficit in training in physical examination skills. PURPOSES:In an attempt to remedy this deficit we implemented a pilot program for 3rd-year medical students consisting of twice-weekly bedside diagnosis rounds as part of their 8-week medicine clerkship. ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2013.857329

    authors: Roberts L,Lu WH,Go RA,Daroowalla F

    更新日期:2014-01-01 00:00:00

  • Proceedings of the 2008 Annual Predoctoral Education Conference of the Society of Teachers of Family Medicine (STFM).

    abstract::The Society of Teachers of Family Medicine (STFM; http://www.stfm.org) is a community of professionals devoted to teaching family medicine through undergraduate, graduate, and continuing medical education. This multidisciplinary group of physicians, educators, behavioral scientists, and researchers works to further ST...

    journal_title:Teaching and learning in medicine

    pub_type:

    doi:10.1080/10401330802384946

    authors: Little DN,Hatch RL

    更新日期:2008-10-01 00:00:00

  • A "medical student practice profile" as the foundation for a case-based curriculum revision.

    abstract:BACKGROUND:In 2001, the Keck School of Medicine of the University of Southern California initiated a major curriculum reform with fully integrated teaching of the basic sciences. DESCRIPTION:The new curriculum integrated a set of selected clinical cases called the student practice profile (SPP). The SPP cases were des...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2010.512834

    authors: Abbott A,Sullivan M,Nyquist J,Mylona E,Taylor C

    更新日期:2010-10-01 00:00:00

  • Relationship between dean's letter rankings and later evaluations by residency program directors.

    abstract:BACKGROUND:It is not known how well dean's letter rankings predict later performance in residency. PURPOSE:To assess the accuracy of dean's letter rankings to predict clinical performance in internship. METHOD:Participants were medical students who graduated from the University of Rochester School of Medicine and Den...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401330701366523

    authors: Lurie SJ,Lambert DR,Grady-Weliky TA

    更新日期:2007-07-01 00:00:00

  • The effect of resident duty-hours restrictions on internal medicine clerkship experiences: surveys of medical students and clerkship directors.

    abstract:UNLABELLED:PHENOMENON: Medical students receive much of their inpatient teaching from residents who now experience restructured teaching services to accommodate the 2011 duty-hour regulations (DHR). The effect of DHR on medical student educational experiences is unknown. We examined medical students' and clerkship dire...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2014.979187

    authors: Kogan JR,Lapin J,Aagaard E,Boscardin C,Aiyer MK,Cayea D,Cifu A,Diemer G,Durning S,Elnicki M,Fazio SB,Khan AR,Lang VJ,Mintz M,Nixon LJ,Paauw D,Torre DM,Hauer KE

    更新日期:2015-01-01 00:00:00

  • First aid and basic life support skills training early in the medical curriculum: curriculum issues, outcomes, and confidence of students.

    abstract:BACKGROUND:The importance of training physicians and other health care professionals in first aid and basic life support (BLS) is now widely recognized. The Faculty of Medicine and Health Sciences of the United Arab Emirates University have been offering formal training in these skills to 1st-year medical students sinc...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/S15328015TLM1304_05

    authors: Das M,Elzubeir M

    更新日期:2001-10-01 00:00:00

  • A comparison of in-class learner engagement across lecture, problem-based learning, and team learning using the STROBE classroom observation tool.

    abstract:BACKGROUND:Having recently introduced team learning into the preclinical medical curriculum, evidence of the relative impact of this instructional method on in-class learner engagement was sought. PURPOSE:To compare patterns of engagement behaviors among learners in class sessions across 3 distinct instructional metho...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/s15328015tlm1702_4

    authors: Kelly PA,Haidet P,Schneider V,Searle N,Seidel CL,Richards BF

    更新日期:2005-04-01 00:00:00

  • Examining the medical student body: peer physical exams and genital, rectal, or breast exams.

    abstract:BACKGROUND:Some medical schools have replaced all student practice peer physical examinations in the curriculum with examinations of standardized patients (SPs). PURPOSE:To assess attitudes of medical students toward practicing physical examinations with classmates, including genital, rectal, or female breast componen...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1207/s15328015tlm1704_5

    authors: Power DV,Center BA

    更新日期:2005-10-01 00:00:00

  • Authorship Order in Medical Education Publications: In Search of Practical Guidance for the Community.

    abstract::Phenomenon: With scholarly collaborations come questions about the order of authorship. Authorship order is an important consideration because it often used as an indicator of seniority, expertise, leadership, and scholarly productivity. As a result, authorship order factors into decisions about hiring, salary, resour...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2018.1533836

    authors: Mavis B,Durning SJ,Uijtdehaage S

    更新日期:2019-06-01 00:00:00

  • Impact of group size on the effectiveness of a resuscitation simulation curriculum for medical students.

    abstract:BACKGROUND:Simulation requires involvement from participants. However, it is unknown to what extent simulation effectiveness is a function of the number of participants. PURPOSE:This study assessed the impact of varying group size on medical students' subjective experience of simulation and on postsimulation exam perf...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章,随机对照试验

    doi:10.1080/10401334.2011.586920

    authors: Rezmer J,Begaz T,Treat R,Tews M

    更新日期:2011-07-01 00:00:00

  • Evaluation of the University of Virginia Leadership in Academic Medicine Program.

    abstract:BACKGROUND:Developing effective transformational leaders is essential for a diverse and complex academic medical system. The purpose of this study is to elicit faculty perspectives on their leadership skills before and after participation in a Leadership in Academic Medicine (LAM) Program. SUMMARY:The two-part study c...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章

    doi:10.1080/10401334.2011.611773

    authors: Sanfey H,Harris I,Pollart S,Schwartz A

    更新日期:2011-10-01 00:00:00