An Extended Validity Argument for Assessing Feedback Culture.


UNLABELLED:NEGEA 2015 CONFERENCE ABSTRACT (EDITED): Measuring an Organization's Culture of Feedback: Can It Be Done? Steven Rougas and Brian Clyne. CONSTRUCT: This study sought to develop a construct for measuring formative feedback culture in an academic emergency medicine department. Four archetypes (Market, Adhocracy, Clan, Hierarchy) reflecting an organization's values with respect to focus (internal vs. external) and process (flexibility vs. stability and control) were used to characterize one department's receptiveness to formative feedback. The prevalence of residents' identification with certain archetypes served as an indicator of the department's organizational feedback culture. BACKGROUND:New regulations have forced academic institutions to implement wide-ranging changes to accommodate competency-based milestones and their assessment. These changes challenge residencies that use formative feedback from faculty as a major source of data for determining training advancement. Though various approaches have been taken to improve formative feedback to residents, there currently exists no tool to objectively measure the organizational culture that surrounds this process. Assessing organizational culture, commonly used in the business sector to represent organizational health, may help residency directors gauge their program's success in fostering formative feedback. The Organizational Culture Assessment Instrument (OCAI) is widely used, extensively validated, applicable to survey research, and theoretically based and may be modifiable to assess formative feedback culture in the emergency department. APPROACH:Using a modified Delphi technique and several iterations of focus groups amongst educators at one institution, four of the original six OCAI domains (which each contain 4 possible responses) were modified to create a 16-item Formative Feedback Culture Tool (FFCT) that was administered to 26 residents (response rate = 55%) at a single academic emergency medicine department. The mean score of each item on the FFCT (range = 0-100) was analyzed. Convergent and divergent properties of the four archetypes were assessed using a multitrait-multimethod matrix of Pearson's coefficients. Expecting that items in one archetype would diverge from the others, whereas items within an archetype should have strong convergent properties, convergent validity was assessed by comparing items across domains that all related to the same archetype. Similarly, divergent validity was assessed by comparing the correlation of items within an archetype to the correlations of those items within a hetero-domain block (i.e., to other items within the same domain). RESULTS:Three of the four domains of the FFCT (Overall Departmental Characteristics 35.4 ± 15.4, Departmental Foundation of Feedback 46.1 ± 16.7, and Departmental Emphasis of Feedback 30.3 ± 17.7) had the highest mean in the Market archetype (results/achievement oriented), whereas the final domain (Departmental Definition of Successful Feedback 34.8 ± 22.1) had the highest mean in the Clan archetype (personal growth/team achievement). Item responses in the Clan and Hierarchy archetypes had the strongest convergent and divergent validity, respectively. Item responses in the Adhocracy archetype had the weakest convergent and divergent validity. CONCLUSIONS:Although the sample size was small, this initial study demonstrates that a modified organizational culture assessment tool can feasibly be utilized to identify the primary formative feedback archetype of a cohort of residents. This may have future implications for measuring changes in culture after the implementation of strategic programs to address formative feedback. Future studies should examine the generalizability of the FFCT to other institutions, as well as address the weak validity evidence of the Adhocracy archetype in the FFCT.


Teach Learn Med


Rougas S,Clyne B,Cianciolo AT,Chan TM,Sherbino J,Yarris LM




Has Abstract


2015-01-01 00:00:00












  • An evidence-based domestic violence education program for internal medicine residents.

    abstract:BACKGROUND:Domestic violence (DV) is prevalent but often unrecognized, and it is a challenge to teach. This article presents an evidence-based DV education program for medical residents and incorporates it into a women's medicine curriculum. DESCRIPTION:An initial 3-hr seminar included video and case discussion, liter...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Korenstein D,Thomas DC,Foldes C,Ross J,Halm E,McGinn T

    更新日期:2003-10-01 00:00:00

  • Resident performance on the in-training and board examinations in obstetrics and gynecology: implications for the ACGME Outcome Project.

    abstract:BACKGROUND:The Accreditation Council on Graduate Medical Education (ACGME) Outcomes Project has endorsed the in-training examination (ITE) as an example of a multiple-choice question examination that is a valid measure of a resident's attainment of medical knowledge. An outcome measure for performance on the ITE would ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Withiam-Leitch M,Olawaiye A

    更新日期:2008-04-01 00:00:00

  • Common Ground: Frameworks for Teaching Improvisational Ability in Medical Education.

    abstract::Issue: The practice of medicine is intrinsically unpredictable. Clinicians must respond skillfully to this uncertainty; therefore, medical educators are using improvisational theater training methods to teach improvisational ability in areas such as communication and professionalism. This teaching approach is called "...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Fu B

    更新日期:2019-06-01 00:00:00

  • Griev_Ing: death notification skills and applications for fourth-year medical students.

    abstract:BACKGROUND AND PURPOSE:Our study examined whether GRIEV_ING improved death notification skills of medical students, whether pretesting with simulated survivors primed learners and improved results of the intervention, and whether feedback on the simulated encounter improved student performance. METHODS:GRIEV_ING train...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Hobgood CD,Tamayo-Sarver JH,Hollar DW Jr,Sawning S

    更新日期:2009-07-01 00:00:00

  • Consensus of the Millennium Conference on Teaching High Value Care.

    abstract:ISSUE:Healthcare costs have spiraled out of control, yet students and residents may lack the knowledge and skills to provide high value care, which emphasizes the best possible care while reducing unnecessary costs. EVIDENCE:Mainly national campaigns are aimed at physicians to reconsider their test ordering behaviors,...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Huang GC,Tibbles CD,Newman LR,Schwartzstein RM

    更新日期:2016-01-01 00:00:00

  • Alliance for clinical education perspective paper: recommendations for redesigning the "final year" of medical school.

    abstract:BACKGROUND:Although medical school typically lasts 4 years, little attention has been devoted to the structure of the educational experience that takes place during the final year of medical school. SUMMARY:In this perspectives paper, we outline goals for the 4th year of medical school to facilitate a transition from ...

    journal_title:Teaching and learning in medicine

    pub_type: 社论


    authors: Reddy ST,Chao J,Carter JL,Drucker R,Katz NT,Nesbit R,Roman B,Wallenstein J,Beck GL

    更新日期:2014-01-01 00:00:00

  • Comparison of clinical skills of 3rd-year students who completed structured clinical skills program with 6th-year students who acquired clinical skills in unsystematic way.

    abstract:BACKGROUND:The timing and methods of teaching clinical skills are some of the main concerns of medical education. PURPOSE:The aim of this study was to compare clinical skills of the 3rd and 6th-year students who acquired clinical skills training within different years and methods. METHODS:Randomly chosen students wer...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Güldal D,Ozçakar N,Yeniçeri N,Dontlu C,Ulusel B

    更新日期:2005-01-01 00:00:00

  • Human dimensions in bedside teaching: focus group discussions of teachers and learners.

    abstract:BACKGROUND:Clinical teaching has moved from the bedside to conference rooms; many reasons are described for this shift. Yet, essential clinical skills, professionalism, and humanistic patient interactions are best taught at the bedside. PURPOSE:Clinical teaching has moved from the bedside to conference rooms; many rea...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Ramani S,Orlander JD

    更新日期:2013-01-01 00:00:00

  • Association of Use of the NEJM Knowledge+ Product and Performance on the ABIM IM-MOC Exam.

    abstract::Phenomenon: Internal medicine physicians in the United States must pass the American Board of Internal Medicine Internal Medicine Maintenance of Certification (ABIM IM-MOC) examination as part of their ABIM IM-MOC requirements. Many of these physicians use an examination product to help them prepare, such as e-Learnin...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Healy MG,Gray BM,Duhigg LM,O'Rourke M,Marchwinski L,Elmore M,Phitayakorn R

    更新日期:2020-09-15 00:00:00

  • Learning in a High-Stress Clinical Environment: Stressors Associated with Medical Students' Clerkship Training on Labor and Delivery.

    abstract::Phenomenon: Obstetrics and gynecology can be a high-stress clinical environment for medical students due to the specialty's fast-paced and unpredictable nature. Little is known about learning stressors for medical students on the obstetrics and gynecology clerkship to inform stress-reducing interventions and improve l...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Johnson NR,Pelletier A,Chen X,Manning-Geist BL

    更新日期:2019-08-01 00:00:00

  • A "medical student practice profile" as the foundation for a case-based curriculum revision.

    abstract:BACKGROUND:In 2001, the Keck School of Medicine of the University of Southern California initiated a major curriculum reform with fully integrated teaching of the basic sciences. DESCRIPTION:The new curriculum integrated a set of selected clinical cases called the student practice profile (SPP). The SPP cases were des...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Abbott A,Sullivan M,Nyquist J,Mylona E,Taylor C

    更新日期:2010-10-01 00:00:00

  • Cheating in OSCEs: The Impact of Simulated Security Breaches on OSCE Performance.

    abstract::Construct: Valid score interpretation is important for constructs in performance assessments such as objective structured clinical examinations (OSCEs). An OSCE is a type of performance assessment in which a series of standardized patients interact with the student or candidate who is scored by either the standardized...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章,随机对照试验


    authors: Gotzmann A,De Champlain A,Homayra F,Fotheringham A,de Vries I,Forgie M,Pugh D

    更新日期:2017-01-01 00:00:00

  • Curricular reform of the 4th year of medical school: the colleges model.

    abstract:BACKGROUND:The practice of medicine has changed dramatically over the last 3 decades. Medical education has struggled to keep up with these changes, with only limited success. The 4th year of medical school offers a tremendous opportunity for curricular innovation, but little change has occurred in the past 30 years. ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Slavin SJ,Wilkes MS,Usatine RP,Hoffman JR

    更新日期:2003-07-01 00:00:00

  • Benefits of testable concept maps for learning about pathogenesis of disease.

    abstract:BACKGROUND:Concept maps can assist learning by integrating new information with existing cognitive structure to facilitate meaningful understanding. The benefits of testable concept maps to illustrate cause-and-effect sequences in the pathogenesis of disease have not yet been determined. PURPOSE:A controlled trial was...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Kumar S,Dee F,Kumar R,Velan G

    更新日期:2011-04-01 00:00:00

  • Validity Evidence for a Residency Admissions Standardized Assessment Letter for Pediatrics.

    abstract::Construct: This study aims to provide validity evidence for the standardized Assessment Letter for Pediatrics as a measure of competencies expected of a 1st-year pediatrics resident as part of a pediatric residency admissions process. BACKGROUND:The Narrative Letter of Recommendation is a frequently used tool in the ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Bajwa NM,Yudkowsky R,Belli D,Vu NV,Park YS

    更新日期:2018-04-01 00:00:00

  • Team-based learning from theory to practice: faculty reactions to the innovation.

    abstract:BACKGROUND:Limited studies have examined the factors associated with the implementation of team-based learning (TBL). PURPOSE:The purpose of this study was to identify faculty reactions (successes and challenges) associated with the implementation of a modified TBL in undergraduate anatomy teaching. METHOD:To obtain ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Sutherland S,Bahramifarid N,Jalali A

    更新日期:2013-01-01 00:00:00

  • Attitudes of affiliate faculty members toward medical student summative evaluation for clinical clerkships: a qualitative analysis.

    abstract:BACKGROUND:Summative evaluation of medical students is a critical component of the educational process. Despite extensive literature on evaluation, few studies have centered on affiliate faculty members' attitudes toward summative evaluation of students, though it has been suggested that these attitudes influence their...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Wang KE,Fitzpatrick C,George D,Lane L

    更新日期:2012-01-01 00:00:00

  • A new oral health elective for medical students at the University of Washington.

    abstract:BACKGROUND:Oral health is an important but inadequately addressed area in medical school curricula. Primary care practitioners are in an ideal position to help prevent oral disease but lack the knowledge to do so. PURPOSES:We developed an oral health elective that targeted 1st- and 2nd-year medical students as part of...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Mouradian WE,Reeves A,Kim S,Lewis C,Keerbs A,Slayton RL,Gupta D,Oskouian R,Schaad D,Kalet T,Marshall SG

    更新日期:2006-10-01 00:00:00

  • Collaborative Reflection Under the Microscope: Using Conversation Analysis to Study the Transition From Case Presentation to Discussion in GP Residents' Experience Sharing Sessions.

    abstract:UNLABELLED:PHENOMENON: In higher education, reflection sessions are often used when participants learn in the workplace. In the Netherlands, all General Practitioner training programs include regular meetings called Exchange of Experiences, in which General Practitioner trainees are expected to learn collaboratively fr...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Veen M,de la Croix A

    更新日期:2016-01-01 00:00:00

  • Student Perceptions of M.D.-Ph.D. Programs: A Qualitative Identification of Barriers Facing Prospective M.D.-Ph.D. Applicants.

    abstract::Phenomenon: Despite a high degree of interest in research among matriculating M.D. students, very few apply to combined M.D.-Ph.D. training programs. Even fewer of those applicants are female, leading to a gender disparity among M.D.-Ph.D. trainees. We used a qualitative approach to understand why students choose not ...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Kersbergen CJ,Bowen CJ,Dykema AG,Koretzky MO,Tang O,Beach MC

    更新日期:2020-01-01 00:00:00

  • A review of U.S. Medical schools' promotion standards for educational excellence.

    abstract::Phenomenon: Given the growing number of medical science educators, an examination of institutions' promotion criteria related to educational excellence and scholarship is timely. This study investigates the extent to which medical schools' promotion criteria align with published standards for documenting and evaluatin...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Hoffman LA,Lufler RS,Brown KM,DeVeau K,DeVaul N,Fatica LM,Mussell J,Byram JN,Dunham SM,Wilson AB

    更新日期:2020-04-01 00:00:00

  • Teaching medical students ultrasound to measure liver size: comparison with experienced clinicians using physical examination alone.

    abstract:BACKGROUND:Ultrasound is increasingly recognized as a valuable addition to medical school curriculum. PURPOSE:In this study, we tested the ability of rising second year students to learn and conduct an ultrasound examination of vertical liver span at the point of care. METHODS:Six patients from a GI clinic volunteere...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章,随机对照试验


    authors: Mouratev G,Howe D,Hoppmann R,Poston MB,Reid R,Varnadoe J,Smith S,McCallum B,Rao V,DeMarco P

    更新日期:2013-01-01 00:00:00

  • A comprehensive medical student career development program improves medical student satisfaction with career planning.

    abstract::In 1999, the University of Michigan Medical School (UMMS) initiated a new career development program (CDP). The CDP incorporates the 4-phase career development model described by the Association of American Medical Colleges (AAMC) Careers in Medicine (CiM). The CDP offers self-assessment exercises with guidance from t...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Zink BJ,Hammoud MM,Middleton E,Moroney D,Schigelone A

    更新日期:2007-01-01 00:00:00

  • Teaching geriatric assessment in home visits: the family physician/geriatrician attachment.

    abstract:BACKGROUND:Geriatric clinical clerkships in Israel teach mostly about the hospitalized elder patient with almost no ambulatory experience. Meanwhile, primary care physicians provide most of the health care to the elderly in the community. This article describes an innovation in the curriculum of the 5th-year family med...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Tandeter H,Peleg R,Menahem S,Biderman A,Fried VA

    更新日期:2003-04-01 00:00:00

  • Seeing Is Believing: Evaluating a Point-of-Care Ultrasound Curriculum for 1st-Year Medical Students.

    abstract:PROBLEM:Point-of-care ultrasound has been a novel addition to undergraduate medical education at a few medical schools. The impact is not fully understood, and few rigorous assessments of educational outcomes exist. This study assessed the impact of a point-of-care ultrasound curriculum on image acquisition, interpreta...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Nelson BP,Hojsak J,Dei Rossi E,Karani R,Narula J

    更新日期:2017-01-01 00:00:00

  • Proceedings of the 2008 Annual Predoctoral Education Conference of the Society of Teachers of Family Medicine (STFM).

    abstract::The Society of Teachers of Family Medicine (STFM; is a community of professionals devoted to teaching family medicine through undergraduate, graduate, and continuing medical education. This multidisciplinary group of physicians, educators, behavioral scientists, and researchers works to further ST...

    journal_title:Teaching and learning in medicine



    authors: Little DN,Hatch RL

    更新日期:2008-10-01 00:00:00

  • A review of primary care training programs in correctional health for physicians.

    abstract:BACKGROUND:Despite the health consequences of incarceration, there are no national standards for teaching physicians about correctional health. The purpose of this study was to survey existing correctional health training programs for primary care physicians. SUMMARY:Programs were identified through literature and Web...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Min I,Schonberg D,Anderson M

    更新日期:2012-01-01 00:00:00

  • Relationship between dean's letter rankings and later evaluations by residency program directors.

    abstract:BACKGROUND:It is not known how well dean's letter rankings predict later performance in residency. PURPOSE:To assess the accuracy of dean's letter rankings to predict clinical performance in internship. METHOD:Participants were medical students who graduated from the University of Rochester School of Medicine and Den...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Lurie SJ,Lambert DR,Grady-Weliky TA

    更新日期:2007-07-01 00:00:00

  • Standard Setting Methods for Pass/Fail Decisions on High-Stakes Objective Structured Clinical Examinations: A Validity Study.

    abstract:UNLABELLED:CONSTRUCT: Authentic standard setting methods will demonstrate high convergent validity evidence of their outcomes, that is, cutoff scores and pass/fail decisions, with most other methods when compared with each other. BACKGROUND:The objective structured clinical examination (OSCE) was established for valid...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Yousuf N,Violato C,Zuberi RW

    更新日期:2015-01-01 00:00:00

  • The Feasibility of Longitudinal Patient Contacts in a Large Medical School.

    abstract:PROBLEM:Longitudinal patient contacts are being implemented worldwide as a way to enhance a patient-centered orientation among medical students. In large medical schools, longitudinal integrated clerkships may not be feasible, so other ways must be sought to expose students to prolonged contact with patients. INTERVEN...

    journal_title:Teaching and learning in medicine

    pub_type: 杂志文章


    authors: Mol SS,Chen HC,Steerneman AH,de Groot E,Zwart DL

    更新日期:2019-04-01 00:00:00