Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope.

Abstract:

CONTEXT:Programme evaluation is perpetually mandated in health professions education. Correspondingly, there has been an expansion in prescriptive methodological guides about 'how' to engage in various best practices in evaluation. However, what has gained less attention is an examination of the 'value' that different stakeholders seek to gain from programme evaluation. Evaluation utilisation theory and research can help us understand the diversity in both the driving forces for and the impact of programme evaluation. Awareness of the heterogeneity of evaluation utilisation priorities has implications for evaluation practices, including both methodological choices and understanding of the impact of programme evaluation in our field. METHODS:In this article, I expound on the concept of evaluation utilisation by drawing on evaluation theory and research. 'Evaluation utilisation' refers to the application of programme evaluation processes and findings to influence thinking and action. Herein, different forms of 'evaluation utilisation' (including instrumental, conceptual, process and persuasive (symbolic and legitimative) utilisation) are discussed, as well as the related concept of 'evaluation misuse'. Furthermore, how the prioritisation of different forms of 'evaluation utilisation' can influence the scope and impact of evaluation scholarship are also discussed. CONCLUSIONS:Programme evaluation is a form of inquiry that requires more than the exercise of robust methodological techniques. Rather, it necessitates attention to the, sometimes divergent, priorities of different stakeholder groups. Although there is scant research on evaluation practices in health professions education, evaluation utilisation theory can inform critical examination of evaluation practices and impact in our field. Critically, understanding this body of work can help inform those engaged in evaluation about what they are (or should be) prioritising when they conduct programme evaluation, and better align evaluation methodologies with their scholarly, curricular and administrative intentions. Implications for future research and high-quality, transparent evaluation scholarship are presented.

journal_name

Med Educ

journal_title

Medical education

authors

Onyura B

doi

10.1111/medu.14281

subject

Has Abstract

pub_date

2020-12-01 00:00:00

pages

1100-1108

issue

12

eissn

0308-0110

issn

1365-2923

journal_volume

54

pub_type

杂志文章,评审
  • Pre-registration house officers' views on studying under a reformed medical curriculum in the UK.

    abstract::In 1996 the University of Liverpool introduced a new curriculum based on the recommendations published in Tomorrow's Doctors. This work examines how graduates of that course view their undergraduate curriculum and whether they consider it prepared them well for the pre-registration year. Five focus groups were arrange...

    journal_title:Medical education

    pub_type: 杂志文章,多中心研究

    doi:10.1111/j.1365-2929.2006.02545.x

    authors: Watmough S,Garden A,Taylor D

    更新日期:2006-09-01 00:00:00

  • Speaking up, support, control and work engagement of medical residents. A structural equation modelling analysis.

    abstract:OBJECTIVES:Medical residents can play key roles in improving health care quality by speaking up and giving suggestions for improvements. However, previous research on speaking up by medical residents has shown that speaking up is difficult for residents. This study explored: (i) whether two main aspects of medical resi...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.13951

    authors: Voogt JJ,Taris TW,van Rensen ELJ,Schneider MME,Noordegraaf M,van der Schaaf MF

    更新日期:2019-11-01 00:00:00

  • Application of continuous quality improvement to medical education.

    abstract:CONTEXT:The explicit, intentional and systematic application of continuous quality improvement (QI) in medical education practice and research can improve medical education and help it achieve its goals. Quality improvement and medical education share a foundation centred on learning-experiencing, reflecting, thinking ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.14351

    authors: Wong BM,Headrick LA

    更新日期:2021-01-01 00:00:00

  • Educational intervention in pharmacy students' attitudes to HIV/AIDS and drug misuse.

    abstract::By providing injecting equipment to drug misusers, community pharmacists in the UK may become involved in preventing the spread of HIV via the intravenous route. Over 60% of pharmacy graduates from the School of Pharmacy enter community pharmacy and, as part of their undergraduate course, attend a series of lectures a...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1994.tb02725.x

    authors: Sheridan J,Bates IP,Webb DG,Barber ND

    更新日期:1994-11-01 00:00:00

  • The medical school as a social organization: the sources of resistance to change.

    abstract::Medical schools vary by nation and by culture but, for students, the experience appears to be very similar. Also, despite a half-century of radical changes in medical practice, education as a process of socialization for the profession is relatively unchanged. At the same time, medical educators have frequently instit...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1989.tb01538.x

    authors: Bloom SW

    更新日期:1989-05-01 00:00:00

  • Justice and care: decision making by medical school student promotions committees.

    abstract:CONTEXT:The function of medical school entities that determine student advancement or dismissal has gone largely unexplored. The decision making of 'academic progress' or student promotions committees is examined using a theoretical framework contrasting ethics of justice and care, with roots in the moral development w...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.13280

    authors: Green EP,Gruppuso PA

    更新日期:2017-06-01 00:00:00

  • Short-listing of applicants from UCCA forms: the structure of pre-selection judgements.

    abstract::Applicants for admission to St Mary's Hospital Medical School in 1986 were short-listed for interview by one of four assessors, who each made their assessments on a nine-item pro forma. One short-lister had also been studied in detail during 1981. Short-listers used the full range of possible judgements, in approximat...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1989.tb00877.x

    authors: McManus IC,Maitlis SL,Richards P

    更新日期:1989-03-01 00:00:00

  • Comparison of medical and non-medical student attitudes to social issues in medicine.

    abstract::A 63-item questionnaire on attitudes to psychosocial issues in medicine was administered to final-year students in the Faculties of Medicine (n = 104), Pharmacy (n = 57) and Arts and Social Sciences (n = 75) of the University of Benin. The responses given were analysed by the summation and Cornell scalogram techniques...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1995.tb02848.x

    authors: Nnodim JO,Osuji CU

    更新日期:1995-07-01 00:00:00

  • The impact of three years' ethics teaching, in an integrated medical curriculum, on students' proposed behaviour on meeting ethical dilemmas.

    abstract:OBJECTIVE:To evaluate the impact of 3 years' ethics teaching (30 hours mainly small group teaching in year 1, 14 hours mainly lecture-type teaching in years 2 and 3) on students' proposed behaviour on encountering ethical dilemmas. DESIGN:Cohort design. SETTING:University of Glasgow Medical School. SUBJECTS:A cohort...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2002.01176.x

    authors: Goldie J,Schwartz L,McConnachie A,Morrison J

    更新日期:2002-05-01 00:00:00

  • Observational analysis of near-peer and faculty tutoring in problem-based learning groups.

    abstract:CONTEXT:Near-peer and faculty staff tutors may facilitate problem-based learning (PBL) through different means. Near-peer tutors are thought to compensate for their lack of subject matter expertise with greater adeptness at group facilitation and a better understanding of their learners. However, theoretical explanatio...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.12969

    authors: Cianciolo AT,Kidd B,Murray S

    更新日期:2016-07-01 00:00:00

  • Medical students' attitudes towards and perception of the basic sciences: a comparison between students in the old and the new curriculum at the University Medical Center Utrecht, The Netherlands.

    abstract:OBJECTIVES:The attitudes towards the basic sciences of medical students enrolled in either of 2 different curricula at the University of Utrecht Medical School in The Netherlands were investigated. The purpose of this study was threefold: first, to compare students (beginning clerks) in a conventional and an innovative...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2002.01371.x

    authors: Custers EJ,Cate OT

    更新日期:2002-12-01 00:00:00

  • Surgical e-learning: validation of multimedia web-based lectures.

    abstract:BACKGROUND:Distance learning has been advocated increasingly as a modern efficient method of teaching surgery. Efficiency of knowledge transfer and validity of web-based courses have not been subjected to rigorous study to date. METHODS:An entirely web-based surgical 5-week lecture course was designed. Fifty per cent ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2929.2006.02669.x

    authors: Ridgway PF,Sheikh A,Sweeney KJ,Evoy D,McDermott E,Felle P,Hill AD,O'Higgins NJ

    更新日期:2007-02-01 00:00:00

  • Current efforts in medical education to incorporate national health priorities.

    abstract::As a reflection on the Edinburgh Declaration, this conceptual synthesis presents six important challenges in relation to the role of medical education in meeting current national health priorities. CONTEXT:This paper presents a conceptual synthesis of current efforts in medical education to incorporate national healt...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.13395

    authors: Nair M,Fellmeth G

    更新日期:2018-01-01 00:00:00

  • Problems of comprehension of medical education vocabulary.

    abstract::Medical teachers' understanding of the vocabulary of medical education has been studied by asking a group of medical teachers whether they understood twenty-two commonly used and important medical educational terms, and whether they could distinguish between the terms. All the words presented some problem of comprehen...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1981.tb02640.x

    authors: Barker VF,Harden RM

    更新日期:1981-07-01 00:00:00

  • CeMENT: evaluation of a regional development programme integrating hospital and general practice clinical teaching for medical undergraduates. The Community-Based Medical Education in North Thames.

    abstract:OBJECTIVES:To determine the feasibility and effectiveness of shared hospital and general practice clinical teaching for medical undergraduates. DESIGN:A multifaceted approach employing quantitative and qualitative techniques. SETTING:All medical schools in North Thames Region. SUBJECTS:Students, GP tutors and hospit...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2001.00763.x

    authors: Wallace P,Berlin A,Murray E,Southgate L

    更新日期:2001-02-01 00:00:00

  • A solution to the cueing effects of multiple choice questions: the Un-Q format.

    abstract::Although the cueing effects inherent in conventional multiple choice questions (MCQs) present serious limitations, this format continues to dominate testing programmes. The present study was undertaken to estimate the effects of cueing when MCQs are used to test medical students, and to evaluate the reliability, valid...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1993.tb00284.x

    authors: Veloski JJ,Rabinowitz HK,Robeson MR

    更新日期:1993-07-01 00:00:00

  • Chaos, complexity and complicatedness: lessons from rocket science.

    abstract:CONTEXT:Recently several authors have drawn parallels between educational research and some theories of natural science, in particular complexity theory and chaos theory. The central claim is that both the natural science theories are useful metaphors for education research in that they deal with phenomena that involve...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2011.03945.x

    authors: Norman G

    更新日期:2011-06-01 00:00:00

  • Medical specialization in Europe: the way forward. Promotion of educational quality.

    abstract::The medical specialties are being intensively reconsidered in the countries of Europe. The important need is for promoting improvement of educational training programmes. Evaluation and improvement of specialist training programmes is the priority and not the setting up of qualifying examinations. The first necessity ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1993.tb00274.x

    authors: Karle H,Nystrup J,Walton H

    更新日期:1993-07-01 00:00:00

  • Moral reasoning as a criterion for admission to medical school.

    abstract::To determine whether admission interviews could differentiate applicants on their personal qualities (such as integrity, empathy and commitment) 456 applicants from two medical schools were tested on the Defining Issues Test (DIT), which measures the amount of principled or post-conventional moral reasoning. No differ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1984.tb01297.x

    authors: Benor DE,Notzer N,Sheehan TJ,Norman GR

    更新日期:1984-11-01 00:00:00

  • Ethnography in health professions education: Slowing down and thinking deeply.

    abstract:CONTEXT:Ethnography has been gaining appreciation in the field of health professions education (HPE) research, yet it remains misunderstood. Our article contributes to this growing literature by describing some of the key tensions with which both aspiring and seasoned ethnographers should productively struggle. METHOD...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.14033

    authors: Bressers G,Brydges M,Paradis E

    更新日期:2020-03-01 00:00:00

  • Introducing expert systems to medical students using ESTA, expert system teaching aid.

    abstract::Medical students are allocated little curriculum time for exposure to expert systems. ESTA, a computer model of an expert system, was developed to make best use of this time. The nature of the students' interaction with ESTA is described, and their reactions to the expert system concepts and the place of expert system...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1988.tb00418.x

    authors: McWilliams AT

    更新日期:1988-03-01 00:00:00

  • An exploration of the attitudes, personality and performance of dental students.

    abstract::This study investigates (1) the behavioural patterns, performance and attitudes to dentistry and (2) the personality traits, in male and female dental students (n = 219) studying at Cardiff Dental School. The first part deals with the results of a questionnaire investigating aspects of the students' training course, i...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1985.tb01313.x

    authors: Reeve PE,Watson CJ

    更新日期:1985-05-01 00:00:00

  • In search of educational efficiency: 30 years of Medical Education's top-cited articles.

    abstract:CONTEXT:Academic journals represent shared spaces wherein the significance of thematic areas, methodologies and paradigms are debated and shaped through collective engagement. By studying journals in their historical and cultural contexts, the academic community can gain insight into the ways in which authors and audie...

    journal_title:Medical education

    pub_type: 历史文章,杂志文章,评审

    doi:10.1111/medu.13349

    authors: Rangel JC,Cartmill C,Martimianakis MA,Kuper A,Whitehead CR

    更新日期:2017-09-01 00:00:00

  • Self-explanation in learning clinical reasoning: the added value of examples and prompts.

    abstract:CONTEXT:Recent studies suggest that self-explanation (SE) while diagnosing cases fosters the development of clinical reasoning in medical students; however, the conditions that optimise the impact of SE remain unknown. The example-based learning framework justifies an exploration of students' use of their own SEs combi...

    journal_title:Medical education

    pub_type: 杂志文章,随机对照试验

    doi:10.1111/medu.12623

    authors: Chamberland M,Mamede S,St-Onge C,Setrakian J,Bergeron L,Schmidt H

    更新日期:2015-02-01 00:00:00

  • Evaluating family counselling skills training for family practice.

    abstract::Educational programmes for family practice should develop family counselling skills of students to moderate levels of competence. Few specific training programmes are part of the regular curriculum and of these few are evaluated. Twenty-three residents enrolled in a 2-year family practice programme in Toronto, Canada ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1990.tb02656.x

    authors: Rudner HL,Bestvater D,Bader E

    更新日期:1990-09-01 00:00:00

  • Non-analytical models of clinical reasoning: the role of experience.

    abstract:OBJECTIVE:This paper aims to summarise the evidence supporting the role of experience-based, non-analytic reasoning (NAR) or pattern recognition as a central feature of expert medical diagnosis. METHODS:The authors examine a series of studies, primarily from their own research programme at McMaster University, that de...

    journal_title:Medical education

    pub_type: 杂志文章,评审

    doi:10.1111/j.1365-2923.2007.02914.x

    authors: Norman G,Young M,Brooks L

    更新日期:2007-12-01 00:00:00

  • Supporting the problem-based learning process in the clinical years: evaluation of an online Clinical Reasoning Guide.

    abstract:PURPOSE:Implementing problem-based learning (PBL) in the clinical years of a medical degree presents particular challenges. This study investigated the effectiveness of using an online Clinical Reasoning Guide to assist integration of PBL in the clinical setting and promote further development of students' clinical rea...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2929.2004.01839.x

    authors: Ryan G,Dolling T,Barnet S

    更新日期:2004-06-01 00:00:00

  • Length of in-patient stay in teaching hospitals in Japan.

    abstract::The average length of in-patient stay (LOS) in Japan is longer than that of the other Organization for Economic Cooperation and Development (OECD) countries. The tendency towards long LOS is also apparent in teaching hospitals in Japan. This paper examines factors responsible for the long LOS in teaching hospitals, fo...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1993.tb00269.x

    authors: Yamaoka K,Kobayashi Y,Yano E

    更新日期:1993-05-01 00:00:00

  • Training clinical teachers in psychiatry to teach interviewing skills to medical students.

    abstract::To study the feasibility of training all clinical teachers in psychiatry to teach interviewing skills to medical students, 24 (unselected clinicians were assigned to one of four different training methods. They received either experiential or didactic instruction, and their initial teaching sessions were either superv...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1986.tb01062.x

    authors: Naji SA,Maguire GP,Fairbairn SA,Goldberg DP,Faragher EB

    更新日期:1986-03-01 00:00:00

  • Analysis of verbal interactions in tutorial groups: a process study.

    abstract:INTRODUCTION:Collaborative learning, including problem-based learning (PBL), is a powerful learning method. Group interaction plays a crucial role in stimulating student learning. However, few studies on learning processes in medical education have examined group interactions. Most studies on collaboration within PBL u...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2929.2005.02368.x

    authors: Visschers-Pleijers AJ,Dolmans DH,de Leng BA,Wolfhagen IH,van der Vleuten CP

    更新日期:2006-02-01 00:00:00