Script concordance testing: a review of published validity evidence.

Abstract:

CONTEXT:Script concordance test (SCT) scores are intended to reflect respondents' competence in interpreting clinical data under conditions of uncertainty. The validity of inferences based on SCT scores has not been rigorously established. OBJECTIVES:This study was conducted in order to develop a structured validity argument for the interpretation of test scores derived through use of the script concordance method. METHODS:We searched the PubMed, EMBASE and PsycINFO databases for articles pertaining to script concordance testing. We then reviewed these articles to evaluate the construct validity of the script concordance method, following an established approach for analysing validity data from five categories: content; response process; internal structure; relations to other variables, and consequences. RESULTS:Content evidence derives from clear guidelines for the creation of authentic, ill-defined scenarios. High internal consistency reliability supports the internal structure of SCT scores. As might be expected, SCT scores correlate poorly with assessments of pure factual knowledge, in which correlations for more advanced learners are lower. The validity of SCT scores is weakly supported by evidence pertaining to examinee response processes and educational consequences. CONCLUSIONS:Published research generally supports the use of SCT to assess the interpretation of clinical data under conditions of uncertainty, although specifics of the validity argument vary and require verification in different contexts and for particular SCTs. Our review identifies potential areas of further validity inquiry in all five categories of evidence. In particular, future SCT research might explore the impact of the script concordance method on teaching and learning, and examine how SCTs integrate with other assessment methods within comprehensive assessment programmes.

journal_name

Med Educ

journal_title

Medical education

authors

Lubarsky S,Charlin B,Cook DA,Chalk C,van der Vleuten CP

doi

10.1111/j.1365-2923.2010.03863.x

subject

Has Abstract

pub_date

2011-04-01 00:00:00

pages

329-38

issue

4

eissn

0308-0110

issn

1365-2923

journal_volume

45

pub_type

杂志文章,评审
  • Evaluation of an instruction programme on diabetes diet by means of a teaching machine.

    abstract::In 119 patients with diabetes, student nurses, social workers, dietitians and medical students a pretest was carried out with multiple-choice questions on the subject of diabetes diet. The group was then exposed to programmed diet instruction with a teaching machine. There was a significant learning effect as measured...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1976.tb00482.x

    authors: Teuscher A,Heidecker B

    更新日期:1976-11-01 00:00:00

  • Realist methods in medical education research: what are they and what can they contribute?

    abstract:CONTEXT:Education is a complex intervention which produces different outcomes in different circumstances. Education researchers have long recognised the need to supplement experimental studies of efficacy with a broader range of study designs that will help to unpack the 'how' and 'why' questions and illuminate the man...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2011.04045.x

    authors: Wong G,Greenhalgh T,Westhorp G,Pawson R

    更新日期:2012-01-01 00:00:00

  • Medical education accreditation in Mexico and the Philippines: impact on student outcomes.

    abstract:CONTEXT:Accreditation of medical education programmes is becoming increasingly prevalent worldwide, but beyond the face validity of these quality assurance methods, data linking accreditation to improved student outcomes are limited. Mexico and the Philippines both have voluntary systems of medical education accreditat...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2011.04212.x

    authors: van Zanten M,McKinley D,Durante Montiel I,Pijano CV

    更新日期:2012-06-01 00:00:00

  • Teaching communication skills to pre-clinical medical students: a general practice based approach.

    abstract::This paper describes a method of teaching communication skills to pre-clinical medical students in the setting of general practice. By focusing on the nature of the patient's problem this teaching tries to place interviewing and diagnostic procedures in their proper context in the doctor-patient relationship. ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1979.tb00926.x

    authors: Armstrong D,Hicks BH,Higgins PM,Weinman JA

    更新日期:1979-03-01 00:00:00

  • Smoking history-taking skills: a simple guide to teach medical students.

    abstract::Many doctors rate themselves as ineffective smoking cessation counsellors. It is logical to initiate training efforts with medical students. We incorporated smoking history-taking in the physical diagnosis course at the University of North Carolina at Chapel Hill using a simple method to teach smoking history-taking s...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1996.tb00830.x

    authors: Boehlecke B,Sperber AD,Kowlowitz V,Becker M,Contreras A,McGaghie WC

    更新日期:1996-07-01 00:00:00

  • A rating scale for tutor evaluation in a problem-based curriculum: validity and reliability.

    abstract::An instrument has been developed to assess tutor performance in problem-based tutorial groups. This tutor evaluation questionnaire consists of 13 statements reflecting the tutor's behaviour. The statements are based on a description of the tasks set for the tutor. This study reports results on the validity and reliabi...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1994.tb02735.x

    authors: Dolmans DH,Wolfhagen IH,Schmidt HG,van der Vleuten CP

    更新日期:1994-11-01 00:00:00

  • Reliability of the clinical teaching effectiveness instrument.

    abstract:INTRODUCTION:The Clinical Teaching Effectiveness Instrument (CTEI) was developed to evaluate the quality of the clinical teaching of educators. Its authors reported evidence supporting content and criterion validity and found favourable reliability findings. We tested the validity and reliability of this instrument in ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2929.2005.02245.x

    authors: van der Hem-Stokroos HH,van der Vleuten CP,Daelmans HE,Haarman HJ,Scherpbier AJ

    更新日期:2005-09-01 00:00:00

  • An exploration of the attitudes, personality and performance of dental students.

    abstract::This study investigates (1) the behavioural patterns, performance and attitudes to dentistry and (2) the personality traits, in male and female dental students (n = 219) studying at Cardiff Dental School. The first part deals with the results of a questionnaire investigating aspects of the students' training course, i...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1985.tb01313.x

    authors: Reeve PE,Watson CJ

    更新日期:1985-05-01 00:00:00

  • Sexual harassment during medical training: the perceptions of medical students at a university medical school in Australia.

    abstract:CONTEXT:A survey of medical students' experiences of sexual harassment during medical training. OBJECTIVE:To assess the prevalence and nature of workplace sexual harassment as perceived by undergraduate medical students in order to address their learning needs concerning setting and maintaining sexual boundaries. DES...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2000.00684.x

    authors: White GE

    更新日期:2000-12-01 00:00:00

  • Undergraduate education in the cost-effective use of laboratory and radiological investigations in clinical decision-making.

    abstract::This paper describes a strategy for helping students to learn about laboratory and radiological investigations in diagnosis and management, and to make decisions about the use of investigations, taking into account such features as safety, patient comfort and cost. Learning takes place through the use of a programmed ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1981.tb02310.x

    authors: Clarke RM

    更新日期:1981-01-01 00:00:00

  • Computer analysis of the student log diary: an aid in the teaching of general practice and family medicine.

    abstract::At the University of Queensland all medical students in their fifth year rotate through a 2-week attachment to a general practitioner tutor as part of the course in community practice. As a means of learning from this experience each student records information in a log diary on seventy-five consecutive patients seen,...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1984.tb00976.x

    authors: Munro JG

    更新日期:1984-03-01 00:00:00

  • Teaching and learning in the clinical setting: a qualitative study of the perceptions of students and teachers.

    abstract:OBJECTIVE:To describe the perceptions of medical students and clinical teachers of teaching and learning in the clinical setting. DESIGN:Qualitative study of focus groups with undergraduate medical students and semistructured interviews with hospital consultant clinical teachers. SETTING:The School of Medicine, Unive...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2003.01675.x

    authors: Stark P

    更新日期:2003-11-01 00:00:00

  • The dual nature of medical enculturation in postgraduate medical training and practice.

    abstract:OBJECTIVES:Enculturation is a normal and continuing part of human development. This study examined how medical graduates perceive the process of enculturation after graduation. METHODS:We conducted a qualitative study of the values of medical graduates associated with Sydney Medical School to identify processes that c...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2012.04301.x

    authors: Gordon J,Markham P,Lipworth W,Kerridge I,Little M

    更新日期:2012-09-01 00:00:00

  • Special provisions for women doctors to train and practise in medicine after graduation: a report of a survey.

    abstract::The findings of a survey of 3262 women doctors registered between 1945 and 1974 are reported. The results are based on 2433 returned questionnaires (75% response). The survey confirmed that the special forms of assistance provided to enable women doctors to continue training and to work after graduation are inadequate...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1979.tb01514.x

    authors: Beaumont B

    更新日期:1979-07-01 00:00:00

  • Prior academic background and student performance in assessment in a graduate entry programme.

    abstract:OBJECTIVES:This study aims to identify whether non-science graduates perform as well as science graduates in Basic and Clinical Sciences (B & CS) assessments during Years 1-3 of a four-year graduate-entry programme at the University of Sydney (the 'USydMP'). METHODS:Students were grouped into five categories: Health P...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2929.2004.02043.x

    authors: Craig PL,Gordon JJ,Clark RM,Langendyk V

    更新日期:2004-11-01 00:00:00

  • The mixed impact of medical school on medical students' implicit and explicit weight bias.

    abstract:CONTEXT:Health care trainees demonstrate implicit (automatic, unconscious) and explicit (conscious) bias against people from stigmatised and marginalised social groups, which can negatively influence communication and decision making. Medical schools are well positioned to intervene and reduce bias in new physicians. ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.12770

    authors: Phelan SM,Puhl RM,Burke SE,Hardeman R,Dovidio JF,Nelson DB,Przedworski J,Burgess DJ,Perry S,Yeazel MW,van Ryn M

    更新日期:2015-10-01 00:00:00

  • CeMENT: evaluation of a regional development programme integrating hospital and general practice clinical teaching for medical undergraduates. The Community-Based Medical Education in North Thames.

    abstract:OBJECTIVES:To determine the feasibility and effectiveness of shared hospital and general practice clinical teaching for medical undergraduates. DESIGN:A multifaceted approach employing quantitative and qualitative techniques. SETTING:All medical schools in North Thames Region. SUBJECTS:Students, GP tutors and hospit...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2001.00763.x

    authors: Wallace P,Berlin A,Murray E,Southgate L

    更新日期:2001-02-01 00:00:00

  • Resuscitating paediatric induction: an action research approach.

    abstract:OBJECTIVES:To establish teaching objectives, methods and assessment for a paediatric induction programme. DESIGN:Action research over a 2-year period. PARTICIPANTS:88 doctors from three hospitals. MAIN OUTCOME MEASURES:These were end-of-course evaluations, using Likert scales, and free comments; independent evaluati...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2001.00982.x

    authors: Melville C,Wall D,Samuels M

    更新日期:2001-08-01 00:00:00

  • Predicting medical students' intentions to take up rural practice after graduation.

    abstract:OBJECTIVES:Using a novel longitudinal tracking project, this study develops and evaluates the performance of a predictive model and index of rural medical practice intention based on the characteristics of incoming medical students. METHODS:Medical school entry survey data were obtained from the Medical Schools Outcom...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2009.03506.x

    authors: Jones M,Humphreys J,Prideaux D

    更新日期:2009-10-01 00:00:00

  • Do patients have an obligation to participate in student teaching?

    abstract:CONTEXT:Patients may participate in teaching in many ways, in different settings and with different degrees of expert supervision. The majority of patients are generally very willing to participate in teaching. At times, however, patients may decline to see students because they are too sick, wish to maintain their pri...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2007.02948.x

    authors: Lowe M,Kerridge I,McPhee J,Hart C

    更新日期:2008-03-01 00:00:00

  • Scripts and clinical reasoning.

    abstract:CONTEXT:Each clinical encounter represents an amazing series of psychological events: perceiving the features of the situation; quickly accessing relevant hypotheses; checking for signs and symptoms that confirm or rule out competing hypotheses, and using related knowledge to guide appropriate investigations and treatm...

    journal_title:Medical education

    pub_type: 杂志文章,评审

    doi:10.1111/j.1365-2923.2007.02924.x

    authors: Charlin B,Boshuizen HP,Custers EJ,Feltovich PJ

    更新日期:2007-12-01 00:00:00

  • Students conducting consultations in general practice and the acceptability to patients.

    abstract:OBJECTIVES:The General Medical Council has recommended that medical students should gain more experience in general practice. The study set out to determine patients' reactions to consultations conducted by a medical student alone prior to seeing their GP. DESIGN:A random sample of patients attending general practice ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.1999.00410.x

    authors: Bentham J,Burke J,Clark J,Svoboda C,Vallance G,Yeow M

    更新日期:1999-09-01 00:00:00

  • Postgraduate training and career choices: an analysis of the National Physicians Survey in Japan.

    abstract:OBJECTIVES:This study analyses and discusses recent changes in young Japanese doctors' career paths, in terms of their distribution in different types of facilities and specialties, following changes to the postgraduate clinical training system in 2004. METHODS:Data from the National Survey of Physicians, Dentists and...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2009.03582.x

    authors: Koike S,Ide H,Yasunaga H,Kodama T,Matsumoto S,Imamura T

    更新日期:2010-03-01 00:00:00

  • Test-enhanced learning and its effect on comprehension and diagnostic accuracy.

    abstract:CONTEXT:In health professions education, tests have traditionally been used to assess the skills and knowledge of learners. More recently, research in psychology and education has shown that tests can also be used to enhance student memory; a phenomenon called the 'testing effect'. Much of the research in this domain h...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/medu.12302

    authors: Baghdady M,Carnahan H,Lam EW,Woods NN

    更新日期:2014-02-01 00:00:00

  • Medical students' health-related quality of life: roles of social and behavioural factors.

    abstract:OBJECTIVES:The long and demanding course of medical education may be accompanied by distress with serious consequences. This study was conducted to assess the multidimensional health-related quality of life (HR-QoL) in medical students in different stages of medical education and its associations with behavioural and s...

    journal_title:Medical education

    pub_type: 杂志文章,多中心研究

    doi:10.1111/medu.12247

    authors: Jamali A,Tofangchiha S,Jamali R,Nedjat S,Jan D,Narimani A,Montazeri A

    更新日期:2013-10-01 00:00:00

  • The assessment of poorly performing doctors: the development of the assessment programmes for the General Medical Council's Performance Procedures.

    abstract:BACKGROUND:Modernization of medical regulation has included the introduction of the Professional Performance Procedures by the UK General Medical Council in 1995. The Council now has the power to assess any registered practitioner whose performance may be seriously deficient, thus calling registration (licensure) into ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:

    authors: Southgate L,Cox J,David T,Hatch D,Howes A,Johnson N,Jolly B,Macdonald E,McAvoy P,McCrorie P,Turner J

    更新日期:2001-12-01 00:00:00

  • To be and not to be: the paradox of the emerging professional stance.

    abstract:PURPOSE:Understanding how students resolve professional conflict is essential to teaching and evaluating professionalism. The purpose of this study was to refine an existing coding structure of rationalizations of student behaviour, and to further our understanding of students' reasoning strategies in the face of perce...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1046/j.1365-2923.2003.01326.x

    authors: Ginsburg S,Regehr G,Lingard L

    更新日期:2003-04-01 00:00:00

  • Phenomenological analysis of patient experiences of medical student teaching encounters.

    abstract:CONTEXT:It is important to know how patients are affected by becoming opportunistically involved in medical student education. In previous studies, researchers rather than patients set the research agenda and expert patients or people well known to teachers were more often involved than ordinary people. OBJECTIVES:Thi...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.2012.04332.x

    authors: McLachlan E,King N,Wenger E,Dornan T

    更新日期:2012-10-01 00:00:00

  • Non-analytical models of clinical reasoning: the role of experience.

    abstract:OBJECTIVE:This paper aims to summarise the evidence supporting the role of experience-based, non-analytic reasoning (NAR) or pattern recognition as a central feature of expert medical diagnosis. METHODS:The authors examine a series of studies, primarily from their own research programme at McMaster University, that de...

    journal_title:Medical education

    pub_type: 杂志文章,评审

    doi:10.1111/j.1365-2923.2007.02914.x

    authors: Norman G,Young M,Brooks L

    更新日期:2007-12-01 00:00:00

  • Training primary health care workers in Upper Volta.

    abstract::In collaboration with aid agencies and government health workers, Save the Children Fund medical staff have been involved in the training of primary health care workers in the Sahel region of upper Volta. This report deals specifically with the problems involved in training illiterate health workers. ...

    journal_title:Medical education

    pub_type: 杂志文章

    doi:10.1111/j.1365-2923.1982.tb01233.x

    authors: Cross I

    更新日期:1982-03-01 00:00:00