Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed?

Abstract:

PURPOSE:Many criteria have been developed to rate the quality of online health information. To effectively evaluate quality, consumers must use quality criteria that can be reliably assessed. However, few instruments have been validated for inter-rater agreement. Therefore, we assessed the degree to which two raters could reliably assess 22 popularly cited quality criteria on a sample of 42 complementary and alternative medicine Web sites. METHODS:We determined the degree of inter-rater agreement by calculating the percentage agreement, Cohen's kappa, and prevalence- and bias-adjusted kappa (PABAK). RESULTS:Our un-calibrated analysis showed poor inter-rater agreement on eight of the 22 quality criteria. Therefore, we created operational definitions for each of the criteria, decreased the number of assessment choices and defined where to look for the information. As a result 18 of the 22 quality criteria were reliably assessed (inter-rater agreement > or = 0.6). CONCLUSIONS:We conclude that even with precise definitions, some commonly used quality criteria cannot be reliably assessed. However, inter-rater agreement can be improved with precise operational definitions.

journal_name

Int J Med Inform

authors

Bernstam EV,Sagaram S,Walji M,Johnson CW,Meric-Bernstam F

doi

10.1016/j.ijmedinf.2005.02.002

subject

Has Abstract

pub_date

2005-08-01 00:00:00

pages

675-83

issue

7-8

eissn

1386-5056

issn

1872-8243

pii

S1386-5056(05)00014-6

journal_volume

74

pub_type

杂志文章
  • Example of a Human Factors Engineering approach to a medication administration work system: potential impact on patient safety.

    abstract:OBJECTIVE:The objectives of this paper are: In this approach, the implementation of such a complex IT solution is considered a major redesign of the work system. The paper describes the Human Factor (HF) tasks embedded in the project lifecycle: (1) analysis and modelling of the current work system and usability assessm...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2009.07.002

    authors: Beuscart-Zéphir MC,Pelayo S,Bernonville S

    更新日期:2010-04-01 00:00:00

  • Diagnostic accuracy of chest X-rays acquired using a digital camera for low-cost teleradiology.

    abstract::Store-and-forward telemedicine, using e-mail to send clinical data and digital images, offers a low-cost alternative for physicians in developing countries to obtain second opinions from specialists. To explore the potential usefulness of this technique, 91 chest X-ray images were photographed using a digital camera a...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2003.10.002

    authors: Szot A,Jacobson FL,Munn S,Jazayeri D,Nardell E,Harrison D,Drosten R,Ohno-Machado L,Smeaton LM,Fraser HS

    更新日期:2004-02-01 00:00:00

  • Assessment of hospital processes using a process mining technique: Outpatient process analysis at a tertiary hospital.

    abstract:INTRODUCTION:Many hospitals are increasing their efforts to improve processes because processes play an important role in enhancing work efficiency and reducing costs. However, to date, a quantitative tool has not been available to examine the before and after effects of processes and environmental changes, other than ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2015.12.018

    authors: Yoo S,Cho M,Kim E,Kim S,Sim Y,Yoo D,Hwang H,Song M

    更新日期:2016-04-01 00:00:00

  • Self-assessed health among older people in Europe and internet use.

    abstract:BACKGROUND:About ten years ago, an age-related digital divide was identified, where 'the elderly' denoted a group of people at risk of losing the benefits of a digital society. The aims of this work are to find a relationship between self-assessed health and internet use by older people in European countries and to asc...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2020.104240

    authors: Tavares AI

    更新日期:2020-09-01 00:00:00

  • To decay is system: the challenges of keeping a health information system alive.

    abstract::Health information system (HIS) architecture and socio-technical approaches for system deployment have been topics of systematic research for decades. Sustainable operation in gradually changing environments, however, has not yet received sufficient attention. Even HIS that have gone life to the satisfaction of their ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2006.05.011

    authors: Wetter T

    更新日期:2007-06-01 00:00:00

  • A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach.

    abstract:BACKGROUND:Clinical information systems have long been used in intensive care units but reports on their adoption and benefits are limited. This study evaluated a Critical Care Information System implementation. METHODS:A case study summative evaluation was conducted, employing observation, interview, and document ana...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2015.03.001

    authors: Yusof MM

    更新日期:2015-07-01 00:00:00

  • Reasoning about diabetes and its relationship to the use of telecommunication technology by patients and physicians.

    abstract:UNLABELLED:Health care is moving toward a team effort, with patients as partners. This requires effective communication between physicians and patients, who have different understandings of health and illness. These understandings in turn guide their decisions about management of health and illness. With the introducti...

    journal_title:International journal of medical informatics

    pub_type: 临床试验,杂志文章

    doi:10.1016/s1386-5056(98)00111-7

    authors: Cytryn KN,Patel VL

    更新日期:1998-08-01 00:00:00

  • Videoconference compared to telephone in healthcare delivery: A systematic review.

    abstract:INTRODUCTION:Telehealth is an important tool for ensuring accessible healthcare access particularly over geographical distances. Videoconference and telephone are two common telehealth modalities, yet little is known as to the relative advantage of these modalities. Synthesizing evidence as to the impact of these modal...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2018.07.007

    authors: Rush KL,Howlett L,Munro A,Burton L

    更新日期:2018-10-01 00:00:00

  • Empowering nurses--improving care nurses' response to the new Health Services Reform Programme in Ireland.

    abstract::The Health Service Reform Programme in Ireland will be the biggest change ever seen in the health service. This programme will be dependent on information structures that provide appropriate and timely information. The Nursing Informatics Project Team in the Mater Misericordiae University Hospital, Dublin is in the pr...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2005.07.041

    authors: Murnane R

    更新日期:2005-12-01 00:00:00

  • Problem list completeness in electronic health records: A multi-site study and assessment of success factors.

    abstract:OBJECTIVE:To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness. METHODS:We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.ijmedinf.2015.06.011

    authors: Wright A,McCoy AB,Hickman TT,Hilaire DS,Borbolla D,Bowes WA 3rd,Dixon WG,Dorr DA,Krall M,Malholtra S,Bates DW,Sittig DF

    更新日期:2015-10-01 00:00:00

  • Methodologies for assessing telemedicine: a systematic review of reviews.

    abstract:BACKGROUND AND OBJECTIVES:Previous reviews have expressed concerns about the quality of telemedicine studies. There is debate about shortcomings and appropriate methodologies. The aim of this review of systematic reviews of telemedicine is to summarize methodologies used in telemedicine research, discuss knowledge gaps...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,评审

    doi:10.1016/j.ijmedinf.2011.10.009

    authors: Ekeland AG,Bowes A,Flottorp S

    更新日期:2012-01-01 00:00:00

  • Classifying disease outbreak reports using n-grams and semantic features.

    abstract:INTRODUCTION:This paper explores the benefits of using n-grams and semantic features for the classification of disease outbreak reports, in the context of the BioCaster disease outbreak report text mining system. A novel feature of this work is the use of a general purpose semantic tagger - the USAS tagger - to generat...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2009.03.010

    authors: Conway M,Doan S,Kawazoe A,Collier N

    更新日期:2009-12-01 00:00:00

  • Electronic diagnostic algorithms to assist mid-level health care workers in Nepal: a mixed-method exploratory study.

    abstract:INTRODUCTION:The use of mobile health applications for data collection and disease management by rural health care workers in developing countries has been shown to be accepted by patients and health care workers. However, the acceptances of diagnostic decision applications have not been studied. Moreover, verbal accep...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2015.01.011

    authors: Knoble SJ,Bhusal MR

    更新日期:2015-05-01 00:00:00

  • Reporting systems, reporting rates and completeness of data reported from primary healthcare to a Swedish quality register--the National Diabetes Register.

    abstract:OBJECTIVE:The aims of this paper were to study the reporting rate and completeness of data reported from primary healthcare centres (PHCCs) in Sweden to the Swedish National Diabetes Register (NDR), with a special attention on the relation between these measures and the reporting system used by the PHCCs. METHOD:A nat...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2011.06.006

    authors: Adolfsson ET,Rosenblad A

    更新日期:2011-09-01 00:00:00

  • Information on adverse drug reactions-Proof of principle for a structured database that allows customization of drug information.

    abstract:BACKGROUND:The drug information most commonly requested by patients is to learn more about potential adverse drug reactions (ADRs) of their drugs. Such information should be customizable to individual information needs. While approaches to automatically aggregate ADRs by text-mining processes and establishment of respe...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2019.103970

    authors: Kusch MKP,Zien A,Hachenberg C,Haefeli WE,Seidling HM

    更新日期:2020-01-01 00:00:00

  • Reliability of diagnoses coding with ICD-10.

    abstract:OBJECTIVE:Reliability of diagnoses coding is essential for the use of routine data in a national health care system. The present investigation compares reliability of diagnoses coding with ICD-10 between three groups of coding subjects. METHOD:One hundred and eighteen students coded 15 diagnoses lists, 27 medical mana...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2006.11.005

    authors: Stausberg J,Lehmann N,Kaczmarek D,Stein M

    更新日期:2008-01-01 00:00:00

  • FindZebra: a search engine for rare diseases.

    abstract:BACKGROUND:The web has become a primary information resource about illnesses and treatments for both medical and non-medical users. Standard web search is by far the most common interface to this information. It is therefore of interest to find out how well web search engines work for diagnostic queries and what factor...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2013.01.005

    authors: Dragusin R,Petcu P,Lioma C,Larsen B,Jørgensen HL,Cox IJ,Hansen LK,Ingwersen P,Winther O

    更新日期:2013-06-01 00:00:00

  • Impact of simulation training on self-efficacy of outpatient health care providers to use electronic health records.

    abstract:OBJECTIVES:To examine whether simulation training enhanced self-efficacy of physicians and nurses who work in the outpatient setting to use electronic medical records, and whether the training changed their perceptions about the importance of electronic medical records (EMRs) in helping patients and improving patients'...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2015.02.003

    authors: Vuk J,Anders ME,Mercado CC,Kennedy RL,Casella J,Steelman SC

    更新日期:2015-06-01 00:00:00

  • Adding insight: a qualitative cross-site study of physician order entry.

    abstract::The research questions, strategies, and results of a 7-year qualitative study of computerized physician order entry implementation (CPOE) at successful sites are reviewed over time. The iterative nature of qualitative inquiry stimulates a consecutive stream of research foci, which, with each iteration, add further ins...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.ijmedinf.2005.05.005

    authors: Ash JS,Sittig DF,Seshadri V,Dykstra RH,Carpenter JD,Stavri PZ

    更新日期:2005-08-01 00:00:00

  • Angiocardiographic digital still images compressed via irreversible methods: concepts and experiments.

    abstract::We defined, implemented and tested two new methods for irreversible compression of angiocardiographic still images: brightness error limitation (BEL) and pseudo-gradient adaptive brightness and contrast error limitation (PABCEL). The scan path used to compress the digital images is based on the Peano-Hilbert plane-fil...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/s1386-5056(97)00063-4

    authors: Portoni L,Combi C,Pozzi G,Pinciroli F,Fritsch JP,Brennecke R

    更新日期:1997-10-01 00:00:00

  • A novel concept for integrating and delivering health information using a comprehensive digital dashboard: An analysis of healthcare professionals' intention to adopt a new system and the trend of its real usage.

    abstract:OBJECTIVE:To introduce a new concept of medical dashboard system called BESTBoard. Such a system was implemented in all wards in a tertiary academic hospital to explore the development process, core designs, functions, usability and feasibility. METHODS:The task-force team made user interface designs for 6 months base...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2016.10.001

    authors: Lee K,Jung SY,Hwang H,Yoo S,Baek HY,Baek RM,Kim S

    更新日期:2017-01-01 00:00:00

  • LinkEHR-Ed: a multi-reference model archetype editor based on formal semantics.

    abstract:PURPOSE:To develop a powerful archetype editing framework capable of handling multiple reference models and oriented towards the semantic description and standardization of legacy data. METHODS:The main prerequisite for implementing tools providing enhanced support for archetypes is the clear specification of archetyp...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2009.03.006

    authors: Maldonado JA,Moner D,Boscá D,Fernández-Breis JT,Angulo C,Robles M

    更新日期:2009-08-01 00:00:00

  • Smartphone application training program improves smartphone usage competency and quality of life among the elderly in an elder university in China: A randomized controlled trial.

    abstract:BACKGROUND:Given that China has free universities dedicated solely to the elderly, such universities may be the perfect place to provide large numbers of elderly with a systematic program of self-management. Yet, testing health applications in this location has not occurred, partly because time-consuming training is re...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.ijmedinf.2019.104010

    authors: Zhao X,Wang L,Ge C,Zhen X,Chen Z,Wang J,Zhou Y

    更新日期:2020-01-01 00:00:00

  • Validating a decision support system for anti-epileptic drug treatment. Part II: adjusting anti-epileptic drug treatment.

    abstract::A model of expertise for monitoring antiepileptic drug treatment was implemented in a decision support system. We validated the advice of the system regarding treatment decisions at first follow-up with 265 paper cases based on patient records. The reference for comparison is based on the opinions of neurologists. We ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/s1386-5056(99)00052-0

    authors: Smeets R,Talmon J,Meinardi H,Hasman A

    更新日期:1999-11-01 00:00:00

  • The effect of registry-based performance feedback via short text messages and traditional postal letters on prescribing parenteral steroids by general practitioners--A randomized controlled trial.

    abstract:BACKGROUND:It is conjectured that providing feedback on physicians' prescribing behavior improves quality of drug prescriptions. However, the effectiveness of feedback provision and mode of feedback delivery is not well understood. The objective of this study was to assess and compare the effect of traditional paper le...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.ijmedinf.2015.12.008

    authors: Nejad AS,Noori MR,Haghdoost AA,Bahaadinbeigy K,Abu-Hanna A,Eslami S

    更新日期:2016-03-01 00:00:00

  • Telemedicine--a bibliometric and content analysis of 17,932 publication records.

    abstract:OBJECTIVES:We aimed to provide an up-to-date contemporary bibliometric view of the telemedicine and telehealth literature and a longitudinal analysis of changes in content themes. METHODS:Software tools were used to extract and process MEDLINE entries. Frequencies of papers by year of publication and outlet were calcu...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2014.07.001

    authors: Armfield NR,Edirippulige S,Caffery LJ,Bradford NK,Grey JW,Smith AC

    更新日期:2014-10-01 00:00:00

  • Co-authorship network analysis in cardiovascular research utilizing machine learning (2009-2019).

    abstract:BACKGROUND:With the recent advances in computational science, machine-learning methods have been increasingly used in medical research. Because such projects usually require both a clinician and a computational data scientist, there is a need for interdisciplinary research collaboration. However, there has been no publ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2020.104274

    authors: Higaki A,Uetani T,Ikeda S,Yamaguchi O

    更新日期:2020-11-01 00:00:00

  • Patients as actors: the patient's role in detecting, preventing, and recovering from medical errors.

    abstract:PURPOSE:Patients have the most to gain from reducing medical errors; yet, little research has been done to investigate the role they could or already do play in detecting and preventing errors in their own health care. The purpose of this study is to examine patient's role in detecting, preventing, and recovering from ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2006.05.021

    authors: Unruh KT,Pratt W

    更新日期:2007-06-01 00:00:00

  • Technology, governance and patient safety: systems issues in technology and patient safety.

    abstract::Technology and equipment are often identified as contributors to adverse medical events, however technology is seldom the focal point of investigation as a source of medical error or adverse event. It is often seen as both a means of reducing error (e.g., automated drug dispensing machines) or as a major contributing ...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章,评审

    doi:10.1016/j.ijmedinf.2006.05.046

    authors: Balka E,Doyle-Waters M,Lecznarowicz D,FitzGerald JM

    更新日期:2007-06-01 00:00:00

  • Predictive factors of telemedicine service acceptance and behavioral intention of physicians.

    abstract:PURPOSE:Despite the proliferation of telemedicine technology, telemedicine service acceptance has been slow in actual healthcare settings. The purpose of this research is to develop a theoretical model for explaining the predictive factors influencing physicians' willingness to use telemedicine technology to provide he...

    journal_title:International journal of medical informatics

    pub_type: 杂志文章

    doi:10.1016/j.ijmedinf.2014.05.005

    authors: Rho MJ,Choi IY,Lee J

    更新日期:2014-08-01 00:00:00