当前位置： SCI文献检索 > COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE期刊下所有文献 > Evaluating topic model interpretability from a primary care physician perspective.

Evaluating topic model interpretability from a primary care physician perspective.

Abstract：

BACKGROUND AND OBJECTIVE:Probabilistic topic models provide an unsupervised method for analyzing unstructured text. These models discover semantically coherent combinations of words (topics) that could be integrated in a clinical automatic summarization system for primary care physicians performing chart review. However, the human interpretability of topics discovered from clinical reports is unknown. Our objective is to assess the coherence of topics and their ability to represent the contents of clinical reports from a primary care physician's point of view. METHODS:Three latent Dirichlet allocation models (50 topics, 100 topics, and 150 topics) were fit to a large collection of clinical reports. Topics were manually evaluated by primary care physicians and graduate students. Wilcoxon Signed-Rank Tests for Paired Samples were used to evaluate differences between different topic models, while differences in performance between students and primary care physicians (PCPs) were tested using Mann-Whitney U tests for each of the tasks. RESULTS:While the 150-topic model produced the best log likelihood, participants were most accurate at identifying words that did not belong in topics learned by the 100-topic model, suggesting that 100 topics provides better relative granularity of discovered semantic themes for the data set used in this study. Models were comparable in their ability to represent the contents of documents. Primary care physicians significantly outperformed students in both tasks. CONCLUSION:This work establishes a baseline of interpretability for topic models trained with clinical reports, and provides insights on the appropriateness of using topic models for informatics applications. Our results indicate that PCPs find discovered topics more coherent and representative of clinical reports relative to students, warranting further research into their use for automatic summarization.

journal_name

Comput Methods Programs Biomed

journal_title

Computer methods and programs in biomedicine

authors

Arnold CW,Oh A,Chen S,Speier W

doi

10.1016/j.cmpb.2015.10.014

subject

Has Abstract

pub_date

2016-02-01 00:00:00

pages

67-75

eissn

0169-2607

issn

1872-7565

pii

S0169-2607(15)00274-6

journal_volume

124

pub_type

杂志文章

在线工具

Evaluating topic model interpretability from a primary care physician perspective.