当前位置： SCI文献检索 > JOURNAL OF BIOMEDICAL INFORMATICS期刊下所有文献 > An unsupervised and customizable misspelling generator for mining noisy health-related text sources.

An unsupervised and customizable misspelling generator for mining noisy health-related text sources.

Abstract：

BACKGROUND:Data collection and extraction from noisy text sources such as social media typically rely on keyword-based searching/listening. However, health-related terms are often misspelled in such noisy text sources due to their complex morphology, resulting in the exclusion of relevant data for studies. In this paper, we present a customizable data-centric system that automatically generates common misspellings for complex health-related terms, which can improve the data collection process from noisy text sources. MATERIALS AND METHODS:The spelling variant generator relies on a dense vector model learned from large, unlabeled text, which is used to find semantically close terms to the original/seed keyword, followed by the filtering of terms that are lexically dissimilar beyond a given threshold. The process is executed recursively, converging when no new terms similar (lexically and semantically) to the seed keyword are found. The weighting of intra-word character sequence similarities allows further problem-specific customization of the system. RESULTS:On a dataset prepared for this study, our system outperforms the current state-of-the-art medication name variant generator with best F1-score of 0.69 and F14-score of 0.78. Extrinsic evaluation of the system on a set of cancer-related terms demonstrated an increase of over 67% in retrieval rate from Twitter posts when the generated variants are included. DISCUSSION:Our proposed spelling variant generator has several advantages over past spelling variant generators-(i) it is capable of filtering out lexically similar but semantically dissimilar terms, (ii) the number of variants generated is low, as many low-frequency and ambiguous misspellings are filtered out, and (iii) the system is fully automatic, customizable and easily executable. While the base system is fully unsupervised, we show how supervision may be employed to adjust weights for task-specific customizations. CONCLUSION:The performance and relative simplicity of our proposed approach make it a much-needed spelling variant generation resource for health-related text mining from noisy sources. The source code for the system has been made publicly available for research.

journal_name

J Biomed Inform

journal_title

Journal of biomedical informatics

authors

Sarker A,Gonzalez-Hernandez G

doi

10.1016/j.jbi.2018.11.007

subject

Has Abstract

pub_date

2018-12-01 00:00:00

pages

98-107

eissn

1532-0464

issn

1532-0480

pii

S1532-0464(18)30216-8

journal_volume

pub_type

杂志文章

在线工具

An unsupervised and customizable misspelling generator for mining noisy health-related text sources.