Abstract:
:The sudden increase in coronavirus disease 2019 (COVID-19) cases puts high pressure on healthcare services worldwide. At this stage, fast, accurate, and early clinical assessment of the disease severity is vital. In general, there are two issues to overcome: (1) Current deep learning-based works suffer from multimodal data adequacy issues; (2) In this scenario, multimodal (e.g., text, image) information should be taken into account together to make accurate inferences. To address these challenges, we propose a multi-modal knowledge graph attention embedding for COVID-19 diagnosis. Our method not only learns the relational embedding from nodes in a constituted knowledge graph but also has access to medical knowledge, aiming at improving the performance of the classifier through the mechanism of medical knowledge attention. The experimental results show that our approach significantly improves classification performance compared to other state-of-the-art techniques and possesses robustness for each modality from multi-modal data. Moreover, we construct a new COVID-19 multi-modal dataset based on text mining, consisting of 1393 doctor-patient dialogues and their 3706 images (347 X-ray + 2598 CT + 761 ultrasound) about COVID-19 patients and 607 non-COVID-19 patient dialogues and their 10754 images (9658 X-ray + 494 CT + 761 ultrasound), and the fine-grained labels of all. We hope this work can provide insights to the researchers working in this area to shift the attention from only medical images to the doctor-patient dialogue and its corresponding medical images.
journal_name
Inf Fusionjournal_title
An international journal on information fusionauthors
Zheng W,Yan L,Gou C,Zhang ZC,Jason Zhang J,Hu M,Wang FYdoi
10.1016/j.inffus.2021.05.015keywords:
["COVID-19 diagnose","Knowledge attention mechanism","Knowledge embedding","Knowledge-based representation learning"]subject
Has Abstractpub_date
2021-11-01 00:00:00pages
168-185eissn
1566-2535issn
1872-6305pii
S1566-2535(21)00115-9journal_volume
75pub_type
杂志文章相关文献
文献大全abstract::Multimodal fusion in neuroimaging combines data from multiple imaging modalities to overcome the fundamental limitations of individual modalities. Neuroimaging fusion can achieve higher temporal and spatial resolution, enhance contrast, correct imaging distortions, and bridge physiological and cognitive information. I...
journal_title:An international journal on information fusion
pub_type: 杂志文章
doi:10.1016/j.inffus.2020.07.006
更新日期:2020-12-01 00:00:00
abstract::Despite subjects with Dominantly-Inherited Alzheimer's Disease (DIAD) represent less than 1% of all Alzheimer's Disease (AD) cases, the Dominantly Inherited Alzheimer Network (DIAN) initiative constitutes a strong impact in the understanding of AD disease course with special emphasis on the presyptomatic disease phase...
journal_title:An international journal on information fusion
pub_type: 杂志文章
doi:10.1016/j.inffus.2020.01.001
更新日期:2020-06-01 00:00:00