SDAE-GAN: Enable high-dimensional pathological images in liver cancer survival prediction with a policy gradient based data augmentation method.

Abstract:

:High-dimensional pathological images produced by Immunohistochemistry (IHC) methods consist of many pathological indexes, which play critical roles in cancer treatment planning. However, these indexes currently cannot be utilized in survival prediction because joining them with patients' clinicopathological features (e.g., age and tumor size) is challenging due to their high dimension and sparse features. To address this problem, we propose a novel two-stage survival prediction model named ICSPM to join the IHC images and clinicopathological features. For the first stage, our proposed SDAE-GAN compresses high-dimensional IHC images to flat, compact and representative feature vectors by compressing and reconstructing them. For the first time, SDAE-GAN integrates dense blocks, the stacked auto-encoder and the GAN architecture to maximize the ability to detect patterns in IHC images. In addition, we propose a novel policy gradient based data augmentation method to involve the diversity in IHC images without breaking patterns inside them. For the second stage, ICSPM adopts a DenseNet to join feature vectors and clinicopathological features for survival prediction. Experimental results demonstrate that ICSPM reached a state-of-the-art prediction accuracy of 0.72 on the five-year survival. ICSPM is the first work to enable high-dimensional IHC images in cancer survival prediction. We prove that high-dimensional IHC images and clinicopathological features provide valuable and complementary information in survival prediction.

journal_name

Med Image Anal

journal_title

Medical image analysis

authors

Wu H,Gao R,Sheng YP,Chen B,Li S

doi

10.1016/j.media.2020.101640

subject

Has Abstract

pub_date

2020-05-01 00:00:00

pages

101640

eissn

1361-8415

issn

1361-8423

pii

S1361-8415(20)30007-4

journal_volume

62

pub_type

杂志文章
  • Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net.

    abstract::We propose a novel airway segmentation method in volumetric chest computed tomography (CT) and evaluate its performance on multiple datasets. The segmentation is performed voxel-by-voxel by a 2.5D convolutional neural net (2.5D CNN) trained in a supervised manner. To enhance the accuracy of the segmented airway tree, ...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2018.10.006

    authors: Yun J,Park J,Yu D,Yi J,Lee M,Park HJ,Lee JG,Seo JB,Kim N

    更新日期:2019-01-01 00:00:00

  • Comparison of atlas-based techniques for whole-body bone segmentation.

    abstract::We evaluate the accuracy of whole-body bone extraction from whole-body MR images using a number of atlas-based segmentation methods. The motivation behind this work is to find the most promising approach for the purpose of MRI-guided derivation of PET attenuation maps in whole-body PET/MRI. To this end, a variety of a...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2016.11.003

    authors: Arabi H,Zaidi H

    更新日期:2017-02-01 00:00:00

  • Ultrasonic and elasticity imaging to model disease-induced changes in soft-tissue structure.

    abstract::Ultrasonic techniques are presented for the study of soft biological tissue structure and function. Changes in echo waveforms caused by microscopic variations in the mechanical properties of tissue can reveal disease mechanism, in vivo. On a larger scale, elasticity imaging describes the macroscopic mechanical propert...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/s1361-8415(98)80014-5

    authors: Chaturvedi P,Insana MF,Hall TJ

    更新日期:1998-12-01 00:00:00

  • A comprehensive study of stent visualization enhancement in X-ray images by image processing means.

    abstract::In this work we propose a comprehensive study of Digital Stent Enhancement (DSE), from the analysis of the requirements to the validation of the proposed solution. First, we derive the stent visualization requirements in the context of the clinical application and workflow. Then, we propose a DSE algorithm combining a...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2011.03.002

    authors: Bismuth V,Vaillant R,Funck F,Guillard N,Najman L

    更新日期:2011-08-01 00:00:00

  • CT male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation.

    abstract::Accurate segmentation of the prostate and organs at risk (e.g., bladder and rectum) in CT images is a crucial step for radiation therapy in the treatment of prostate cancer. However, it is a very challenging task due to unclear boundaries, large intra- and inter-patient shape variability, and uncertain existence of bo...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.03.003

    authors: Wang S,He K,Nie D,Zhou S,Gao Y,Shen D

    更新日期:2019-05-01 00:00:00

  • Automated comprehensive Adolescent Idiopathic Scoliosis assessment using MVC-Net.

    abstract::Automated quantitative estimation of spinal curvature is an important task for the ongoing evaluation and treatment planning of Adolescent Idiopathic Scoliosis (AIS). It solves the widely accepted disadvantage of manual Cobb angle measurement (time-consuming and unreliable) which is currently the gold standard for AIS...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2018.05.005

    authors: Wu H,Bailey C,Rasoulinejad P,Li S

    更新日期:2018-08-01 00:00:00

  • Semi-supervised mp-MRI data synthesis with StitchLayer and auxiliary distance maximization.

    abstract::The availability of a large amount of annotated data is critical for many medical image analysis applications, in particular for those relying on deep learning methods which are known to be data-hungry. However, annotated medical data, especially multimodal data, is often scarce and costly to obtain. In this paper, we...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.101565

    authors: Wang Z,Lin Y,Cheng KT,Yang X

    更新日期:2020-01-01 00:00:00

  • Development and comparison of new hybrid motion tracking for bronchoscopic navigation.

    abstract::This paper presents a new hybrid camera motion tracking method for bronchoscopic navigation combining SIFT, epipolar geometry analysis, Kalman filtering, and image registration. In a thorough evaluation, we compare it to state-of-the-art tracking methods. Our hybrid algorithm for predicting bronchoscope motion uses SI...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.11.001

    authors: Luó X,Feuerstein M,Deguchi D,Kitasaka T,Takabatake H,Mori K

    更新日期:2012-04-01 00:00:00

  • Unsupervised boundary delineation of spinal neural foramina using a multi-feature and adaptive spectral segmentation.

    abstract::As a common disease in the elderly, neural foramina stenosis (NFS) brings a significantly negative impact on the quality of life due to its symptoms including pain, disability, fall risk and depression. Accurate boundary delineation is essential to the clinical diagnosis and treatment of NFS. However, existing clinica...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2016.10.009

    authors: He X,Zhang H,Landis M,Sharma M,Warrington J,Li S

    更新日期:2017-02-01 00:00:00

  • Shape regression machine and efficient segmentation of left ventricle endocardium from 2D B-mode echocardiogram.

    abstract::We present a machine learning approach called shape regression machine (SRM) for efficient segmentation of an anatomic structure that exhibits a deformable shape in a medical image, e.g., left ventricle endocardial wall in an echocardiogram. The SRM achieves efficient segmentation via statistical learning of the inter...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.04.002

    authors: Zhou SK

    更新日期:2010-08-01 00:00:00

  • Predicting the progression of mild cognitive impairment using machine learning: A systematic, quantitative and critical review.

    abstract::We performed a systematic review of studies focusing on the automatic prediction of the progression of mild cognitive impairment to Alzheimer's disease (AD) dementia, and a quantitative analysis of the methodological choices impacting performance. This review included 172 articles, from which 234 experiments were extr...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101848

    authors: Ansart M,Epelbaum S,Bassignana G,Bône A,Bottani S,Cattai T,Couronné R,Faouzi J,Koval I,Louis M,Thibeau-Sutre E,Wen J,Wild A,Burgos N,Dormont D,Colliot O,Durrleman S

    更新日期:2021-01-01 00:00:00

  • Statistical shape models for 3D medical image segmentation: a review.

    abstract::Statistical shape models (SSMs) have by now been firmly established as a robust tool for segmentation of medical images. While 2D models have been in use since the early 1990 s, wide-spread utilization of three-dimensional models appeared only in recent years, primarily made possible by breakthroughs in automatic dete...

    journal_title:Medical image analysis

    pub_type: 杂志文章,评审

    doi:10.1016/j.media.2009.05.004

    authors: Heimann T,Meinzer HP

    更新日期:2009-08-01 00:00:00

  • A spatiotemporal statistical atlas of motion for the quantification of abnormal myocardial tissue velocities.

    abstract::In this paper, we present a new method for the automatic comparison of myocardial motion patterns and the characterization of their degree of abnormality, based on a statistical atlas of motion built from a reference healthy population. Our main contribution is the computation of atlas-based indexes that quantify the ...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.12.006

    authors: Duchateau N,De Craene M,Piella G,Silva E,Doltra A,Sitges M,Bijnens BH,Frangi AF

    更新日期:2011-06-01 00:00:00

  • Piecewise-diffeomorphic image registration: application to the motion estimation between 3D CT lung images with sliding conditions.

    abstract::In this paper, we propose a new strategy for modelling sliding conditions when registering 3D images in a piecewise-diffeomorphic framework. More specifically, our main contribution is the development of a mathematical formalism to perform Large Deformation Diffeomorphic Metric Mapping registration with sliding condit...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2012.10.001

    authors: Risser L,Vialard FX,Baluwala HY,Schnabel JA

    更新日期:2013-02-01 00:00:00

  • Self-similarity weighted mutual information: a new nonrigid image registration metric.

    abstract::Mutual information (MI) has been widely used as a similarity measure for rigid registration of multi-modal and uni-modal medical images. However, robust application of MI to deformable registration is challenging mainly because rich structural information, which are critical cues for successful deformable registration...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2013.12.003

    authors: Rivaz H,Karimaghaloo Z,Collins DL

    更新日期:2014-02-01 00:00:00

  • Intrasubject multimodal groupwise registration with the conditional template entropy.

    abstract::Image registration is an important task in medical image analysis. Whereas most methods are designed for the registration of two images (pairwise registration), there is an increasing interest in simultaneously aligning more than two images using groupwise registration. Multimodal registration in a groupwise setting r...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2018.02.003

    authors: Polfliet M,Klein S,Huizinga W,Paulides MM,Niessen WJ,Vandemeulebroucke J

    更新日期:2018-05-01 00:00:00

  • Major depressive disorder identification by referenced multiset canonical correlation analysis with clinical scores.

    abstract::A novel method based on multiset canonical correlation analysis (mCCA) and linear discriminant analysis (LDA) is presented to identify the major depressive disorder (MDD). The new method comprises two parts, namely, the mCCA-rreg and sparse LDA models. The mCCA-rreg model extends the classical canonical correlation mo...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.101600

    authors: Lin W,Lv D,Han Z,Dong J,Yang L

    更新日期:2020-02-01 00:00:00

  • Reducing annotation effort in digital pathology: A Co-Representation learning framework for classification tasks.

    abstract::Classification of digital pathology images is imperative in cancer diagnosis and prognosis. Recent advancements in deep learning and computer vision have greatly benefited the pathology workflow by developing automated solutions for classification tasks. However, the cost and time for acquiring high quality task-speci...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101859

    authors: Pati P,Foncubierta-Rodríguez A,Goksel O,Gabrani M

    更新日期:2021-01-01 00:00:00

  • A gradient-based optical-flow cardiac motion estimation method for cine and tagged MR images.

    abstract::A new method is proposed to quantify the myocardial motion from both 2D C(ine)-MRI and T(agged)-MRI sequences. The tag pattern offers natural landmarks within the image that makes it possible to accurately quantify the motion within the myocardial wall. Therefore, several methods have been proposed for T-MRI. However,...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.06.016

    authors: Wang L,Clarysse P,Liu Z,Gao B,Liu W,Croisille P,Delachartre P

    更新日期:2019-10-01 00:00:00

  • MR to ultrasound registration for image-guided prostate interventions.

    abstract::A deformable registration method is described that enables automatic alignment of magnetic resonance (MR) and 3D transrectal ultrasound (TRUS) images of the prostate gland. The method employs a novel "model-to-image" registration approach in which a deformable model of the gland surface, derived from an MR image, is r...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.11.003

    authors: Hu Y,Ahmed HU,Taylor Z,Allen C,Emberton M,Hawkes D,Barratt D

    更新日期:2012-04-01 00:00:00

  • Group-level cortical surface parcellation with sulcal pits labeling.

    abstract::Sulcal pits are the points of maximal depth within the folds of the cortical surface. These shape descriptors give a unique opportunity to access to a rich, fine-scale representation of the geometry and the developmental milestones of the cortical surface. However, using sulcal pits analysis at group level requires ne...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101749

    authors: Kaltenmark I,Deruelle C,Brun L,Lefèvre J,Coulon O,Auzias G

    更新日期:2020-12-01 00:00:00

  • SNR enhancement of highly-accelerated real-time cardiac MRI acquisitions based on non-local means algorithm.

    abstract::Real-time cardiac MRI appears as a promising technique to evaluate the mechanical function of the heart. However, ultra-fast MRI acquisitions come with an important signal-to-noise ratio (SNR) penalty, which drastically reduces the image quality. Hence, a real-time denoising approach would be desirable for SNR amelior...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2009.05.006

    authors: Naegel B,Cernicanu A,Hyacinthe JN,Tognolini M,Vallée JP

    更新日期:2009-08-01 00:00:00

  • An efficient Riemannian statistical shape model using differential coordinates: With application to the classification of data from the Osteoarthritis Initiative.

    abstract::We propose a novel Riemannian framework for statistical analysis of shapes that is able to account for the nonlinearity in shape variation. By adopting a physical perspective, we introduce a differential representation that puts the local geometric variability into focus. We model these differential coordinates as ele...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2017.09.004

    authors: von Tycowicz C,Ambellan F,Mukhopadhyay A,Zachow S

    更新日期:2018-01-01 00:00:00

  • Deformable image registration by combining uncertainty estimates from supervoxel belief propagation.

    abstract::Discrete optimisation strategies have a number of advantages over their continuous counterparts for deformable registration of medical images. For example: it is not necessary to compute derivatives of the similarity term; dense sampling of the search space reduces the risk of becoming trapped in local optima; and (in...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2015.09.005

    authors: Heinrich MP,Simpson IJ,Papież BW,Brady SM,Schnabel JA

    更新日期:2016-01-01 00:00:00

  • Clavicle segmentation in chest radiographs.

    abstract::Automated delineation of anatomical structures in chest radiographs is difficult due to superimposition of multiple structures. In this work an automated technique to segment the clavicles in posterior-anterior chest radiographs is presented in which three methods are combined. Pixel classification is applied in two s...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2012.06.009

    authors: Hogeweg L,Sánchez CI,de Jong PA,Maduskar P,van Ginneken B

    更新日期:2012-12-01 00:00:00

  • Efficient multi-modal dense field non-rigid registration: alignment of histological and section images.

    abstract::We describe a new algorithm for non-rigid registration capable of estimating a constrained dense displacement field from multi-modal image data. We applied this algorithm to capture non-rigid deformation between digital images of histological slides and digital flat-bed scanned images of cryotomed sections of the lary...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2005.04.003

    authors: du Bois d'Aische A,Craene MD,Geets X,Gregoire V,Macq B,Warfield SK

    更新日期:2005-12-01 00:00:00

  • Unmixing dynamic PET images with variable specific binding kinetics.

    abstract::To analyze dynamic positron emission tomography (PET) images, various generic multivariate data analysis techniques have been considered in the literature, such as principal component analysis (PCA), independent component analysis (ICA), factor analysis and nonnegative matrix factorization (NMF). Nevertheless, these c...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2018.07.011

    authors: Cavalcanti YC,Oberlin T,Dobigeon N,Stute S,Ribeiro M,Tauber C

    更新日期:2018-10-01 00:00:00

  • Segmentation of pulmonary nodules of various densities with morphological approaches and convexity models.

    abstract::Accurate segmentation of a pulmonary nodule is an important and active area of research in medical image processing. Although many algorithms have been reported in literature for this problem, those that are applicable to various density types have not been available until recently. In this paper, we propose a new alg...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.08.005

    authors: Kubota T,Jerebko AK,Dewan M,Salganicoff M,Krishnan A

    更新日期:2011-02-01 00:00:00

  • Multi-modal volume registration by maximization of mutual information.

    abstract::A new information-theoretic approach is presented for finding the registration of volumetric medical images of differing modalities. Registration is achieved by adjustment of the relative position and orientation until the mutual information between the images is maximized. In our derivation of the registration proced...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/s1361-8415(01)80004-9

    authors: Wells WM 3rd,Viola P,Atsumi H,Nakajima S,Kikinis R

    更新日期:1996-03-01 00:00:00

  • Computer aided diagnosis of thyroid nodules based on the devised small-datasets multi-view ensemble learning.

    abstract::With the development of deep learning, its application in diagnosis of benign and malignant thyroid nodules has been widely concerned. However, it is difficult to obtain medical images, resulting in insufficient number of data, which contradicts the large amount of data required for acquiring effective deep learning d...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101819

    authors: Chen Y,Li D,Zhang X,Jin J,Shen Y

    更新日期:2021-01-01 00:00:00