Self-Supervised Feature Learning via Exploiting Multi-Modal Data for Retinal Disease Diagnosis.

Abstract:

:The automatic diagnosis of various retinal diseases from fundus images is important to support clinical decision-making. However, developing such automatic solutions is challenging due to the requirement of a large amount of human-annotated data. Recently, unsupervised/self-supervised feature learning techniques receive a lot of attention, as they do not need massive annotations. Most of the current self-supervised methods are analyzed with single imaging modality and there is no method currently utilize multi-modal images for better results. Considering that the diagnostics of various vitreoretinal diseases can greatly benefit from another imaging modality, e.g., FFA, this paper presents a novel self-supervised feature learning method by effectively exploiting multi-modal data for retinal disease diagnosis. To achieve this, we first synthesize the corresponding FFA modality and then formulate a patient feature-based softmax embedding objective. Our objective learns both modality-invariant features and patient-similarity features. Through this mechanism, the neural network captures the semantically shared information across different modalities and the apparent visual similarity between patients. We evaluate our method on two public benchmark datasets for retinal disease diagnosis. The experimental results demonstrate that our method clearly outperforms other self-supervised feature learning methods and is comparable to the supervised baseline. Our code is available at GitHub.

journal_name

IEEE Trans Med Imaging

authors

Li X,Jia M,Islam MT,Yu L,Xing L

doi

10.1109/TMI.2020.3008871

subject

Has Abstract

pub_date

2020-12-01 00:00:00

pages

4023-4033

issue

12

eissn

0278-0062

issn

1558-254X

journal_volume

39

pub_type

杂志文章
  • Elevation direction deconvolution in three-dimensional ultrasound imaging.

    abstract::One-dimensional (1-D) linear transducer arrays can be used for three-dimensional (3-D) ultrasound image acquisition. However, the relatively low spatial resolution of these arrays in the elevation direction results in blurry 3-D images. Here, the authors introduce an elevation direction deconvolution (EDD) method that...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.500148

    authors: Wang X,Ritchie CJ,Kim Y

    更新日期:1996-01-01 00:00:00

  • 3-D reconstruction of tissue components for atherosclerotic human arteries using ex vivo high-resolution MRI.

    abstract::Automatic computer-based methods are well suited for the image analysis of the different components in atherosclerotic plaques. Although several groups work on such analysis some of the methods used are oversimplified and require improvements when used within a computational framework for predicting meaningful stress ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2006.870485

    authors: Auer M,Stollberger R,Regitnig P,Ebner F,Holzapfel GA

    更新日期:2006-03-01 00:00:00

  • A fast Bayesian reconstruction algorithm for emission tomography with entropy prior converging to feasible images.

    abstract::The development and tests of an iterative reconstruction algorithm for emission tomography based on Bayesian statistical concepts are described. The algorithm uses the entropy of the generated image as a prior distribution, can be accelerated by the choice of an exponent, and converges uniformly to feasible images by ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.56340

    authors: Nunez J,Llacer J

    更新日期:1990-01-01 00:00:00

  • Digital processing for improvement of ultrasonic abdominal images.

    abstract::Digital processing that increases resolution by spatial deconvolution and histogram-based amplitude mapping has been used to improve ultrasonic abdominal image quality. The processing was applied to pulse-echo ultrasound data obtained from clinical imaging instrumentation modified to permit digital recording of signal...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.1983.4307617

    authors: Liu CN,Fatemi M,Waag RC

    更新日期:1983-01-01 00:00:00

  • Authors' reply.

    abstract::The comment of Stig Steenstrup on the use of gray level entropy function in our paper' has been addressed. The entropy measure has not been used in the procedure of enhancement, but it has only been used to show statistically that the enhanced images have larger gray level entropy, and therefore better utilize the gra...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.1987.4307802

    authors: Dhawan AP,Gordon R

    更新日期:1987-01-01 00:00:00

  • Statistical distributions of DCT coefficients and their application to an interframe compression algorithm for 3-D medical images.

    abstract::Displacement estimated interframe (DEI) coding, a coding scheme for 3-D medical image data sets such as X-ray computed tomography (CT) or magnetic resonance (MR) images, is presented. To take advantage of the correlation between contiguous slices, a displacement-compensated difference image based on the previous image...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.241875

    authors: Lee H,Kim Y,Rowberg AH,Riskin EA

    更新日期:1993-01-01 00:00:00

  • Adaptive reproducing kernel particle method for extraction of the cortical surface.

    abstract::We propose a novel adaptive approach based on the Reproducing Kernel Particle Method (RKPM) to extract the cortical surfaces of the brain from three-dimensional (3-D) magnetic resonance images (MRIs). To formulate the discrete equations of the deformable model, a flexible particle shape function is employed in the Gal...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/tmi.2006.873614

    authors: Xu M,Thompson PM,Toga AW

    更新日期:2006-06-01 00:00:00

  • Nonrigid image registration using conditional mutual information.

    abstract::Maximization of mutual information (MMI) is a popular similarity measure for medical image registration. Although its accuracy and robustness has been demonstrated for rigid body image registration, extending MMI to nonrigid image registration is not trivial and an active field of research. We propose conditional mutu...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2009.2021843

    authors: Loeckx D,Slagmolen P,Maes F,Vandermeulen D,Suetens P

    更新日期:2010-01-01 00:00:00

  • Optoacoustic Dermoscopy of the Human Skin: Tuning Excitation Energy for Optimal Detection Bandwidth With Fast and Deep Imaging in vivo.

    abstract::Optoacoustic (photoacoustic) dermoscopy offers two principal advantages over conventional optical imaging applied in dermatology. First, it yields high-resolution cross-sectional images of the skin at depths not accessible to other non-invasive optical imaging methods. Second, by resolving absorption spectra at multip...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2017.2664142

    authors: Schwarz M,Soliman D,Omar M,Buehler A,Ovsepian SV,Aguirre J,Ntziachristos V

    更新日期:2017-06-01 00:00:00

  • Robust super-resolution volume reconstruction from slice acquisitions: application to fetal brain MRI.

    abstract::Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acq...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2010.2051680

    authors: Gholipour A,Estroff JA,Warfield SK

    更新日期:2010-10-01 00:00:00

  • Minimization of imaging gradient effects in diffusion tensor imaging.

    abstract::A new, sample independent optimization criterion for minimizing the effect of the imaging gradients, including the directional awareness they create, is defined for diffusion tensor imaging (DTI) experiments. The DTI linear algebraic framework is expanded to a normed space to design optimal diffusion gradient schemes ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2010.2090539

    authors: Özcan A

    更新日期:2011-03-01 00:00:00

  • High-Frame-Rate Echocardiography Using Coherent Compounding With Doppler-Based Motion-Compensation.

    abstract::High-frame-rate ultrasonography based on coherent compounding of unfocused beams can potentially transform the assessment of cardiac function. As it requires successive waves to be combined coherently, this approach is sensitive to high-velocity tissue motion. We investigated coherent compounding of tilted diverging w...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2016.2523346

    authors: Poree J,Posada D,Hodzic A,Tournoux F,Cloutier G,Garcia D

    更新日期:2016-07-01 00:00:00

  • Pinhole SPECT imaging: compact projection/backprojection operator for efficient algebraic reconstruction.

    abstract::We describe the efficient algebraic reconstruction (EAR) method, which applies to cone-beam tomographic reconstruction problems with a circular symmetry. Three independant steps/stages are presented, which use two symmetries and a factorization of the point spread functions (PSFs), each reducing computing times and ev...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2005.861707

    authors: Israel-Jost V,Choquet P,Salmon S,Blondet C,Sonnendrücker E,Constantinesco A

    更新日期:2006-02-01 00:00:00

  • Bidirectional Mapping-Based Domain Adaptation for Nucleus Detection in Cross-Modality Microscopy Images.

    abstract::Cell or nucleus detection is a fundamental task in microscopy image analysis and has recently achieved state-of-the-art performance by using deep neural networks. However, training supervised deep models such as convolutional neural networks (CNNs) usually requires sufficient annotated image data, which is prohibitive...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2020.3042789

    authors: Xing F,Cornish TC,Bennett TD,Ghosh D

    更新日期:2020-12-07 00:00:00

  • Lung 4D CT Image Registration Based on High-Order Markov Random Field.

    abstract::To solve the problem that traditional image registration methods based on continuous optimization for large motion lung 4D CT image sequences are easy to fall into local optimal solutions and lead to serious misregistration, a novel image registration method based on high-order Markov Random Field (MRF) is proposed. B...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2019.2937458

    authors: Xue P,Dong E,Ji H

    更新日期:2020-04-01 00:00:00

  • Computation of the mid-sagittal plane in 3-D brain images.

    abstract::We present a new method to automatically compute, reorient, and recenter the mid-sagittal plane in anatomical and functional three-dimensional (3-D) brain images. This iterative approach is composed of two steps. At first, given an initial guess of the mid-sagittal plane (generally, the central plane of the image grid...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.993131

    authors: Prima S,Ourselin S,Ayache N

    更新日期:2002-02-01 00:00:00

  • A new algorithm for border description of polarized light surface microscopic images of pigmented skin lesions.

    abstract::The aim of this study was to provide mathematical descriptors for the border of pigmented skin lesion images and to assess their efficacy for distinction among different lesion groups. New descriptors such as lesion slope and lesion slope regularity are introduced and mathematically defined. A new algorithm based on t...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2003.815901

    authors: Grana C,Pellacani G,Cucchiara R,Seidenari S

    更新日期:2003-08-01 00:00:00

  • Optimizing rod window width in positron emission tomography.

    abstract::A technique determines the optimal window width for orbiting rod transmission studies in positron emission tomography (PET). Windowing reduces noise in orbiting rod transmission studies. Lines-of-response (LOR) which intersect the rods generate primarily true coincidence events. LOR which pass far from the rods genera...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.387708

    authors: Jones WF,Digby WM,Luk WK,Casey ME,Byars LG

    更新日期:1995-01-01 00:00:00

  • Fluorescence Diffusion in the Presence of Optically Clear Tissues in a Mouse Head Model.

    abstract::Diffuse Optical Tomography commonly neglects or assumes as insignificant the presence of optically clear regions in biological tissues, estimating their contribution as a small perturbation to light transport. The inaccuracy introduced by this practice is examined in detail in the context of a complete, based on reali...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2016.2646518

    authors: Ancora D,Zacharopoulos A,Ripoll J,Zacharakis G

    更新日期:2017-05-01 00:00:00

  • Spatial transformation of DWI data using non-negative sparse representation.

    abstract::This paper presents an algorithm to transform and reconstruct diffusion-weighted imaging (DWI) data for alignment of micro-structures in association with spatial transformations. The key idea is to decompose the diffusion signal profile, a function defined on a unit sphere, into a series of weighted diffusion basis fu...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2012.2204766

    authors: Yap PT,Shen D

    更新日期:2012-11-01 00:00:00

  • Co-registration of white matter tractographies by adaptive-mean-shift and Gaussian mixture modeling.

    abstract::In this paper, we present a robust approach to the registration of white matter tractographies extracted from diffusion tensor-magnetic resonance imaging scans. The fibers are projected into a high dimensional feature space based on the sequence of their 3-D coordinates. Adaptive mean-shift clustering is applied to ex...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2009.2029097

    authors: Zvitia O,Mayer A,Shadmi R,Miron S,Greenspan HK

    更新日期:2010-01-01 00:00:00

  • Unified bundling and registration of brain white matter fibers.

    abstract::Magnetic resonance diffusion tensor imaging is being widely used to reconstruct brain white matter fiber tracts. To characterize structural properties of the tracts, reconstructed fibers are often grouped into bundles that correspond to coherent anatomic structures. For further group analysis of fiber bundles, it is d...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2009.2016337

    authors: Xu Q,Anderson AW,Gore JC,Ding Z

    更新日期:2009-09-01 00:00:00

  • Semi-automatic segmentation of vascular network images using a rotating structuring element (ROSE) with mathematical morphology and dual feature thresholding.

    abstract::A method for measuring the spatial concentration of specific categories of vessels in a vascular network consisting of vessels of several diameters, lengths, and orientations is demonstrated. It is shown that a combination of the mathematical morphology operation, opening, with a linear rotating structuring element (R...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.241865

    authors: Thackray BD,Nelson AC

    更新日期:1993-01-01 00:00:00

  • Computer-assisted scan protocol and reconstruction (CASPAR)-reduction of image noise and patient dose.

    abstract::X-ray computed tomography is a powerful medical imaging device. It allows high-resolution 3-D visualization of the human body. However, one drawback is the health risk associated with ionizing radiation. Simply downscaling the radiation intensities over the entire scan results in increased quantum noise. This paper pr...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2009.2034515

    authors: Sperl J,Beque D,Claus B,De Man B,Senzig B,Brokate M

    更新日期:2010-03-01 00:00:00

  • Revealing the Microscopic Structure of Human Renal Cell Carcinoma in Three Dimensions.

    abstract::For fully characterizing renal cell carcinoma (RCC), information about the 3D tissue microstructure is essential. Histopathology, which represents the current diagnostic gold standard, is destructive and only provides 2D information. 3D X-ray histology endeavors to overcome these limitations by generating 3D data. In ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2019.2952028

    authors: Ferstl S,Busse M,Muller M,Kimm MA,Drecoll E,Burkner T,Allner S,Dierolf M,Pfeiffer D,Rummeny EJ,Weichert W,Pfeiffer F

    更新日期:2020-05-01 00:00:00

  • Accuracy of in vivo neuroreceptor quantification by PET and review of steady-state, transient, double injection, and equilibrium models.

    abstract::The accuracy of in vivo dopamine D2 receptor quantification by positron emission tomography (PET) was determined for several models by means of singular-value decomposition, and some of the model assumptions were reviewed. These include steady-state, transient, double injection, and equilibrium approaches. All four mo...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.7783

    authors: Zeeberg BR,Gibson RE,Reba RC

    更新日期:1988-01-01 00:00:00

  • An improved algorithm for 2-D translational motion artifact correction.

    abstract::The quality of magnetic resonance imaging systems has improved to the point that motion is a major limitation in many examinations. Translational motion in the imaging plane causes the phase of the data to be corrupted. An algorithm using computer post-processing is proposed to correct the phase of the data, and hence...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.108589

    authors: Medley M,Yan H,Rosenfeld D

    更新日期:1991-01-01 00:00:00

  • Two iterative image restoration algorithms with applications to nuclear medicine.

    abstract::Two methods for recovering an image that has been degraded while being processed are presented. The restoration problem is formulated as a constrained optimization problem in which a measure of smoothness based on the second derivatives of the restored image is maximized subject to the constraint that noise energy is ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.126904

    authors: Charalambous C,Ghaddar FK,Kouris K

    更新日期:1992-01-01 00:00:00

  • Efficient Compressed Sensing SENSE pMRI Reconstruction With Joint Sparsity Promotion.

    abstract::The theory and techniques of compressed sensing (CS) have shown their potential as a breakthrough in accelerating k-space data acquisition for parallel magnetic resonance imaging (pMRI). However, the performance of CS reconstruction models in pMRI has not been fully maximized, and CS recovery guarantees for pMRI are l...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/TMI.2015.2474383

    authors: Chun IY,Adcock B,Talavage TM

    更新日期:2016-01-01 00:00:00

  • Digital flashing tomosynthesis: a promising technique for angiocardiographic screening.

    abstract::A system for digital flashing tomosynthesis (DFTS) consists of four ECG-gated simultaneously flashed X-ray tubes, a 14-in image intensifier, a unit for digital subtraction angiography (DSA), a PC-hosted transputer network for three-dimensional (3-D) reconstruction as well as for quantitative coronary angiography and v...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章

    doi:10.1109/42.232261

    authors: Stiel GM,Stiel LG,Klotz E,Nienaber CA

    更新日期:1993-01-01 00:00:00