HookNet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images.

Abstract:

:We propose HookNet, a semantic segmentation model for histopathology whole-slide images, which combines context and details via multiple branches of encoder-decoder convolutional neural networks. Concentric patches at multiple resolutions with different fields of view, feed different branches of HookNet, and intermediate representations are combined via a hooking mechanism. We describe a framework to design and train HookNet for achieving high-resolution semantic segmentation and introduce constraints to guarantee pixel-wise alignment in feature maps during hooking. We show the advantages of using HookNet in two histopathology image segmentation tasks where tissue type prediction accuracy strongly depends on contextual information, namely (1) multi-class tissue segmentation in breast cancer and, (2) segmentation of tertiary lymphoid structures and germinal centers in lung cancer. We show the superiority of HookNet when compared with single-resolution U-Net models working at different resolutions as well as with a recently published multi-resolution model for histopathology image segmentation. We have made HookNet publicly available by releasing the source code1 as well as in the form of web-based applications2,3 based on the grand-challenge.org platform.

journal_name

Med Image Anal

journal_title

Medical image analysis

authors

van Rijthoven M,Balkenhol M,Siliņa K,van der Laak J,Ciompi F

doi

10.1016/j.media.2020.101890

subject

Has Abstract

pub_date

2021-02-01 00:00:00

pages

101890

eissn

1361-8415

issn

1361-8423

pii

S1361-8415(20)30254-1

journal_volume

68

pub_type

杂志文章
  • Tongue contour tracking in dynamic ultrasound via higher-order MRFs and efficient fusion moves.

    abstract::Analyses of the human tongue motion as captured from 2D dynamic ultrasound data often requires segmentation of the mid-sagittal tongue contours. However, semi-automatic extraction of the tongue shape presents practical challenges. We approach this segmentation problem by proposing a novel higher-order Markov random fi...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2012.07.001

    authors: Tang L,Bressmann T,Hamarneh G

    更新日期:2012-12-01 00:00:00

  • Quantitative analysis of retinal OCT.

    abstract::Clinical acceptance of 3-D OCT retinal imaging brought rapid development of quantitative 3-D analysis of retinal layers, vasculature, retinal lesions as well as facilitated new research in retinal diseases. One of the cornerstones of many such analyses is segmentation and thickness quantification of retinal layers and...

    journal_title:Medical image analysis

    pub_type: 社论

    doi:10.1016/j.media.2016.06.001

    authors: Sonka M,Abràmoff MD

    更新日期:2016-10-01 00:00:00

  • Directional wavelet based features for colonic polyp classification.

    abstract::In this work, various wavelet based methods like the discrete wavelet transform, the dual-tree complex wavelet transform, the Gabor wavelet transform, curvelets, contourlets and shearlets are applied for the automated classification of colonic polyps. The methods are tested on 8 HD-endoscopic image databases, where ea...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2016.02.001

    authors: Wimmer G,Tamaki T,Tischendorf JJ,Häfner M,Yoshida S,Tanaka S,Uhl A

    更新日期:2016-07-01 00:00:00

  • Sequential conditional reinforcement learning for simultaneous vertebral body detection and segmentation with modeling the spine anatomy.

    abstract::Accurate vertebral body (VB) detection and segmentation are critical for spine disease identification and diagnosis. Existing automatic VB detection and segmentation methods may cause false-positive results to the background tissue or inaccurate results to the desirable VB. Because they usually cannot take both the gl...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101861

    authors: Zhang D,Chen B,Li S

    更新日期:2021-01-01 00:00:00

  • Objective and expert-independent validation of retinal image registration algorithms by a projective imaging distortion model.

    abstract::Fundus camera imaging of the retina is widely used to diagnose and manage ophthalmologic disorders including diabetic retinopathy, glaucoma, and age-related macular degeneration. Retinal images typically have a limited field of view, and multiple images can be joined together using an image registration technique to f...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.04.001

    authors: Lee S,Reinhardt JM,Cattin PC,Abràmoff MD

    更新日期:2010-08-01 00:00:00

  • Disentangled representation learning in cardiac image analysis.

    abstract::Typically, a medical image offers spatial information on the anatomy (and pathology) modulated by imaging specific characteristics. Many imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) can be interpreted in this way. We can venture further and consider that a medical image na...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.101535

    authors: Chartsias A,Joyce T,Papanastasiou G,Semple S,Williams M,Newby DE,Dharmakumar R,Tsaftaris SA

    更新日期:2019-12-01 00:00:00

  • Characterization of task-free and task-performance brain states via functional connectome patterns.

    abstract::Both resting state fMRI (R-fMRI) and task-based fMRI (T-fMRI) have been widely used to study the functional activities of the human brain during task-free and task-performance periods, respectively. However, due to the difficulty in strictly controlling the participating subject's mental status and their cognitive beh...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2013.07.003

    authors: Zhang X,Guo L,Li X,Zhang T,Zhu D,Li K,Chen H,Lv J,Jin C,Zhao Q,Li L,Liu T

    更新日期:2013-12-01 00:00:00

  • Reducing annotation effort in digital pathology: A Co-Representation learning framework for classification tasks.

    abstract::Classification of digital pathology images is imperative in cancer diagnosis and prognosis. Recent advancements in deep learning and computer vision have greatly benefited the pathology workflow by developing automated solutions for classification tasks. However, the cost and time for acquiring high quality task-speci...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2020.101859

    authors: Pati P,Foncubierta-Rodríguez A,Goksel O,Gabrani M

    更新日期:2021-01-01 00:00:00

  • Cardiac image modelling: Breadth and depth in heart disease.

    abstract::With the advent of large-scale imaging studies and big health data, and the corresponding growth in analytics, machine learning and computational image analysis methods, there are now exciting opportunities for deepening our understanding of the mechanisms and characteristics of heart disease. Two emerging fields are ...

    journal_title:Medical image analysis

    pub_type: 社论,评审

    doi:10.1016/j.media.2016.06.027

    authors: Suinesiaputra A,McCulloch AD,Nash MP,Pontre B,Young AA

    更新日期:2016-10-01 00:00:00

  • Automated voxel-based 3D cortical thickness measurement in a combined Lagrangian-Eulerian PDE approach using partial volume maps.

    abstract::Accurate cortical thickness estimation is important for the study of many neurodegenerative diseases. Many approaches have been previously proposed, which can be broadly categorised as mesh-based and voxel-based. While the mesh-based approaches can potentially achieve subvoxel resolution, they usually lack the computa...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2009.07.003

    authors: Acosta O,Bourgeat P,Zuluaga MA,Fripp J,Salvado O,Ourselin S,Alzheimer's Disease Neuroimaging Initiative.

    更新日期:2009-10-01 00:00:00

  • Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow.

    abstract::We propose a method to classify cardiac pathology based on a novel approach to extract image derived features to characterize the shape and motion of the heart. An original semi-supervised learning procedure, which makes efficient use of a large amount of non-segmented images and a small amount of images segmented man...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.06.001

    authors: Zheng Q,Delingette H,Ayache N

    更新日期:2019-08-01 00:00:00

  • IDRiD: Diabetic Retinopathy - Segmentation and Grading Challenge.

    abstract::Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is chal...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.101561

    authors: Porwal P,Pachade S,Kokare M,Deshmukh G,Son J,Bae W,Liu L,Wang J,Liu X,Gao L,Wu T,Xiao J,Wang F,Yin B,Wang Y,Danala G,He L,Choi YH,Lee YC,Jung SH,Li Z,Sui X,Wu J,Li X,Zhou T,Toth J,Baran A,Kori A,Ch

    更新日期:2020-01-01 00:00:00

  • Global localization of 3D anatomical structures by pre-filtered Hough forests and discrete optimization.

    abstract::The accurate localization of anatomical landmarks is a challenging task, often solved by domain specific approaches. We propose a method for the automatic localization of landmarks in complex, repetitive anatomical structures. The key idea is to combine three steps: (1) a classifier for pre-filtering anatomical landma...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2013.02.004

    authors: Donner R,Menze BH,Bischof H,Langs G

    更新日期:2013-12-01 00:00:00

  • Automatic online layer separation for vessel enhancement in X-ray angiograms for percutaneous coronary interventions.

    abstract::Percutaneous coronary intervention is a minimally invasive procedure that is usually performed under image guidance using X-ray angiograms in which coronary arteries are opacified with contrast agent. In X-ray images, 3D objects are projected on a 2D plane, generating semi-transparent layers that overlap each other. T...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2017.04.011

    authors: Ma H,Hoogendoorn A,Regar E,Niessen WJ,van Walsum T

    更新日期:2017-07-01 00:00:00

  • Luminosity and contrast normalization in retinal images.

    abstract::Retinal images are routinely acquired and assessed to provide diagnostic evidence for many important diseases, e.g. diabetes or hypertension. Because of the acquisition process, very often these images are non-uniformly illuminated and exhibit local luminosity and contrast variability. This problem may seriously affec...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2004.07.001

    authors: Foracchia M,Grisan E,Ruggeri A

    更新日期:2005-06-01 00:00:00

  • Accurate estimation of retinal vessel width using bagged decision trees and an extended multiresolution Hermite model.

    abstract::We present an algorithm estimating the width of retinal vessels in fundus camera images. The algorithm uses a novel parametric surface model of the cross-sectional intensities of vessels, and ensembles of bagged decision trees to estimate the local width from the parameters of the best-fit surface. We report comparati...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2013.07.006

    authors: Lupaşcu CA,Tegolo D,Trucco E

    更新日期:2013-12-01 00:00:00

  • A subject-specific technique for respiratory motion correction in image-guided cardiac catheterisation procedures.

    abstract::We describe a system for respiratory motion correction of MRI-derived roadmaps for use in X-ray guided cardiac catheterisation procedures. The technique uses a subject-specific affine motion model that is quickly constructed from a short pre-procedure MRI scan. We test a dynamic MRI sequence that acquires a small numb...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2009.01.003

    authors: King AP,Boubertakh R,Rhode KS,Ma YL,Chinchapatnam P,Gao G,Tangcharoen T,Ginks M,Cooklin M,Gill JS,Hawkes DJ,Razavi RS,Schaeffter T

    更新日期:2009-06-01 00:00:00

  • Piecewise-diffeomorphic image registration: application to the motion estimation between 3D CT lung images with sliding conditions.

    abstract::In this paper, we propose a new strategy for modelling sliding conditions when registering 3D images in a piecewise-diffeomorphic framework. More specifically, our main contribution is the development of a mathematical formalism to perform Large Deformation Diffeomorphic Metric Mapping registration with sliding condit...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2012.10.001

    authors: Risser L,Vialard FX,Baluwala HY,Schnabel JA

    更新日期:2013-02-01 00:00:00

  • High resolution cortical bone thickness measurement from clinical CT data.

    abstract::The distribution of cortical bone in the proximal femur is believed to be a critical component in determining fracture resistance. Current CT technology is limited in its ability to measure cortical thickness, especially in the sub-millimetre range which lies within the point spread function of today's clinical scanne...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2010.01.003

    authors: Treece GM,Gee AH,Mayhew PM,Poole KE

    更新日期:2010-06-01 00:00:00

  • DT-MRI denoising and neuronal fiber tracking.

    abstract::Diffusion tensor imaging can provide the fundamental information required for viewing structural connectivity. However, robust and accurate acquisition and processing algorithms are needed to accurately map the nerve connectivity. In this paper, we present a novel algorithm for extracting and visualizing the fiber tra...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2003.12.001

    authors: McGraw T,Vemuri BC,Chen Y,Rao M,Mareci T

    更新日期:2004-06-01 00:00:00

  • An efficient Riemannian statistical shape model using differential coordinates: With application to the classification of data from the Osteoarthritis Initiative.

    abstract::We propose a novel Riemannian framework for statistical analysis of shapes that is able to account for the nonlinearity in shape variation. By adopting a physical perspective, we introduce a differential representation that puts the local geometric variability into focus. We model these differential coordinates as ele...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2017.09.004

    authors: von Tycowicz C,Ambellan F,Mukhopadhyay A,Zachow S

    更新日期:2018-01-01 00:00:00

  • Adaptive, template moderated, spatially varying statistical classification.

    abstract::A novel image segmentation algorithm was developed to allow the automatic segmentation of both normal and abnormal anatomy from medical images. The new algorithm is a form of spatially varying statistical classification, in which an explicit anatomical template is used to moderate the segmentation obtained by statisti...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/s1361-8415(00)00003-7

    authors: Warfield SK,Kaus M,Jolesz FA,Kikinis R

    更新日期:2000-03-01 00:00:00

  • A deep network for tissue microstructure estimation using modified LSTM units.

    abstract::Diffusion magnetic resonance imaging (dMRI) offers a unique tool for noninvasively assessing tissue microstructure. However, accurate estimation of tissue microstructure described by complicated signal models can be challenging when a reduced number of diffusion gradients are used. Deep learning based microstructure e...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.04.006

    authors: Ye C,Li X,Chen J

    更新日期:2019-07-01 00:00:00

  • A symbolic environment for visualizing activated foci in functional neuroimaging datasets.

    abstract::This paper presents a symbolic visualization environment known as the Corner Cube environment, which was developed to facilitate rapid examination and comparison of activated foci defined by analyses of functional neuroimaging datasets. We have performed a comparative evaluation of this environment against maximum-int...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/s1361-8415(98)80020-0

    authors: Rehm K,Lakshminaryan K,Frutiger S,Schaper KA,Sumners DW,Strother SC,Anderson JR,Rottenberg DA

    更新日期:1998-09-01 00:00:00

  • Ω-Net (Omega-Net): Fully automatic, multi-view cardiac MR detection, orientation, and segmentation with deep neural networks.

    abstract::Pixelwise segmentation of the left ventricular (LV) myocardium and the four cardiac chambers in 2-D steady state free precession (SSFP) cine sequences is an essential preprocessing step for a wide range of analyses. Variability in contrast, appearance, orientation, and placement of the heart between patients, clinical...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2018.05.008

    authors: Vigneault DM,Xie W,Ho CY,Bluemke DA,Noble JA

    更新日期:2018-08-01 00:00:00

  • Bayesian principal geodesic analysis for estimating intrinsic diffeomorphic image variability.

    abstract::In this paper, we present a generative Bayesian approach for estimating the low-dimensional latent space of diffeomorphic shape variability in a population of images. We develop a latent variable model for principal geodesic analysis (PGA) that provides a probabilistic framework for factor analysis in the space of dif...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2015.04.009

    authors: Zhang M,Fletcher PT

    更新日期:2015-10-01 00:00:00

  • Generalised coherent point drift for group-wise multi-dimensional analysis of diffusion brain MRI data.

    abstract::A probabilistic framework for registering generalised point sets comprising multiple voxel-wise data features such as positions, orientations and scalar-valued quantities, is proposed. It is employed for the analysis of magnetic resonance diffusion tensor image (DTI)-derived quantities, such as fractional anisotropy (...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2019.01.001

    authors: Ravikumar N,Gooya A,Beltrachini L,Frangi AF,Taylor ZA

    更新日期:2019-04-01 00:00:00

  • Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis.

    abstract::In prostate cancer radiotherapy, computed tomography (CT) is widely used for dose planning purposes. However, because CT has low soft tissue contrast, it makes manual contouring difficult for major pelvic organs. In contrast, magnetic resonance imaging (MRI) provides high soft tissue contrast, which makes it ideal for...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2017.05.004

    authors: Cao X,Yang J,Gao Y,Guo Y,Wu G,Shen D

    更新日期:2017-10-01 00:00:00

  • Image guidance in orthopaedics and traumatology: A historical perspective.

    abstract::In this note we summarize the history of computer aided surgery in orthopaedics and traumatology from the end of the nineteenth century to currently observable future trends. We concentrate on the two major components of such systems, pre-operative planning and intra-operative execution. The evolution of the necessary...

    journal_title:Medical image analysis

    pub_type: 社论

    doi:10.1016/j.media.2016.06.033

    authors: Székely G,Nolte LP

    更新日期:2016-10-01 00:00:00

  • Sensorless freehand 3D ultrasound in real tissue: speckle decorrelation without fully developed speckle.

    abstract::It has previously been demonstrated that freehand 3D ultrasound can be acquired without a position sensor by measuring the elevational speckle decorrelation from frame to frame. However, this requires that the B-scans contain significant amounts of fully developed speckle. In this paper, we show that this condition is...

    journal_title:Medical image analysis

    pub_type: 杂志文章

    doi:10.1016/j.media.2005.08.001

    authors: Gee AH,James Housden R,Hassenpflug P,Treece GM,Prager RW

    更新日期:2006-04-01 00:00:00