Articulated Multi-Instrument 2-D Pose Estimation Using Fully Convolutional Networks.


:Instrument detection, pose estimation, and tracking in surgical videos are an important vision component for computer-assisted interventions. While significant advances have been made in recent years, articulation detection is still a major challenge. In this paper, we propose a deep neural network for articulated multi-instrument 2-D pose estimation, which is trained on detailed annotations of endoscopic and microscopic data sets. Our model is formed by a fully convolutional detection-regression network. Joints and associations between joint pairs in our instrument model are located by the detection subnetwork and are subsequently refined through a regression subnetwork. Based on the output from the model, the poses of the instruments are inferred using maximum bipartite graph matching. Our estimation framework is powered by deep learning techniques without any direct kinematic information from a robot. Our framework is tested on single-instrument RMIT data, and also on multi-instrument EndoVis and in vivo data with promising results. In addition, the data set annotations are publicly released along with our code and model.


IEEE Trans Med Imaging


Du X,Kurmann T,Chang PL,Allan M,Ourselin S,Sznitman R,Kelly JD,Stoyanov D




Has Abstract


2018-05-01 00:00:00












  • Wavelet analysis for brain-function imaging.

    abstract::The authors present a new algorithmic procedure for the analysis of brain images. This procedure is specifically designed to image the activity and functional organization of the brain. The authors' results are tested on data collected and previously analyzed with the technique known as in vivo optical imaging of intr...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Carmona RA,Hwang WL,Frostig RD

    更新日期:1995-01-01 00:00:00

  • Application of micro-computed tomography with iodine staining to cardiac imaging, segmentation, and computational model development.

    abstract::Micro-computed tomography (micro-CT) has been widely used to generate high-resolution 3-D tissue images from small animals nondestructively, especially for mineralized skeletal tissues. However, its application to the analysis of soft cardiovascular tissues has been limited by poor inter-tissue contrast. Recent ex viv...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Aslanidi OV,Nikolaidou T,Zhao J,Smaill BH,Gilbert SH,Holden AV,Lowe T,Withers PJ,Stephenson RS,Jarvis JC,Hancox JC,Boyett MR,Zhang H

    更新日期:2013-01-01 00:00:00

  • Geometric modeling of the human normal cerebral arterial system.

    abstract::We propose an anatomy-based approach for an efficient construction of a three-dimensional human normal cerebral arterial model from segmented and skeletonized angiographic data. The centerline-based model is used for an accurate angiographic data representation. A vascular tree is represented by tubular segments and b...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Volkau I,Zheng W,Baimouratov R,Aziz A,Nowinski WL

    更新日期:2005-04-01 00:00:00

  • DetexNet: Accurately Diagnosing Frequent and Challenging Pediatric Malignant Tumors.

    abstract::The most frequent extracranial solid tumors of childhood, named peripheral neuroblastic tumors (pNTs), are very challenging to diagnose due to their diversified categories and varying forms. Auxiliary diagnosis methods of such pediatric malignant cancers are highly needed to provide pathologists assistance and reduce ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Liu Y,Yin M,Sun S

    更新日期:2021-01-01 00:00:00

  • Iterative image reconstruction using inverse fourier rebinning for fully 3-D PET.

    abstract::We describe a fast forward and back projector pair based on inverse Fourier rebinning for use in iterative image reconstruction for fully three-dimensional (3-D) positron emission tomography (PET). The projector pair is used as part of a factored system matrix that takes into account detector-pair response by using sh...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Cho S,Li Q,Ahn S,Bai B,Leahy RM

    更新日期:2007-03-01 00:00:00

  • Using human and model performance to compare MRI reconstructions.

    abstract::Magnetic resonance imaging (MRI) reconstruction techniques are often validated with signal-to-noise ratio (SNR), contrast-to-noise ratio, and mean-to-standard-deviation ratio measured on example images. We present human and model observers as a novel approach to evaluating reconstructions for low-SNR magnetic resonanc...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Tisdall MD,Atkins MS

    更新日期:2006-11-01 00:00:00

  • Automatic construction of parts+geometry models for initializing groupwise registration.

    abstract::Groupwise nonrigid image registration is a powerful tool to automatically establish correspondences across sets of images. Such correspondences are widely used for constructing statistical models of shape and appearance. As existing techniques usually treat registration as an optimization problem, a good initializatio...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Zhang P,Cootes TF

    更新日期:2012-02-01 00:00:00

  • Electrical impedance tomography of translationally uniform cylindrical objects with general cross-sectional boundaries.

    abstract::An algorithm is developed for electrical impedance tomography (EIT) of finite cylinders with general cross-sectional boundaries and translationally uniform conductivity distributions. The electrodes for data collection are assumed to be placed around a cross-sectional plane; therefore, the axial variation of the bound...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Ider YZ,Gencer NG,Atalar E,Tosun H

    更新日期:1990-01-01 00:00:00

  • Assessment of perfusion by dynamic contrast-enhanced imaging using a deconvolution approach based on regression and singular value decomposition.

    abstract::The assessment of tissue perfusion by dynamic contrast-enhanced (DCE) imaging involves a deconvolution process. For analysis of DCE imaging data, we implemented a regression approach to select appropriate regularization parameters for deconvolution using the standard and generalized singular value decomposition method...

    journal_title:IEEE transactions on medical imaging

    pub_type: 临床试验,杂志文章


    authors: Koh TS,Wu XY,Cheong LH,Lim CC

    更新日期:2004-12-01 00:00:00

  • Patch-Based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation.

    abstract::Glaucoma is a leading cause of irreversible blindness. Accurate segmentation of the optic disc (OD) and optic cup (OC) from fundus images is beneficial to glaucoma screening and diagnosis. Recently, convolutional neural networks demonstrate promising progress in the joint OD and OC segmentation. However, affected by t...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Wang S,Yu L,Yang X,Fu CW,Heng PA

    更新日期:2019-11-01 00:00:00

  • Multi-material decomposition using statistical image reconstruction for spectral CT.

    abstract::Spectral computed tomography (CT) provides information on material characterization and quantification because of its ability to separate different basis materials. Dual-energy (DE) CT provides two sets of measurements at two different source energies. In principle, two materials can be accurately decomposed from DECT...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Long Y,Fessler JA

    更新日期:2014-08-01 00:00:00

  • A Signal Acquisition Setup for Ultrashort Echo Time Imaging Operating in Parallel on Unmodified Clinical MRI Scanners Achieving an Acquisition Delay of [Formula: see text].

    abstract::Ultrashort echo time imaging on clinical systems is still limited by the rather long radio frequency switching times achievable with standard front end concepts. In this contribution, an independent parallel receive-only system is interfaced to an unmodified clinical MRI system, enabling imaging of species with ultras...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Eder M,Horneff A,Paul J,Storm A,Wunderlich A,Hell E,Ulrici J,Anders J,Rasche V

    更新日期:2020-01-01 00:00:00

  • A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images.

    abstract::Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ul...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Nanayakkara ND,Chiu B,Samani A,Spence JD,Samarabandu J,Fenster A

    更新日期:2008-10-01 00:00:00

  • Anatomical model matching with fuzzy implicit surfaces for segmentation of thoracic volume scans.

    abstract::Many segmentation methods for thoracic volume data require manual input in the form of a seed point, initial contour, volume of interest etc. The aim of the work presented here is to further automate this segmentation initialization step. In this paper an anatomical modeling and matching method is proposed to coarsely...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Lelieveldt BP,van der Geest RJ,Rezaee MR,Bosch JG,Reiber JH

    更新日期:1999-03-01 00:00:00

  • Automated segmentation of multiple sclerosis lesions by model outlier detection.

    abstract::This paper presents a fully automated algorithm for segmentation of multiple sclerosis (MS) lesions from multispectral magnetic resonance (MR) images. The method performs intensity-based tissue classification using a stochastic model for normal brain images and simultaneously detects MS lesions as outliers that are no...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Van Leemput K,Maes F,Vandermeulen D,Colchester A,Suetens P

    更新日期:2001-08-01 00:00:00

  • Evaluation of Fisher Information Matrix-Based Methods for Fast Assessment of Image Quality in Pinhole SPECT.

    abstract::The accurate determination of the local impulse response and the covariance in voxels from penalized maximum likelihood reconstructed images requires performing reconstructions from many noise realizations of the projection data. As this is usually a very time-consuming process, efficient analytical approximations bas...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Pato LR,Vandenberghe S,Vandeghinste B,Van Holen R

    更新日期:2015-09-01 00:00:00

  • A geometric method for automatic extraction of sulcal fundi.

    abstract::Sulcal fundi are 3-D curves that lie in the depths of the cerebral cortex and, in addition to their intrinsic value in brain research, are often used as landmarks for downstream computations in brain imaging. In this paper, we present a geometric algorithm that automatically extracts the sulcal fundi from magnetic res...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Kao CY,Hofer M,Sapiro G,Stem J,Rehm K,Rottenberg DA

    更新日期:2007-04-01 00:00:00

  • Sufficient statistics as a generalization of binning in spectral X-ray imaging.

    abstract::It is well known that the energy dependence of X-ray attenuation can be used to characterize materials. Yet, even with energy discriminating photon counting X-ray detectors, it is still unclear how to best form energy dependent measurements for spectral imaging. Common ideas include binning photon counts based on thei...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Wang AS,Pelc NJ

    更新日期:2011-01-01 00:00:00

  • Development of a Mechanical Scanning Device With High-Frequency Ultrasound Transducer for Ultrasonic Capsule Endoscopy.

    abstract::Wireless capsule endoscopy has opened a new era by enabling remote diagnostic assessment of the gastrointestinal tract in a painless procedure. Video capsule endoscopy is currently commercially available worldwide. However, it is limited to visualization of superficial tissue. Ultrasound (US) imaging is a complementar...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Wang X,Seetohul V,Chen R,Zhang Z,Qian M,Shi Z,Yang G,Mu P,Wang C,Huang Z,Zhou Q,Zheng H,Cochran S,Qiu W

    更新日期:2017-09-01 00:00:00

  • MRI Meets MPI: a bimodal MPI-MRI tomograph.

    abstract::While magnetic particle imaging (MPI) constitutes a novel biomedical imaging technique for tracking superparamagnetic nanoparticles in vivo, unlike magnetic resonance imaging (MRI), it cannot provide anatomical background information. Until now these two modalities have been performed in separate scanners and image co...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Vogel P,Lother S,Rückert MA,Kullmann WH,Jakob PM,Fidler F,Behr VC

    更新日期:2014-10-01 00:00:00

  • Simultaneous Morphological and Flow Imaging Enabled by Megahertz Intravascular Doppler Optical Coherence Tomography.

    abstract::We demonstrate three-dimensional intravascular flow imaging compatible with routine clinical image acquisition workflow by means of megahertz (MHz) intravascular Doppler Optical Coherence Tomography (OCT). The OCT system relies on a 1.1 mm diameter motorized imaging catheter and a 1.5 MHz Fourier Domain Mode Locked (F...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Wang T,Pfeiffer T,Daemen J,Mastik F,Wieser W,van der Steen AFW,Huber R,van Soest G

    更新日期:2020-05-01 00:00:00

  • Noninvasive mapping of transmural potentials during activation in swine hearts from body surface electrocardiograms.

    abstract::The three-dimensional cardiac electrical imaging (3DCEI) technique was previously developed to estimate the initiation site(s) of cardiac activation and activation sequence from the noninvasively measured body surface potential maps (BSPMs). The aim of this study was to develop and evaluate the capability of 3DCEI in ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Liu C,Eggen MD,Swingen CM,Iaizzo PA,He B

    更新日期:2012-09-01 00:00:00

  • Development and validation of a Monte Carlo simulation of photon transport in an Anger camera.

    abstract::The geometric component of the point spread function (PSF) of a gamma camera collimator can be determined analytically, and the penetration component can be calculated readily by numerical ray-tracing. A Monte Carlo simulation of photon transport which includes collimator scatter is developed. The simulation was imple...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: De Vries DJ,Moore SC,Zimmerman RE,Mueller SP,Friedland B,Lanza RC

    更新日期:1990-01-01 00:00:00

  • Accuracy of in vivo neuroreceptor quantification by PET and review of steady-state, transient, double injection, and equilibrium models.

    abstract::The accuracy of in vivo dopamine D2 receptor quantification by positron emission tomography (PET) was determined for several models by means of singular-value decomposition, and some of the model assumptions were reviewed. These include steady-state, transient, double injection, and equilibrium approaches. All four mo...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Zeeberg BR,Gibson RE,Reba RC

    更新日期:1988-01-01 00:00:00

  • Segmentation of prostate boundaries from ultrasound images using statistical shape model.

    abstract::This paper presents a statistical shape model for the automatic prostate segmentation in transrectal ultrasound images. A Gabor filter bank is first used to characterize the prostate boundaries in ultrasound images in both multiple scales and multiple orientations. The Gabor features are further reconstructed to be in...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Shen D,Zhan Y,Davatzikos C

    更新日期:2003-04-01 00:00:00

  • A geometry-driven optical flow warping for spatial normalization of cortical surfaces.

    abstract::Spatial normalization is frequently used to map data to a standard coordinate system by removing intersubject morphological differences, thereby allowing for group analysis to be carried out. The work presented in this paper is motivated by the need for an automated cortical surface normalization technique that will a...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Tosun D,Prince JL

    更新日期:2008-12-01 00:00:00

  • Optimal rebinning of time-of-flight PET data.

    abstract::Time-of-flight (TOF) positron emission tomography (PET) scanners offer the potential for significantly improved signal-to-noise ratio (SNR) and lesion detectability in clinical PET. However, fully 3D TOF PET image reconstruction is a challenging task due to the huge data size. One solution to this problem is to rebin ...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Ahn S,Cho S,Li Q,Lin Y,Leahy RM

    更新日期:2011-10-01 00:00:00

  • Improvement of Chest Region CT Images through Automated Gray-Level Remapping.

    abstract::A software system, CTIP, written in Fortran is discussed which remaps the computerized tomography (CT) image gray level so that both the lung and heart regions are clearly visible with a "natural anatomical appearance." The system is adaptive to image statistics derived from the gray-level histogram of the entire imag...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Davis G,Wallenslager ST

    更新日期:1986-01-01 00:00:00

  • Spatiotemporal forward solution of the EEG and MEG using network modeling.

    abstract::Dynamic systems have proven to be well suited to describe a broad spectrum of human coordination behavior such synchronization with auditory stimuli. Simultaneous measurements of the spatiotemporal dynamics of electroencephalographic (EEG) and magnetoencephalographic (MEG) data reveals that the dynamics of the brain s...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Jirsa VK,Jantzen KJ,Fuchs A,Kelso JA

    更新日期:2002-05-01 00:00:00

  • Deep Learning for Fast and Spatially Constrained Tissue Quantification From Highly Accelerated Data in Magnetic Resonance Fingerprinting.

    abstract::Magnetic resonance fingerprinting (MRF) is a quantitative imaging technique that can simultaneously measure multiple important tissue properties of human body. Although MRF has demonstrated improved scan efficiency as compared to conventional techniques, further acceleration is still desired for translation into routi...

    journal_title:IEEE transactions on medical imaging

    pub_type: 杂志文章


    authors: Fang Z,Chen Y,Liu M,Xiang L,Zhang Q,Wang Q,Lin W,Shen D

    更新日期:2019-10-01 00:00:00