Multimodal mixed reality visualisation for intraoperative surgical guidance.

Abstract:

PURPOSE:In the last decade, there has been a great effort to bring mixed reality (MR) into the operating room to assist surgeons intraoperatively. However, progress towards this goal is still at an early stage. The aim of this paper is to propose a MR visualisation platform which projects multiple imaging modalities to assist intraoperative surgical guidance. METHODOLOGY:In this work, a MR visualisation platform has been developed for the Microsoft HoloLens. The platform contains three visualisation components, namely a 3D organ model, volumetric data, and tissue morphology captured with intraoperative imaging modalities. Furthermore, a set of novel interactive functionalities have been designed including scrolling through volumetric data and adjustment of the virtual objects' transparency. A pilot user study has been conducted to evaluate the usability of the proposed platform in the operating room. The participants were allowed to interact with the visualisation components and test the different functionalities. Each surgeon answered a questionnaire on the usability of the platform and provided their feedback and suggestions. RESULTS:The analysis of the surgeons' scores showed that the 3D model is the most popular MR visualisation component and neurosurgery is the most relevant speciality for this platform. The majority of the surgeons found the proposed visualisation platform intuitive and would use it in their operating rooms for intraoperative surgical guidance. Our platform has several promising potential clinical applications, including vascular neurosurgery. CONCLUSION:The presented pilot study verified the potential of the proposed visualisation platform and its usability in the operating room. Our future work will focus on enhancing the platform by incorporating the surgeons' suggestions and conducting extensive evaluation on a large group of surgeons.

authors

Cartucho J,Shapira D,Ashrafian H,Giannarou S

doi

10.1007/s11548-020-02165-4

subject

Has Abstract

pub_date

2020-05-01 00:00:00

pages

819-826

issue

5

eissn

1861-6410

issn

1861-6429

pii

10.1007/s11548-020-02165-4

journal_volume

15

pub_type

杂志文章
  • Predicting surgical skill from the first N seconds of a task: value over task time using the isogony principle.

    abstract:PURPOSE:Most evaluations of surgical workflow or surgeon skill use simple, descriptive statistics (e.g., time) across whole procedures, thereby deemphasizing critical steps and potentially obscuring critical inefficiencies or skill deficiencies. In this work, we examine off-line, temporal clustering methods that chunk ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-017-1606-5

    authors: French A,Lendvay TS,Sweet RM,Kowalewski TM

    更新日期:2017-07-01 00:00:00

  • Feature detection in biological tissues using multi-band and narrow-band imaging.

    abstract:PURPOSE:In the past decade, augmented reality systems have been expected to support surgical operations by making it possible to view invisible objects that are inside or occluded by the skull, hands, or organs. However, the properties of biological tissues that are non-rigid and featureless require a large number of d...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-016-1458-4

    authors: Tamura Y,Mashita T,Kuroda Y,Kiyokawa K,Takemura H

    更新日期:2016-12-01 00:00:00

  • Four-dimensional fully convolutional residual network-based liver segmentation in Gd-EOB-DTPA-enhanced MRI.

    abstract:PURPOSE:Gadolinium-ethoxybenzyl-diethylenetriamine pentaacetic acid (Gd-EOB-DTPA)-enhanced magnetic resonance imaging (MRI) tends to show higher diagnostic accuracy than other modalities. There is a demand for computer-assisted detection (CAD) software for Gd-EOB-DTPA-enhanced MRI. Segmentation with high accuracy is im...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-019-01935-z

    authors: Takenaga T,Hanaoka S,Nomura Y,Nemoto M,Murata M,Nakao T,Miki S,Yoshikawa T,Hayashi N,Abe O

    更新日期:2019-08-01 00:00:00

  • Automatic detection of perforator vessels using infrared thermography in reconstructive surgery.

    abstract:PURPOSE:Knowing the location of the blood vessels supplying the skin and subcutaneous tissue is a requirement during the planning of tissue transfer in reconstructive surgery. Commonly used imaging techniques such as computed tomography angiography and indocyanine green angiography expose the patient to radiation or a ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1892-6

    authors: Unger M,Markfort M,Halama D,Chalopin C

    更新日期:2019-03-01 00:00:00

  • Real-time marker-free patient registration for electromagnetic navigated bronchoscopy: a phantom study.

    abstract:PURPOSE:To perform bronchoscopy safely and smoothly, it is very important to develop a bronchoscopic guidance system. Transbronchial lung biopsy (TBLB) with a bronchoscopic guidance system especially should permit safe image-guided procedures. Recently, electromagnetic tracking (EMT) is utilized to track the tip of the...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-011-0626-9

    authors: Deguchi D,Feuerstein M,Kitasaka T,Suenaga Y,Ide I,Murase H,Imaizumi K,Hasegawa Y,Mori K

    更新日期:2012-05-01 00:00:00

  • Altered brain anatomical networks and disturbed connection density in brain tumor patients revealed by diffusion tensor tractography.

    abstract:PURPOSE:Brain tumor patients are usually accompanied by impairments in cognitive functions, and these dysfunctions arise from the altered diffusion tensor of water molecules and disrupted neuronal conduction in white matter. Diffusion tensor imaging (DTI) is a powerful noninvasive imaging technique that can reflect dif...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-015-1330-y

    authors: Yu Z,Tao L,Qian Z,Wu J,Liu H,Yu Y,Song J,Wang S,Sun J

    更新日期:2016-11-01 00:00:00

  • FCNN-based axon segmentation for convection-enhanced delivery optimization.

    abstract:PURPOSE:Glioblastoma multiforme treatment is a challenging task in clinical oncology. Convection- enhanced delivery (CED) is showing encouraging but still suboptimal results due to drug leakages. Numerical models can predict drug distribution within the brain, but require retrieving brain physical properties, such as t...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-01911-z

    authors: Vidotto M,De Momi E,Gazzara M,Mattos LS,Ferrigno G,Moccia S

    更新日期:2019-03-01 00:00:00

  • Path generator control system and virtual compliance calculator for maxillofacial massage robots.

    abstract:PURPOSE:Oral disorders such as temporomandibular joint disorders (TMD) and dry mouth are common and often require treatment. Maxillofacial massage is used as a complementary and alternative therapy for these disorders. We developed an oral rehabilitation robot that massages the maxillofacial tissues for this purpose. I...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-009-0383-1

    authors: Ishii H,Koga H,Obokawa Y,Solis J,Takanishi A,Katsumata A

    更新日期:2010-01-01 00:00:00

  • Optimal setting of image bounding box can improve registration accuracy of diffusion tensor tractography.

    abstract:PURPOSE:When we register diffusion tensor tractography (DTT) to anatomical images such as fast imaging employing steady-state acquisition (FIESTA), we register the B0 image to FIESTA. Precise registration of the DTT B0 image to FIESTA is possible with non-rigid registration compared to rigid registration, although the ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-013-0934-3

    authors: Yoshino M,Kin T,Saito T,Nakagawa D,Nakatomi H,Kunimatsu A,Oyama H,Saito N

    更新日期:2013-08-20 00:00:00

  • Incorporating prior shape knowledge via data-driven loss model to improve 3D liver segmentation in deep CNNs.

    abstract:PURPOSE:Convolutional neural networks (CNNs) have obtained enormous success in liver segmentation. However, there are several challenges, including low-contrast images, and large variations in the shape, and appearance of the liver. Incorporating prior knowledge in deep CNN models improves their performance and general...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-019-02085-y

    authors: Mohagheghi S,Foruzan AH

    更新日期:2020-02-01 00:00:00

  • Cost-effective, personalized, 3D-printed liver model for preoperative planning before laparoscopic liver hemihepatectomy for colorectal cancer metastases.

    abstract:PURPOSE:Three-dimensional (3D) printing for preoperative planning has been intensively developed in the recent years. However, the implementation of these solutions in hospitals is still difficult due to high costs, extremely expensive industrial-grade printers, and software that is difficult to obtain and learn along ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-017-1527-3

    authors: Witowski JS,Pędziwiatr M,Major P,Budzyński A

    更新日期:2017-12-01 00:00:00

  • Multi-input deep learning architecture for predicting breast tumor response to chemotherapy using quantitative MR images.

    abstract:PURPOSE:Neoadjuvant chemotherapy (NAC) aims to minimize the tumor size before surgery. Predicting response to NAC could reduce toxicity and delays to effective intervention. Computational analysis of dynamic contrast-enhanced magnetic resonance images (DCE-MRI) through deep convolution neural network (CNN) has shown a ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-020-02209-9

    authors: El Adoui M,Drisis S,Benjelloun M

    更新日期:2020-09-01 00:00:00

  • Decision support system for the planning of minimally invasive aortic valve replacement surgery.

    abstract:PURPOSE:Minimally invasive aortic valve replacement (MIAVR) procedures remain more complex and technically challenging compared to conventional full sternotomy surgery. This technique involves a restricted surgical field and a limited workspace, which is, at present, strictly reserved for the most experienced surgeons....

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1725-7

    authors: Li H,Castro M,Haigron P,Verhoye JP,Ruggieri VG

    更新日期:2018-08-01 00:00:00

  • Temporal enhancement of 2D color Doppler echocardiography sequences by fragment-based frame reordering and refinement.

    abstract:PURPOSE:The goal of this study was to develop an algorithm that enhances the temporal resolution of two-dimensional color Doppler echocardiography (2D CDE) by reordering all the acquired frames and filtering out the frames corrupted by out-of-plane motion and arrhythmia. METHODS:The algorithm splits original frame seq...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-019-01926-0

    authors: Terentjev AB,Perrin DP,Settlemier SH,Zurakowski D,Smirnov PO,Del Nido PJ,Shturts IV,Vasilyev NV

    更新日期:2019-04-01 00:00:00

  • Preliminary study on magnetic tracking-based planar shape sensing and navigation for flexible surgical robots in transoral surgery: methods and phantom experiments.

    abstract:PURPOSE:Flexible surgical robot can work in confined and complex environments, which makes it a good option for minimally invasive surgery. In order to utilize flexible manipulators in complicated and constrained surgical environments, it is of great significance to monitor the position and shape of the curvilinear man...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-017-1672-8

    authors: Song S,Zhang C,Liu L,Meng MQ

    更新日期:2018-02-01 00:00:00

  • Interactive navigation of segmented MR angiograms using simultaneous curved planar and volume visualizations.

    abstract:PURPOSE:Interactive visualization is required to inspect and monitor the automatic segmentation of vessels derived from contrast-enhanced magnetic resonance angiography (CE-MRA). A dual-view visualization scheme consisting of curved planar reformation (CPR) and direct volume rendering (DVR) was developed for this purpo...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-010-0534-4

    authors: van Schooten BW,van Dijk EM,Suinesiaputra A,Reiber JH

    更新日期:2011-09-01 00:00:00

  • Ultrasound texture-based CAD system for detecting neuromuscular diseases.

    abstract:PURPOSE:Diagnosis of neuromuscular diseases in ultrasonography is a challenging task since experts are often unable to discriminate between healthy and pathological cases. A computer-aided diagnosis (CAD) system for skeletal muscle ultrasonography was developed and tested for myositis detection in ultrasound images of ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-014-1133-6

    authors: König T,Steffen J,Rak M,Neumann G,von Rohden L,Tönnies KD

    更新日期:2015-09-01 00:00:00

  • Continuous roadmapping in liver TACE procedures using 2D-3D catheter-based registration.

    abstract:PURPOSE:Fusion of pre/perioperative images and intra-operative images may add relevant information during image-guided procedures. In abdominal procedures, respiratory motion changes the position of organs, and thus accurate image guidance requires a continuous update of the spatial alignment of the (pre/perioperative)...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-015-1218-x

    authors: Ambrosini P,Ruijters D,Niessen WJ,Moelker A,van Walsum T

    更新日期:2015-09-01 00:00:00

  • Estimating needle tip deflection in biological tissue from a single transverse ultrasound image: application to brachytherapy.

    abstract:PURPOSE:This paper proposes a method to predict the deflection of a flexible needle inserted into soft tissue based on the observation of deflection at a single point along the needle shaft. METHODS:We model the needle-tissue as a discretized structure composed of several virtual, weightless, rigid links connected by ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-015-1329-4

    authors: Rossa C,Sloboda R,Usmani N,Tavakoli M

    更新日期:2016-07-01 00:00:00

  • Augmented reality patient-specific reconstruction plate design for pelvic and acetabular fracture surgery.

    abstract:PURPOSE:The objective of this work is to develop a preoperative reconstruction plate design system for unilateral pelvic and acetabular fracture reduction and internal fixation surgery, using computer graphics and augmented reality (AR) techniques, in order to respect the patient-specific morphology and to reduce surgi...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-012-0775-5

    authors: Shen F,Chen B,Guo Q,Qi Y,Shen Y

    更新日期:2013-03-01 00:00:00

  • Robust motion tracking in liver from 2D ultrasound images using supporters.

    abstract:PURPOSE:Effectiveness of image-guided radiation therapy with precise dose delivery depends highly on accurate target localization, which may involve motion during treatment due to, e.g., breathing and drift. Therefore, it is important to track the motion and adjust the radiation delivery accordingly. Tracking generally...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-017-1559-8

    authors: Ozkan E,Tanner C,Kastelic M,Mattausch O,Makhinya M,Goksel O

    更新日期:2017-06-01 00:00:00

  • Dual-robot ultrasound-guided needle placement: closing the planning-imaging-action loop.

    abstract:PURPOSE:Precise needle placement is an important task during several medical procedures. Ultrasound imaging is often used to guide the needle toward the target region in soft tissue. This task remains challenging due to the user's dependence on image quality, limited field of view, moving target, and moving needle. In ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-016-1408-1

    authors: Kojcev R,Fuerst B,Zettinig O,Fotouhi J,Lee SC,Frisch B,Taylor R,Sinibaldi E,Navab N

    更新日期:2016-06-01 00:00:00

  • Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks.

    abstract:PURPOSE:During needle interventions, successful automated detection of the needle immediately after insertion is necessary to allow the physician identify and correct any misalignment of the needle and the target at early stages, which reduces needle passes and improves health outcomes. METHODS:We present a novel appr...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1798-3

    authors: Pourtaherian A,Ghazvinian Zanjani F,Zinger S,Mihajlovic N,Ng GC,Korsten HHM,de With PHN

    更新日期:2018-09-01 00:00:00

  • Designing and validating a PVA liver phantom with respiratory motion for needle-based interventions.

    abstract:PURPOSE:The purpose is to design and validate an anthropomorphic polyvinyl alcohol (PVA) liver phantom with respiratory motion to simulate needle-based interventions. Such a system can, for example, be used as a validation tool for novel needles. METHODS:Image segmentations of CT scans of four patients during inspirat...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-019-02029-6

    authors: de Jong TL,Moelker A,Dankelman J,van den Dobbelsteen JJ

    更新日期:2019-12-01 00:00:00

  • Automatic bone segmentation in whole-body CT images.

    abstract:PURPOSE:Many diagnostic or treatment planning applications critically depend on the successful localization of bony structures in CT images. Manual or semiautomatic bone segmentation is tedious, however, and often not practical in clinical routine. In this paper, we present a reliable and fully automatic bone segmentat...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1883-7

    authors: Klein A,Warszawski J,Hillengaß J,Maier-Hein KH

    更新日期:2019-01-01 00:00:00

  • Long-term experience with setup and implementation of an IHE-based image management and distribution system in intersectoral clinical routine.

    abstract:PURPOSE:Sharing of medical data is crucial for the proper treatment of patients as it could reduce the risk of duplicated medical tests and speed up the care process if all documents are readily available. Despite great technical progress, sharing patient data while maintaining full control over the process in an inter...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1819-2

    authors: Schwind F,Münch H,Schröter A,Brandner R,Kutscha U,Brandner A,Heinze O,Bergh B,Engelmann U

    更新日期:2018-11-01 00:00:00

  • Planning nonlinear access paths for temporal bone surgery.

    abstract:PURPOSE:Interventions at the otobasis operate in the narrow region of the temporal bone where several highly sensitive organs define obstacles with minimal clearance for surgical instruments. Nonlinear trajectories for potential minimally invasive interventions can provide larger distances to risk structures and optimi...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1712-z

    authors: Fauser J,Sakas G,Mukhopadhyay A

    更新日期:2018-05-01 00:00:00

  • Automated aorta segmentation in low-dose chest CT images.

    abstract:PURPOSE:Abnormalities of aortic surface and aortic diameter can be related to cardiovascular disease and aortic aneurysm. Computer-based aortic segmentation and measurement may aid physicians in related disease diagnosis. This paper presents a fully automated algorithm for aorta segmentation in low-dose non-contrast CT...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-013-0924-5

    authors: Xie Y,Padgett J,Biancardi AM,Reeves AP

    更新日期:2014-03-01 00:00:00

  • Patient-specific catheter shaping for the minimally invasive closure of the left atrial appendage.

    abstract:PURPOSE:The minimally invasive closure of the left atrial appendage is a promising alternative to anticoagulation for stroke prevention in patients suffering from atrial fibrillation. One of the challenges of this procedure is the correct positioning and the coaxial alignment of the tip of the catheter sheath to the im...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-018-1752-4

    authors: Graf EC,Ott I,Praceus J,Bourier F,Lueth TC

    更新日期:2018-06-01 00:00:00

  • A practical marker-less image registration method for augmented reality oral and maxillofacial surgery.

    abstract:BACKGROUND:Image registration lies in the core of augmented reality (AR), which aligns the virtual scene with the reality. In AR surgical navigation, the performance of image registration is vital to the surgical outcome. METHODS:This paper presents a practical marker-less image registration method for AR-guided oral ...

    journal_title:International journal of computer assisted radiology and surgery

    pub_type: 杂志文章

    doi:10.1007/s11548-019-01921-5

    authors: Wang J,Shen Y,Yang S

    更新日期:2019-05-01 00:00:00