当前位置： SCI文献检索 > International Journal of Computer Assisted Radiology and Surgery期刊下所有文献 > Language-based translation and prediction of surgical navigation steps for endoscopic wayfinding assistance in minimally invasive surgery.

Language-based translation and prediction of surgical navigation steps for endoscopic wayfinding assistance in minimally invasive surgery.

Abstract：

PURPOSE:In the context of aviation and automotive navigation technology, assistance functions are associated with predictive planning and wayfinding tasks. In endoscopic minimally invasive surgery, however, assistance so far relies primarily on image-based localization and classification. We show that navigation workflows can be described and used for the prediction of navigation steps. METHODS:A natural description vocabulary for observable anatomical landmarks in endoscopic images was defined to create 3850 navigation workflow sentences from 22 annotated functional endoscopic sinus surgery (FESS) recordings. Resulting FESS navigation workflows showed an imbalanced data distribution with over-represented landmarks in the ethmoidal sinus. A transformer model was trained to predict navigation sentences in sequence-to-sequence tasks. The training was performed with the Adam optimizer and label smoothing in a leave-one-out cross-validation study. The sentences were generated using an adapted beam search algorithm with exponential decay beam rescoring. The transformer model was compared to a standard encoder-decoder-model, as well as HMM and LSTM baseline models. RESULTS:The transformer model reached the highest prediction accuracy for navigation steps at 0.53, followed by 0.35 of the LSTM and 0.32 for the standard encoder-decoder-network. With an accuracy of sentence generation of 0.83, the prediction of navigation steps at sentence-level benefits from the additional semantic information. While standard class representation predictions suffer from an imbalanced data distribution, the attention mechanism also considered underrepresented classes reasonably well. CONCLUSION:We implemented a natural language-based prediction method for sentence-level navigation steps in endoscopic surgery. The sentence-level prediction method showed a potential that word relations to navigation tasks can be learned and used for predicting future steps. Further studies are needed to investigate the functionality of path prediction. The prediction approach is a first step in the field of visuo-linguistic navigation assistance for endoscopic minimally invasive surgery.

journal_name

Int J Comput Assist Radiol Surg

journal_title

International journal of computer assisted radiology and surgery

authors

Bieck R,Heuermann K,Pirlich M,Neumann J,Neumuth T

doi

10.1007/s11548-020-02264-2

subject

Has Abstract

pub_date

2020-12-01 00:00:00

pages

2089-2100

issue

eissn

1861-6410

issn

1861-6429

pii

10.1007/s11548-020-02264-2

journal_volume

pub_type

杂志文章

在线工具

Language-based translation and prediction of surgical navigation steps for endoscopic wayfinding assistance in minimally invasive surgery.