当前位置： SCI文献检索 > International Journal of Computer Assisted Radiology and Surgery期刊下所有文献 > Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms.

Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms.

Abstract：

PURPOSE:Context-aware systems for the operating room (OR) provide the possibility to significantly improve surgical workflow through various applications such as efficient OR scheduling, context-sensitive user interfaces, and automatic transcription of medical procedures. Being an essential element of such a system, surgical action recognition is thus an important research area. In this paper, we tackle the problem of classifying surgical actions from video clips that capture the activities taking place in the OR. METHODS:We acquire recordings using a multi-view RGBD camera system mounted on the ceiling of a hybrid OR dedicated to X-ray-based procedures and annotate clips of the recordings with the corresponding actions. To recognize the surgical actions from the video clips, we use a classification pipeline based on the bag-of-words (BoW) approach. We propose a novel feature encoding method that extends the classical BoW approach. Instead of using the typical rigid grid layout to divide the space of the feature locations, we propose to learn the layout from the actual 4D spatio-temporal locations of the visual features. This results in a data-driven and non-rigid layout which retains more spatio-temporal information compared to the rigid counterpart. RESULTS:We classify multi-view video clips from a new dataset generated from 11-day recordings of real operations. This dataset is composed of 1734 video clips of 15 actions. These include generic actions (e.g., moving patient to the OR bed) and actions specific to the vertebroplasty procedure (e.g., hammering). The experiments show that the proposed non-rigid feature encoding method performs better than the rigid encoding one. The classifier's accuracy is increased by over 4 %, from 81.08 to 85.53 %. CONCLUSION:The combination of both intensity and depth information from the RGBD data provides more discriminative power in carrying out the surgical action recognition task as compared to using either one of them alone. Furthermore, the proposed non-rigid spatio-temporal feature encoding scheme provides more discriminative histogram representations than the rigid counterpart. To the best of our knowledge, this is also the first work that presents action recognition results on multi-view RGBD data recorded in the OR.

journal_name

Int J Comput Assist Radiol Surg

journal_title

International journal of computer assisted radiology and surgery

authors

Twinanda AP,Alkan EO,Gangi A,de Mathelin M,Padoy N

doi

10.1007/s11548-015-1186-1

subject

Has Abstract

pub_date

2015-06-01 00:00:00

pages

737-47

issue

eissn

1861-6410

issn

1861-6429

journal_volume

pub_type

杂志文章

在线工具

Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms.