Touchless Interaction in Surgery

Tom Carrell, Helena Mentis, Graeme Penney, Antonio Criminisi, Kenton O'Hara, Abigail Sellen, Mark Rouncefield, Neville Dastur, Varnavas, Bob Corish

Communications of the ACM |

With advances in medical imaging technologies in recent decades, we have seen their widespread adoption in the context of surgical procedures.  While surgeons are increasingly reliant on these technologies, their ability to interact with them during surgery is restricted by traditional touch-based input mechanisms due to the need to maintain sterility.  In response to the need to provide surgeons with control over medical images while maintaining sterility we are seeing a number of research initiatives exploring ways of interacting with these imaging technologies without touching, in particular through the use of gesture and voice control.  Given the growing interest in the area, it is an opportune time to take a reflective look at the corpus of initiatives to highlight key lessons learned as well as some of the issues and challenges relevant to the development of these systems.  As well as the key technical challenges to be faced, we also highlight how key socio-technical concerns play an important role in the ways we approach the design of these systems and illustrate this through some of our own development experiences in this area.  In light of discussion we offer some directions for the future progress of the field.

Touchless Interaction in Surgery

In the surgical theatre, vascular surgeons are reliant on live fluoroscopy images to guide them in inserting stents in the aorta and its major branches, yet currently, because they are scrubbed up, they rely on other people to manipulate the data. Kinect technology presents some real opportunities to allow for touchless interaction. By working with surgeons at St. Thomas’ hospital, researchers in the Imaging Sciences and Biomedical Engineering department at King’s College London, and the School of Computing & Communications at Lancaster University, we have developed a system for manipulating 3D vascular renderings. This demonstration is just the start of the system we are developing with guidance from the surgeons.