Skip to main content

What mixed reality’s amazing new health frontier means for you 

In 1992, scientist and technologist Louis Rosenberg created Virtual Fixtures, one of the first augmented reality systems ever developed. By using a full upper-body exoskeleton, the wearer was able to control two physical robots, while innovative optics aligned the robot arms as an overlay of the user’s own arms.

In the nearly 30 years that followed, technology has advanced considerably, enabling fully immersive “virtual-reality” experiences, as well as “mixed reality”—the result of blending the physical world with the digital world.

While mixed reality is still a relatively new technology in health, it has the potential to make a significant impact on patient care. Its unique ability to project visualizations into physical space and its low barrier of entry is spurring health organizations to experiment in ways that are incredibly promising. Here are three examples of providers that are pioneering mixed reality in medical education, the perioperative pathway, and virtual care.

Enhancing medical education by helping students see the human body in three dimensions

While medical students have traditionally learned through textbooks and hands-on training, this approach has its disadvantages, such as a lack of real-world exposure to multiple anatomical variations. For practicing physicians, on-the-job training is often conducted via mannequins and simulators, which are an improvement over textbooks, but even these sophisticated tools have limitations. Each patient is different, and while mannequins are helpful, nothing beats actual patient care for learning.

Mixed reality enhances physician education by combining the anatomical and procedural to create a more robust education platform. Case Western Reserve University in Cleveland is utilizing mixed-reality devices to accelerate their medical students’ grasp of anatomy. With mixed reality, students can visualize the muscles on top of a skeleton and understand the body’s different layers. They can project any piece of anatomy digitally and examine it in three dimensions, move it around, or make it translucent to see through to what lies underneath.

As a result, students have more freedom to experiment and explore the ways anatomical systems work together, helping them build confidence and empowering them with stronger working knowledge of anatomy.

Expanding the opportunity to learn in a new dimension changes the way medical students and physicians see their patients and the world, opening new avenues to approach medicine from a “hands-on” perspective, not just a theoretical one.

Delivering new intelligence to the perioperative pathway by simulating the physical world

When approaching complicated surgeries, each scenario is unique. Some procedures are more complex than others due to the unique characteristics of the patient’s anatomy and the type of surrounding tissue and organs. A growing number of surgeons have already adopted innovative methods like three-dimensional (3D) printing to prepare for the intricacies of each surgery, but this approach is challenging to scale and hinders collaboration among surgical teams.

Mixed reality takes this innovation further, enabling surgeons to interact with an accurate digital representation of a patient’s unique organ structure, as well as collaborate with their teams to orchestrate and rehearse procedures. The University of Oslo is leveraging mixed reality to plan complex procedures, such as liver surgery. By creating a digital 3D model of the patient’s liver from a computed tomography (CT) scan, surgeons can move, scale, rotate and isolate different parts of the organ, as well as switch layers of the model off and on with simple hand gestures. Multiple surgeons can also share the same experience through separate devices.

The new technique enables doctors to navigate around the patient’s other organs and leave more of their healthy liver tissue undisturbed, improving their ability to withstand surgery during treatment. Researchers are looking for ways to apply this technology to patients undergoing other complicated procedures such as cardiac surgery.

Applying mixed reality to perioperative planning enables a customized approach for each patient, offering surgeons enhanced visibility into the patient’s unique anatomy. This approach can improve each individual surgery, helping surgical teams prepare more effectively to create the best possible outcome for each patient.

Bringing doctors and patients together virtually when they can’t be in the same place

Within certain patient populations, connecting patients with providers can be time-consuming and costly. For instance, getting to a provider’s office is a major challenge for elderly patients who lack transportation. Those in remote areas often face similar challenges in making the time to travel and finding transportation.

Through mixed-reality devices, doctors can provide high-quality care to patients in their own homes, saving time, money, and hassle while making them feel more comfortable. Silver Chain Group in Australia is doing this already by using Microsoft’s HoloLens to create an “Enhanced Medical Mixed Reality (EMMR) interface. By having a visiting nurse wear a mixed-reality headset at the patient’s bedside, a doctor can see the patient remotely as if they were in the same room. This extends doctors’ reach beyond the walls of the exam room and enables patients to receive care that would otherwise be difficult to obtain. As the world’s population continues to grow older, demand for community-based care will only increase, and innovative telehealth options will be even more critical.

Augmenting reality eliminates logistical challenges to providing care to remote and elderly populations, putting the patient and the problem in front of doctors’ eyes and removing the friction that can get in the way of patients receiving the right care at the right time.

Enable healthcare innovation with mixed reality

For years, technology adoption in medicine has been driven by doctors seeking ways to improve their techniques as well as by patients demanding high-quality care and convenience. Mixed reality is showing great promise in improving medical education, surgical planning, and even office visits, making healthcare delivery easier, faster, and cheaper while driving better results for patients. To learn more about what Microsoft and its partners are achieving with mixed reality, download our eBook on digital transformation in health.