By visualizing 2D medical images in 3D, Virtual Reality (VR)/Augmented Reality (AR) software platforms are helping surgeons operate better.

It was finally a happy ending for Erica Sandoval and Eva Sandoval, 2-year-old conjoined twin sisters, after a 17-hour long groundbreaking surgery performed by a 50-person team of surgeons, physicians, radiologists and caregivers at Stanford University Medical Center December last year. They were successfully separated and are stable.

The girls were joined from the lower chest and upper-abdominal level down, positioned facing each other. Together they had one liver, one bladder, two kidneys and a third leg with seven toes that is not controlled by either. They had separate hearts and lungs but diaphragm muscle and some other anatomical features were shared.

Assisting the doctors in this complex and complicated surgery is a radical virtual reality/augmented reality software platform called True 3D. Built by a privately held, California-based company EchoPixel, True 3D system uses various existing medical image datasets to combine and create virtual reality environments of patient-specific anatomy to enable radiologists, cardiologists, pediatricians and other surgeons to reach early diagnosis and plan complex surgeries.

True 3D isn’t the only platform available to doctors in creating holographic medical imaging. RealView Imaging’s Holoscope is a pioneer in this area. Royal Philips, GE, Integraf, Zebra Imaging, EON Reality, Holoxcia, Nano Live SA, Lincee Tec and Mach7 Technologies are some of the important players in medical holography, an industry projected to touch US$ 5 billion by 2023, with a CAGR of 36.6%.

In recent years, advances in Virtual Reality/Augmented Reality have opened up new possibilities in how the data generated by various diagnostic tools is read and visualized. Even though CTs and MRIs capture data in 3D, they are almost entirely seen in 2D formats, seriously limiting the ability of physicians to leverage the clinically significant information. By generating true-to-size, interactive holographic projections within seconds, VR platforms can –

  • Provide anatomical models that can be used as reference during surgery,
  • Enhance communication and collaboration between surgeons and other OR staff in the surgical teams,
  • Help clinicians to provide better personalized care by improving patient selection for implantable devices,
  • Help in efficiently creating 3D models of human body parts such as bone for better pre-operative planning,
  • Communicate better with patients and their families through better visualizations.

In the case of Erica and Eva, they have undergone many rounds of CT and MRI scans and these were fed into a VR software platform which then produced a 3D, virtual reality model of pelvic bones and blood vessels to help plan the separation. The doctors, wearing 3-D glasses, can view images that can be moved, rotated, dissected and manipulated to better understand complex anatomical features.

This ability to translate 2D medical images into holograms allows doctors to create step-by-step surgical procedure plans with instructions and visualizations. Such plans can further be stored, shared and refined to act as reference points during the surgical procedures. As AR/VR platforms mature, the patient-specific diagnostic data can be shared in real-time to perform remote surgeries.

With growing clinical applications of holography in healthcare, hospitals and medical research centers are increasingly adopting VR and medical holography systems. Doctors can now perform a virtual colonoscopy in place of unpopular and discomforting insertion of colonoscope into the body. Hearts can be entirely mapped in holograms to ascertain if a patient needs an open heart surgery or a minimally invasive surgery suffice.

The growing need to reduce healthcare costs and rising interest of both patients and physicians in the role of AR and VR in patient care management and patient education, increasing investments made in this frontier technology are the major growth factors pushing the medical holography market.

However, high costs of technology, lack of competence in deploying AR/VR solutions and lack of expertise among the medical fraternity in adopting these new technologies are a concern.

Useful VR/AR Posts

Leave a Reply