How AI could expand uses for “mixed reality” technology

"At first blush, the idea of “mixed reality” may not be a comforting notion for patients facing brain surgery.

But the National University Health System in Singapore (NUHS) has started a research and development program to explore the use of mixed reality (MR) technology for, among other things, enabling surgeon to identify brain tumors “quickly and precisely,” while also helping them determine the exact location and the specific angle at which they should make an incision.

In a proof of concept demonstration, a holographic visor was used to create a 3D hologram of a patient's brain scan that was projected into space and superimposed onto a patient's head during surgery. The hologram was generated using a 3D medical software called Virtual Surgery Intelligence by German medical MR platform developer apoQl ar.

"Holographic technology may radically transform the way we practice medicine,” explained NUHS Group CTO Ngiam Kee Yuan, who oversees the hospital group's Holomedicine Programme R&D. "The advancements in technology, especially in the field of MR, has opened new horizons to the medical profession. By merging other technologies such as artificial intelligence, real-time image recognition, and predictive modelling, these MR devices offer clinicians capabilities that were once thought impossible.”

While holomedicine is new in the field of medicine, it has become increasingly prominent in the past year. "It leverages the concept of MR not only to augment our physical environment, but permits interaction with virtual objects superimposed onto the real world. The virtual objects can also be manipulated relative to the real world using natural hand gestures," explained Dr Gao Yujia, Associate Consultant with the Division of Hepatobiliary & Pancreatic Surgery, National University Hospital (NUH), and the programme lead for holomedicine at NUHS.

The NUHS has outlined a holomedicine roadmap that includes near and long-term research projects, integration with existing hospital systems, procedures to onboard users and enhancing the hospital's infrastructure to support the system. It hopes to implement holographic technology in all its hospitals and healthcare institutions "in the near future".

Clinical Associate Professor Yeo Tseng Tsai, Head and Senior Consultant with the Division of Neurosurgery, NUH said, "With this holographic technology, you are able to see inside the brain. You will be able to see the blood vessels and most importantly identify the tumour quickly and precisely, as well as to know which angle and the exact location to make the incision. For over thirty years now, we have been using a handheld navigation system to navigate and identify the location of the tumour. In comparison, this new mixed reality system is more intuitive as we can now see inside the patient's head without the need to look up and refer to a computer screen while performing a procedure.”..." Lire la suite