Surgeon And Researcher Innovate With Mixed Reality And AI For Safer Surgeries

The Oklahoma changemakers have devised a system for using artificial intelligence to visualize superimposed and anatomically aligned 3D CT scan data during surgery.

A University of Oklahoma researcher and a surgeon at OU Health, based in Oklahoma City, had a vision of using AI to visualize superimposed and anatomically aligned 3D CT scan data during surgery. The mission was to augment every surgery.

THE PROBLEM

"Compared to a pilot flying a plane or even a regular Google Maps user on his way to work, surgeons today have their instruments clustered behind them hanging on the wall," said Mohammad Abdul Mukit, an MS student in electrical and computer engineering at the University of Oklahoma, and a graduate fellow and research assistant. His research focuses on applications of computer vision, extended reality and AI in medical surgeries.

"The Google Maps user or the pilot gets constant, real-time updates regarding where they are, what to do next, and other vital data that helps them make split-second decisions," he explained. "They don't have to plan the trip for days or memorize every turn and detail of every landmark along the way. They just do it."

On the other hand, surgeons today have to do rigorous surgical planning, memorize the specifics of each unique case, and know all the necessary steps to ensure the safest possible surgery. Then they engage in complex procedures for several hours, with no targeting or homing devices or head-mounted displays to assist them.

"They have to feel their way to their objective and hope everything goes as they planned," Mukit said. "Through our research, we aim to change this process forever. We are making the 'Google Maps for surgery.'"