HoloLens transforms learning via MRIs and AR

via Case Western Reserve University

March 30, 2016 – Last spring, Professor Mark Griswold told a global audience about the potential of Microsoft HoloLens to transform learning.

Twelve months later, Griswold showed the world exactly how—or at least his hologram did.

Speaking at Microsoft’s annual Build conference for the second consecutive year, Griswold was joined by School of Medicine Dean Pamela Davis and two university developers, Jeff Mlakar and Henry Eastman—all three of whom were actually on stage in person at San Francisco’s Moscone Center.

The group’s task? To demonstrate the immense advantage that three-dimensional images can give to students learning anatomy.

Six months ago, Case Western Reserve and Cleveland Clinic broke ground on a 485,000-square-foot Health Education Campus, a space designed to support interprofessional learning and offer the most advanced technology available.

As part of creating what Davis called a “state-of-the-future” building, the new space will not include any of the traditional cadaver-filled laboratories that for decades have housed anatomy classes. Instead, just as Mlakar and Eastman did Wednesday, medical students will don HoloLens headsets to see the body’s organs and systems.

The pair began with a hologram of the digestive system, with labels on organs including the stomach, gall bladder and liver. Davis asked them about the pancreas, which, nestled behind the stomach, cannot be seen from the front of the body.

“Thankfully with HoloLens,” Mlakar said, making a small hand gesture that caused the holographic body to rotate 180 degrees to reveal the organ, “it’s really easy to get the best view of things.”

It’s also easy to engage with an instructor hundreds of miles away, as Professor Griswold demonstrated when a simple 3-D head and arm representing him entered the sold-out auditorium.

“This is our new system,” he said, “which allows me to teach and interact with you, even though I’m not there. … This is really changing what it means to be ‘in class.’”

Based on an actual patient MRI from Case Western Reserve Professor Cameron McIntyre’s lab, Griswold guided Eastman and Mlakar through an examination of the white-matter tracts of the brain, fibers that allows messages to travel from one area to another. Based on an actual patient’s MRI, the hologram color-coded the fibers by direction, and included a bright red mass that Griswold identified as a tumor.

“Hey Jeff and Henry,” Griswold’s hologram called out as his “face” drew closer to the brain and his “finger” pointed to the red orb and other areas, “Can you see how this tumor intersects the light blue track, but not the green track or the yellow one?”

Indeed they could, illuminating the two lobes whose fibers the tumor crossed. One, the occipital, is responsible for visual processing, while the other controls the sense of touch.

Griswold, a world-renowned radiology researcher, explained that he had worked with datasets of brain MRIs for more than a decade, “and I never fully understood their 3-D structure until I saw them in HoloLens.”

And Davis added that students who had used the HoloLens devices reported that 15 minutes with the three-dimensional images “could have saved them dozens of hours” in their traditional anatomy labs.

“The quicker our students learn facts like these,” Davis said, “the more time they have to think with them. We are teaching them to think like a doctor.”

Case Western Reserve was the only university among 30 organizations that partnered with the company for initial pilot testing of the HoloLens’ potential. Others include NASA, Japan Airlines, Volvo and the Dubai Electricity and Water Authority. Case Western Reserve also was the only organization to give a HoloLens demonstration on the Build stage.

The university’s team was introduced by Microsoft’s Alex Kipman, the lead creator of the HoloLens device. Shortly before Case Western Reserve’s presentation, Kipman and HoloLens co-creator Kudo Tsunoda announced that the first HoloLens holographic computer kits would ship to developers and enterprise clients that very day.

“Today, today is the day, where we leap from dreaming about mixed worlds to having Microsoft HoloLens become a reality,” Kipman said just before Tsunoda joined him on stage and the two embraced.

As Tsunoda left the stage, an emotional Kipman wiped his eye.

“I have waited for this moment,” he said, “for a very long time.”

Next: Check out how a developer used HoloLens to create sound-based navigation for the blind.