Watch the heart do its job with Cardiolens
HoloLens project tracks the heart and visualizes its activity for the wearer
Visualizing a person’s heartbeat is nothing new in the medical field. Electrocardiograms have been around since Willem Einthoven invented the first practical one in 1903, but medicine has come a long way since then. There are new ways to see the activity in a person’s heart, and Microsoft’s Cardiolens research project is one of them.
Headed up by researchers Daniel McDuff, Mar Gonzalez Franco and Christopher Hurter, Cardiolens allows surgeons to not only see a person’s heartbeat, it also provides a view of where blood is flowing. In short, it shows the effects of the heart’s beat throughout the body.
Cardiolens uses augmented reality and mixed reality to measure blood flow and vital signs. It runs on Microsoft’s HoloLens hardware, and it does not require the headset to be connected to any additional hardware.
Cardiolens makes use of a front-facing camera, remote imaging photoplethysmography software and a display that allows the wearer to see another person’s physiological state. This is all accomplished simply by looking at another person while wearing the HoloLens.
To the wearer, the readout appears as a white box around the other person’s face. The person’s pulse and heart rate are displayed near the box, and inside the white box is the person’s blood flow signal, visualized using several processes working in tandem.
The first process is tracking. Face-tracking and skin segmentation algorithms locate regions of interest (also called ROIs) in an image captured by the front-facing camera. The second process is sending, in which the remote imaging photoplethysmography (also known as iPPG) recovers the blood volume pulse, heart rate and heart rate variability of the person in the image. The IPPG is the real trick here; it’s an advanced set of methods that is able to measure this complex information using only a camera and ambient light, without the need for calibration.
Finally, the visualization process uses the aforementioned information to alter the appearance of the person displayed on the HoloLens, displaying a holographic overlay on top of the person’s face to show heart activity.
This heart activity appears as a red hue over the subject’s face. Using the pulse rat detected by Cardiolens, the subject’s heart rate variability and breathing rate can also be determined.
Cardiolens is useful in the medical field because it can be used by doctors to visualize blood flow in different parts of a patient’s body.
One of the big draws of Cardiolens is its simplicity. The application allows the wearer to view the subject’s magnified pulse signal, heart rate and pulse wave in real time, in a mixed-reality environment, using only the Microsoft HoloLens headset. This has promise to help clinicians better visualize medical issues and intervene at the point of care. See the study here.
What do you think?
You are the first to add a thought