Aim: State-of-the art cardiac imaging ought to be based on three-dimensional (3D) datasets for optimal understanding of complex anatomy and reliable quantification. However, displaying 3D information on standard computer monitors may represent a limitation hampering access to important content. We hypothesized that innovative mixed reality technology can be successfully implemented for navigating 3D datasets obtained with state-of-the-art echocardiographic equipment.
Methods: We used 35 datasets (12 transthoracic, 23 transesophageal, 20% full volume acquisitions) for conversion via Carthesian DICOM file format to test the feasibility of mixed reality, head-mounted device overlying holographic image of cardiac data onto real-world viewing by operator. We developed a dedicated software pathway for files conversion, real-time Wi-Fi streaming of 3D rendering from PC to device and manipulation of spatial data including multiuser interface for data sharing. This custom software module has been previously used for advanced visualization of 3D cardiac imaging (CT, 3DRA, MR), allowing to perform volume rendering with color transfer functions, transparency and advanced shading.
Results: All 3D echo datasets were successfully converted and displayed in mixed reality as a holographic image. The quality of visualization was diagnostic without content loss in 91% of datasets as judged by operator, based on datasets with valve disease, congenital disease/percutaneous occluder and cardiac masses. Manual segmentation was applied to optimize content visibility. Volume rendered and maximum intensity views were successfully tested. Navigation in dataset was accessible via hand gestures and voice commands, including 3D manipulation (translation, rotation, scale) and volume cropping. Touchless user interface holds promise for use in interventional theatres without compromising sterility and direct operator control of image by using head-mounted displays rather than fixed screens. Figure shows the operation of mixed reality interactive visualization of 3D echocardiography. Computer demonstration is planned as a presentation add-on.
Conclusions: Mixed reality display using head-mounted device is feasible and shows promise for fully volumetric, intuitive display and navigation in spatial datasets obtained with routine three-dimensional echocardiography. Hands-free Touchless control seems promising for interventional applications. Integrating color flow information and automatizing data conversion for fast, fully real-time experience and segmentation is subject to ongoing developments.