In order to bring you the best possible user experience, this site uses Javascript. If you are seeing this message, it is likely that the Javascript option in your browser is disabled. For optimal viewing of this site, please ensure that Javascript is enabled for your browser.

Member Benefit

This content is only available year-round to ESC Professional Members, Fellows of the ESC, and Young Combined Members

Searching for ultimate cardiac data display - Mixed reality interactive visualization of three-dimensional echocardiographic datasets.

Session Multimodality imaging: from virtual to invasive imaging

Speaker Doctor Maciej Stanuch

Event : ESC Congress 2018

  • Topic : imaging
  • Sub-topic : Cross-Modality and Multi-Modality Imaging Topics, Other
  • Session type : Moderated Posters

Authors : J Pawlowski (Krakow,PL), K Janc (Krakow,PL), U Kaczor (Krakow,PL), J Kaminski (Krakow,PL), JI Kasprzak (Krakow,PL), M Kierepka (Krakow,PL), P Lipiec (Lodz,PL), K Wdowiak-Okrojek (Lodz,PL), J Witowski (Krakow,PL), D Zebrowski (Krakow,PL), J D Kasprzak (Lodz,PL)

J. Pawlowski1 , K. Janc1 , U. Kaczor1 , J. Kaminski2 , J.I. Kasprzak1 , M. Kierepka1 , P. Lipiec3 , K. Wdowiak-Okrojek3 , J. Witowski1 , D. Zebrowski2 , J.D. Kasprzak3 , 1MedApp SA - Krakow - Poland , 2AGH University of Science and Technology and MedApp SA - Krakow - Poland , 3Medical University of Lodz - Lodz - Poland ,

Cross-Modality and Multi-Modality Imaging, Other

European Heart Journal ( 2018 ) 39 ( Supplement ), 449

Aim: State-of-the art cardiac imaging ought to be based on three-dimensional (3D) datasets for optimal understanding of complex anatomy and reliable quantification. However, displaying 3D information on standard computer monitors may represent a limitation hampering access to important content. We hypothesized that innovative mixed reality technology can be successfully implemented for navigating 3D datasets obtained with state-of-the-art echocardiographic equipment.

Methods: We used 35 datasets (12 transthoracic, 23 transesophageal, 20% full volume acquisitions) for conversion via Carthesian DICOM file format to test the feasibility of mixed reality, head-mounted device overlying holographic image of cardiac data onto real-world viewing by operator. We developed a dedicated software pathway for files conversion, real-time Wi-Fi streaming of 3D rendering from PC to device and manipulation of spatial data including multiuser interface for data sharing. This custom software module has been previously used for advanced visualization of 3D cardiac imaging (CT, 3DRA, MR), allowing to perform volume rendering with color transfer functions, transparency and advanced shading.

Results: All 3D echo datasets were successfully converted and displayed in mixed reality as a holographic image. The quality of visualization was diagnostic without content loss in 91% of datasets as judged by operator, based on datasets with valve disease, congenital disease/percutaneous occluder and cardiac masses. Manual segmentation was applied to optimize content visibility. Volume rendered and maximum intensity views were successfully tested. Navigation in dataset was accessible via hand gestures and voice commands, including 3D manipulation (translation, rotation, scale) and volume cropping. Touchless user interface holds promise for use in interventional theatres without compromising sterility and direct operator control of image by using head-mounted displays rather than fixed screens. Figure shows the operation of mixed reality interactive visualization of 3D echocardiography. Computer demonstration is planned as a presentation add-on.

Conclusions: Mixed reality display using head-mounted device is feasible and shows promise for fully volumetric, intuitive display and navigation in spatial datasets obtained with routine three-dimensional echocardiography. Hands-free Touchless control seems promising for interventional applications. Integrating color flow information and automatizing data conversion for fast, fully real-time experience and segmentation is subject to ongoing developments.

Mixed reality interactive 3D display

Member Benefit

This content is only available year-round to ESC Professional Members, Fellows of the ESC, and Young Combined Members

Get your access to resources

Join now
  • 1ESC Professional Members – access all ESC Congress resources 
  • 2ESC Association Members (Ivory, Silver, Gold) – access your Association’s resources
  • 3Under 40 or in training - with a Combined Membership, access all resources
Join now
logo esc

Our mission: To reduce the burden of cardiovascular disease

Who we are