Three-dimensional rotational angiography has been a valuable addition to the congenital cardiac catheterisation laboratory and provides information that is otherwise unattainable with standard 2D angiography. However, viewing a 3D image on a 2D computer screen remains a major limitation of this technology. Extended reality technologies, such as augmented reality, have the potential to bridge available 3D imaging acquisition and advanced visualisation technologies by projecting interactive 3D models into the procedural environment. Reference Sterliński, Borucki, Szumowski, Michałowska, Opolski and Nicińska1–Reference Wake, Bjurlin, Rostami, Chandarana and Huang4 This technology has tremendous potential within the complexity and heterogeneity of CHD and interventional cardiology.
This case series describes the prospective use and feasibility of intraprocedural augmented reality visualisation of three-dimensional rotational angiography in the congenital cardiac catheterisation laboratory.
Methods
This was a single-centre, prospective study in which patients who presented to the congenital cardiac catheterisation laboratory who had a plan for 3D rotational angiography were prospectively enrolled and consented. All 3D rotational angiograms were performed within a Canon (formerly Toshiba) Infinix-I (Tochigi, Japan) fluoroscopic platform, and 3D reconstructions were performed solely using the Vitrea 3D workstation (Vital Solutions, Canon). Models were optimised within Vitrea by selecting the “Remove Fragment” option, followed by the “Visible” function to select areas of interest and “Erode” option to further remove minor artefact. The models were saved as STL files, which were then transferred directly to a Microsoft Hololens 2 mixed reality headset (Microsoft Corporation, Washington, USA) for intraprocedural visualisation while maintaining sterility (Fig 1A). The primary operators identified discrepancies between the augmented reality and computer models and described additional advantages of the augmented reality model. This study was approved by the institutional review board.
(A) Demonstration of intraprocedural augmented reality visualisation of a 3D rotational angiogram, including airway visualisation, in a patient (Case #4) following stage 2 palliation of hypoplastic left heart syndrome. (B and C). Comparison of computer screen (top) and first-person view of augmented reality (bottom) representations of the 3D models for Cases #5 and #3, respectively.
Cases
Augmented reality visualisation of 3D rotational angiography was utilised in five patients with CHD, including right and left heart obstructive lesions, as well as single-ventricle palliation (Table 1). The mean age was 8.1 ± 9.5 years and mean weight was 40.3 ± 45.1 kg. A mean total injection volume of 2.3 ± 1.2 cc/kg was used for 3D rotational angiography with a median contrast dilution to 50% (range 50–66%). Apnoea was performed for four of five 3D rotational angiograms and all five were performed with a 1-second fluoroscopic delay and without rapid ventricular pacing. To maintain a sterile field, the Hololens 2 headset was easily placed on the operators and removed by a separate staff member in all cases.
Patient and 3D model characteristics
3D, three-dimensional; AR, augmented reality; HLHS, hypoplastic left heart syndrome; LPA, left pulmonary artery; LSCA, left subclavian artery; PA, pulmonary artery; RPA, right pulmonary artery; RV, right ventricle; RVOT, right ventricular outflow tract; TOF, tetralogy of Fallot; TPVI, transcatheter pulmonary valve implantation.
Augmented reality visualisation was feasible in all cases (Figs 1B and 1C, Supplementary Fig 1), and the primary operator was able to maintain sterility, perform manipulations, and share the view and manipulations of the model with a second operator (Supplemental Video). The median time from acquisition of 3D rotational angiography to 3D reconstruction was 2.8 minutes (range 1.5–5.4), and the median time from 3D reconstruction to visualisation of the 3D model in augmented reality was 4.2 minutes (range 2.8–4.6). Interventions were performed in four of five cases (Table 1). Only one case showed a minor discrepancy between the augmented reality and computer models, with no major discrepancies reported.
Discussion
This case series reports on the use and feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in the congenital cardiac catheterisation laboratory. This exciting application of mixed reality technology allows the operator to appreciate the 3D model in its true form, rather than relying on a 2D computer screen to represent 3D structures and relationships. In this series of various forms of complex CHD, a wireless mixed reality headset was easily placed on the primary operator while remaining sterile, with full visibility of the procedural environment and with minimal additional procedural time. The models were visualised without significant discrepancies and were able to be manipulated and “sliced” through with simple movements of a finger. This takes the concept and benefits of 3D rotational angiography Reference Aldoss, Fonseca and Truong5–Reference Nguyen, Balzer, Murphy, Nicolas and Shahanavaz7 and combines them with the added visualisation advantages of 3D printing, Reference Poterucha, Foley and Taggart8,Reference Parimi, Buelter and Thanugundla9 to allow for advanced, real-time intraprocedural 3D modelling and an ultimate appreciation of the patient’s anatomy for the operators.
While holographic visualisation of 3D rotational angiography has been reported, Reference Bruckheimer, Rotschild and Dagan10 its application in the procedural environment has been limited by the need for a handheld tool to manipulate the image at a display workstation. Our study is the first report and demonstration to our knowledge of augmented reality visualisation of 3DRA in a “hands-free” procedural environment that allowed for continued operator sterility, full mobility, and preserved ability to visualise the patient, data, and catheterisation laboratory while utilising voice commands and hand gestures to control the model (Supplementary Video). Additionally, a second operator can interact with the same model through a separate headset, allowing for opportunities for enhanced education and unique team engagement, as well as giving the opportunity to include this technology in post-procedural counselling of the patient and family.
Important next steps include development and validation of intraprocedural augmented reality simulation of interventions, such as stent, valve, and device implantation, and the eventual incorporation of computational flow dynamics. Following validation of this tool, future studies should prove its utility for regular intraprocedural decision-making. Practical and feasible potential educational opportunities include facilitating “live case” transmissions for international conferences, remote proctoring for new device technologies and junior faculty, family counselling, and trainee instruction. Industry support for integration of this technology into specific fluoroscopic platforms will be a key future addition, thus allowing for gantry angles and other features to be further utilised within the augmented reality model.
Conclusion
Intraprocedural augmented reality visualisation of 3DRA is feasible in the cardiac catheterisation laboratory. This technology allows for an enhanced appreciation of 3D anatomy in its true form, complementing the existing resources that exist with 3DRA post-processing. Exciting opportunities include live intraprocedural sharing of imaging with a first-person view of the procedural environment, remote proctoring, and critical future development of intraprocedural simulation of transcatheter interventions. These features would provide the ability to acquire, simulate, and intervene all in one setting to provide the ultimate personalisation of patient care.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951122002153
Acknowledgements
The authors thank University of Michigan Emerging Technologies Group for software development, Dr Andrea Les for assistance with regulatory processes, Michelle Benedict for photography and videography, and Melissa Schopp for editing of photographs and videos.
Financial support
The study is funded by Helen L. Kay Foundation Grant, Faith’’s Angels Award, and Johnson Controls Award.
Conflicts of interest
None.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national guidelines on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008, and have been approved by the institutional review board committee of the University of Michigan.
Open access
