by Julian Hettig, Sandy Engelhardt, Christian Hansen, Gabriel Mistelbauer
Abstract:
PURPOSE : Augmented reality (AR) has emerged as a promising approach to support surgeries; however, its application in real world scenarios is still very limited. Besides sophisticated registration tasks that need to be solved, surgical AR visualizations have not been studied in a standardized and comparative manner. To foster the development of future AR applications, a steerable framework is urgently needed to rapidly evaluate new visualization techniques, explore their individual parameter spaces and define relevant application scenarios. METHODS : Inspired by its beneficial usage in the automotive industry, the underlying concept of virtual reality (VR) is capable of transforming complex real environments into controllable virtual ones. We present an interactive VR framework, called Augmented Visualization Box (AVB), in which visualizations for AR can be systematically investigated without explicitly performing an error-prone registration. As use case, a virtual laparoscopic scenario with anatomical surface models was created in a computer game engine. In a study with eleven surgeons, we analyzed this VR setting under different environmental factors and its applicability for a quantitative assessment of different AR overlay concepts. RESULTS : According to the surgeons, the visual impression of the VR scene is mostly influenced by 2D surface details and lighting conditions. The AR evaluation shows that, depending on the visualization used and its capability to encode depth, 37\% to 91\% of the experts made wrong decisions, but were convinced of their correctness. These results show that surgeons have more confidence in their decisions, although they are wrong, when supported by AR visualizations. CONCLUSION : With AVB, intraoperative situations are realistically simulated to quantitatively benchmark current AR overlay methods. Successful surgical task execution in an AR system can only be facilitated if visualizations are customized toward the surgical task.
Reference:
AR in VR: assessing surgical augmented reality visualizations in a steerable virtual reality environment. (Julian Hettig, Sandy Engelhardt, Christian Hansen, Gabriel Mistelbauer), In International journal of computer assisted radiology and surgery, volume 13, 2018.
Bibtex Entry:
@article{hettig_ar_2018,
	title = {{AR} in {VR}: assessing surgical augmented reality visualizations in a steerable virtual reality environment.},
	volume = {13},
	issn = {1861-6429 1861-6410},
	doi = {10.1007/s11548-018-1825-4},
	abstract = {PURPOSE : Augmented reality (AR) has emerged as a promising approach to support surgeries; however, its application in real world scenarios is still very limited. Besides sophisticated registration tasks that need to be solved, surgical AR visualizations have not been studied in a standardized and comparative manner. To foster the development of future AR applications, a steerable framework is urgently needed to rapidly evaluate new visualization techniques, explore their individual parameter spaces and define relevant application scenarios. METHODS : Inspired by its beneficial usage in the automotive industry, the underlying concept of virtual reality (VR) is capable of transforming complex real environments into controllable virtual ones. We present an interactive VR framework, called Augmented Visualization Box (AVB), in which visualizations for AR can be systematically investigated without explicitly performing an error-prone registration. As use case, a virtual laparoscopic scenario with anatomical surface models was created in a computer game engine. In a study with eleven surgeons, we analyzed this VR setting under different environmental factors and its applicability for a quantitative assessment of different AR overlay concepts. RESULTS : According to the surgeons, the visual impression of the VR scene is mostly influenced by 2D surface details and lighting conditions. The AR evaluation shows that, depending on the visualization used and its capability to encode depth, 37\% to 91\% of the experts made wrong decisions, but were convinced of their correctness. These results show that surgeons have more confidence in their decisions, although they are wrong, when supported by AR visualizations. CONCLUSION : With AVB, intraoperative situations are realistically simulated to quantitatively benchmark current AR overlay methods. Successful surgical task execution in an AR system can only be facilitated if visualizations are customized toward the surgical task.},
	language = {eng},
	number = {11},
	journal = {International journal of computer assisted radiology and surgery},
	author = {Hettig, Julian and Engelhardt, Sandy and Hansen, Christian and Mistelbauer, Gabriel},
	month = nov,
	year = {2018},
	pmid = {30043197},
	keywords = {*User-Computer Interface, *Virtual Reality, Anatomic, Computer-Assisted/*methods, Humans, Laparoscopy/*methods, Models, Surgery, Surgical augmented reality, Virtual reality, Visualization},
	pages = {1717--1725}
}