Diegetic User Interfaces in Extended Reality for 3D Medical Visualization

The integration of 3D anatomical models derived from DICOM images into extended reality (XR) interfaces significantly enhances the visualization and interaction with anatomical data, particularly in educational, pre-operative, and intra-operative medical scenarios. This paper presents an XR application developed with Unity 3D for the Meta Quest 3, enabling concurrent visualization of DICOM images and interaction with the 3D model in an immersive XR environment. The application incorporates an advanced diegetic user interface, allowing users to manipulate the 3D model intuitively through hand gestures for actions like annotating, dragging, dropping, and scaling objects. Utilizing a virtual tablet and slider, users can precisely control DICOM slice planes that intersect the 3D model, providing real-time updates of visual content. Enhanced by hand tracking, this interface aims to facilitate the natural manipulation of cross-sectional views, thereby optimizing anatomical visualization.