Publications
Accurate detection of tumor boundaries is critical for the success of oncologic surgical intervention. Traditionally, palpation can handover important information for tumor localization based on the tissue mechanical properties, but in Minimally Invasive Surgery no direct access to the tumor for palpation is feasible. For providing a technical analogy, this feasibility-level study focusses on the simplified problem of detection of an inclusion within a homogeneous silicon phantom. We hypothesized that existence of a relatively stiffer inclusion within an elastomer tissue phantom changes vibroacoustic signatures under forced vibration conditions. In comparison with previous studies, in this work the measurement probe was static, and the short-time (1 s) data package analysis targeted at nearly real-time inclusion detection. The inclusion detection problem was cast into a binary classification of the short-time acquired vibroacoustic signals. The method involves a wavelet-based multilayer perceptron neural network (MLP) that is trained in a supervised manner. A micro-electro-mechanical system (MEMS) sensor proximally attached to a solid probe was used to measure the vibroacoustic signals. Phantoms of simulated healthy tissue with stiffer tumor model inclusions were used for experiments and data collection. From the 120 overall number of experiments, 15 % were used as test data to evaluate the performance. The results show inclusion detection F1 score of 75 %, and 77.8 % accuracy related to the confusion matrix, reflecting the model performance on previously unseen data. Performance of the classifier was discussed in terms of various binary classification metrics, and compared with another established classifier, support vector machine (SVM). While the results support the hypothesis of this proof-of-concept study, extensions like improving the electronic system and refining the method with more experiments on biological tissues remain as the future work.
Exploring 3D surface data often involves navigating complex menus, creating challenges for both experts and new users who may struggle with overloaded interfaces. To address this, we introduce VocalVis, an open-source prototype that leverages real-time voice processing to simplify interaction and analysis of 3D surface data. By capitalizing on advancements in natural language processing, VocalVis enables users to interact with digital content through voice commands, addressing challenges associated with traditional navigation methods. Our study involves both data scientist novices and domain experts, offering insights into how voice interaction can streamline data exploration. The findings demonstrate the potential of voice as a powerful supplementary tool for visually analyzing complex datasets. This innovative approach opens up new possibilities for intuitive, accessible data exploration, making it convenient for a broader range of users to engage with 3D scientific data.
Voice user interfaces for effortless navigation in medical virtual reality environments
Jan Hombeck, Henrik Voigt, Kai Lawonn
In various situations, such as clinical environments with sterile conditions or when hands are occupied with multiple devices, traditional methods of navigation and scene adjustment are impractical or even impossible. We explore a new solution by using voice control to facilitate interaction in virtual worlds to avoid the use of additional controllers. Therefore, we investigate three scenarios: Object Orientation, Visualization Customization, and Analytical Tasks and evaluate whether natural language interaction is possible and promising in each of these scenarios. In our quantitative user study participants were able to control virtual environments effortlessly using verbal instructions. This resulted in rapid orientation adjustments, adaptive visual aids, and accurate data analysis. In addition, user satisfaction and usability surveys showed consistently high levels of acceptance and ease of use. In conclusion, our study shows that the use of natural language can be a promising alternative for the improvement of user interaction in virtual environments. It enables intuitive interactions in virtual spaces, especially in situations where traditional controls have limitations.
Using speech for hands-free interaction in Virtual Reality (VR) is gaining popularity, supported by advances in natural language processing that enable accurate speech-to-text transcription and intent inference. Prior work has shown that voice-based navigation is effective when hand use is restricted or occupied, particularly when visually salient points in the scene, known as landmarks, can be identified and articulated. However, in many virtual environments, such landmarks may be absent or not easily recognizable, limiting the applicability of voice commands. To address this, we propose three landmark-free, user-centered coordinate systems to support speech-based locomotion in VR: Cartesian, cylindrical, and spherical. Each system uses a distinct encoding and interaction style. The Cartesian system employs a simple three-digit code and requires low cognitive effort. The spherical system prioritizes precision but demands higher mental effort. The cylindrical system combines elements of both, offering a balance between usability and accuracy. We evaluated these systems in a quantitative user study with 24 participants, all with backgrounds in information technology. The study assessed the systems’ effectiveness for object positioning and navigation, comparing them to teleportation, the standard non-voice locomotion method in VR. Results show that the Cartesian system enables faster and more intuitive navigation compared to the spherical system. Distinct usage patterns were observed, with users predominantly focusing on the central field of view during navigation. Our findings indicate that user-centered coordinate systems are practical to implement and present a viable alternative for speech-driven, hands-free navigation in VR.
AortaAnalyzer: Interactive, integrated CTA aorta segmentation and quantitative analysis platform
Fabienne von Deylen, Pepe Eulzer, Kai Lawonn
The diagnosis of aortic diseases could be significantly enhanced with modern advances in model-based vessel visualization, objective parameter quantification, as well as information gained through numerical blood flow simulation. Most state-of-the-art methods, however, require heavy processing and are often split across various frameworks that require setting up complex workflows, making many clinical applications unrealistic and hindering research on large datasets. We present the AortaAnalyzer, a unified, end-to-end pipeline for processing computed-tomography angiography (CTA) of the aorta, integrating a state-of-the-art 3D segmentation network (Dice 0.95+-0.01, HD95 5.25+-5.73 mm), interactive correction tools, automated surface extraction, robust centerline computation, inlet/outlet capping for numerical hemodynamics, and clinical metric quantification. All modules share a single GUI, use standard formats (nrrd, STL, OBJ, CSV), and propagate changes automatically, eliminating complex multi-tool workflows. We developed the framework in an iterative process based on evaluations with seven independent experts—two numerical hemodynamics researchers, two vessel visualization researchers, two cardiac surgeons, and one radiologist. The framework received high usefulness ratings and feature requests drove the addition of surface capping and extended metric measurements. To assess efficiency, we compared processing time against 3D Slicer and SimVascular. The AortaAnalyzer demonstrated increased robustness and required substantially less manual interaction and overall processing time. AortaAnalyzer supports both clinical assessment and research purposes by providing rapid visualization of the vessel morphology, reproducible diameter, volume, and landmark analysis, and accelerated pre-processing for blood-flow simulation. It is open access and serves as an extendable platform.
The mechanical properties of tumor tissue differ from those of healthy tissue. Therefore, surgeons palpate accessible surgical sites to determine tumor boundaries prior to resection. However, palpation is not possible during minimally invasive surgery, so instrumented palpation is required instead. This study investigates the suitability of an engineering method that combines mechanical object scanning and indentation to determine Young’s modulus of soft, tissue-like materials. To establish a defined reference, we tested our concept on silicone phantoms containing stiff tumor-like inclusions. We used a sensor consisting of a load cell connected to a rigid probe with a spherical indenter tip. Young’s modulus was calculated by measured force, indentation depth, and indenter geometry. These results were compared with those of a palpation experiment on the same specimens, conducted with surgeons. Validation results reflect the accuracy of the method. Error in estimation of Young’s modulus is: soft material 6.7%, stiff material 44.9%. Repeatability is high, with a standard deviation <7%. By scanning a phantom and creating a stiffness image, we were able to identify the location and shape of the inclusion more clearly than experienced surgeons could using manual palpation. Looking ahead, the prospect of miniaturizing the presented technique for localizing tumor boundaries during surgery seems promising.