Mixed reality is a blend of the physical and virtual worlds that includes both real and computer-generated objects. The use of this ground-breaking technology has the power to significantly change how surgical procedures can be performed, providing virtual dimensions in exchange for a real impact in the next-generation operating rooms. The overlay of holographic informative content (patient-specific anatomy, images, clinical data) in a non-occlusive way over the surgical field can optimize treatment delivery and safety, with the end goal of improving patient outcomes and quality of care. Patient-specific 3D models generated from preoperative images (e.g., CT, MRI) are superimposed on the real views of the surgical field to provide surgeons with improved visualizations of the anatomical structures and/or to assist and guide them throughout the procedure. The view of MR anatomical structures can be supplemented with different kinds of data such as pre-procedural raw images, clinical data, and real-time acquired information, easily accessible by the surgeon using only gaze, gesture, and vocal commands. Besides the possibility to have a high-fidelity representation of the operative field, which cannot be accessed by direct vision in most of the image-guided-procedures, a Mixed Reality (MR) environment can also include navigation support capable of displaying not only a virtual model of the anatomical target but also virtual models of surgical instruments and catheters, whose position and orientation with respect to the target is precisely acquired at all times – flawlessly – during the procedure, so to allow for real-time automated registration and visualization of the virtual instruments within the virtual anatomy. In the next future, this will represent a disruptive innovation in the field of endovascular procedure potentially allowing the avoidance of fluoroscopic images, and empowering the capability of interventionalists while doing these complex procedures. This is potentially doable considering the increasing sources of Three-Dimensional (3D) information available in interventional procedures today, such as intraoperative devices capable of gathering data about their shape and position via device tracking technologies like optical tracking, electromagnetic sensing, and fiber-optic sensing. However, when significant deformations arise as a result of instrument-tissue contact, anatomical models created from preoperative imaging do not always accurately reflect the dynamic conditions of the real anatomy. In this scenario, real-time simulations could pave the way to a new frontier of holographic dynamic visualization and procedural awareness. A model able to account for the deformations arising from both surgical manipulations and physiological movements can indeed be used to facilitate intra-operative navigation giving a more realistic representation of the organs inside the patient abruptly reducing the exposure to damaging radiations in all phases of the procedure. All these considerations need to coexist in an ideal MR system, where no perceivable difference between the user’s natural experience of the world and his augmented experience through the MR interface should exist. For this goal, the conditions to be satisfied are twofold: an accurate spatial registration between real-world scenes and computer-generated content and real-time interaction between the MR scene and real patient data. This dissertation presents the work towards overcoming the aforementioned technical and clinical challenges to apply MR-based approaches for image-guided surgery, particularly tailoring two main clinical fields, namely neurosurgery and endovascular treatments.
La realtà mista è una sovrapposizione tra il mondo fisico e virtuale che include sia oggetti reali che digitali generati dal computer. L'uso di questa tecnologia innovativa ha il potere di cambiare in modo significativo il modo in cui le procedure chirurgiche possono essere eseguite, fornendo dimensioni virtuali in cambio di un impatto reale nelle sale operatorie di nuova generazione. La sovrapposizione di contenuti informativi olografici (anatomia specifica del paziente, immagini, dati clinici) in modo non occlusivo sul campo chirurgico può ottimizzare l'erogazione e la sicurezza del trattamento, con l'obiettivo finale di migliorare l’outcome del paziente e la qualità dell'assistenza. I modelli 3D specifici del paziente generati dalle immagini preoperatorie (ad es. TAC, MRI) vengono sovrapposti alle viste reali del campo chirurgico per fornire ai chirurghi una migliore visualizzazione delle strutture anatomiche e/o per assisterli e guidarli durante l'intera procedura. La vista delle strutture anatomiche olografiche può essere integrata con diversi tipi di dati come immagini grezze pre-procedurali, dati clinici e informazioni acquisite in tempo reale, facilmente accessibili dal chirurgo utilizzando solo lo sguardo, i gesti e i comandi vocali. Oltre alla possibilità di avere una rappresentazione ad alta fedeltà del campo operatorio, non accessibile per visione diretta nella maggior parte delle procedure guidate da immagini, un ambiente di Mixed Reality (MR) può includere anche un supporto alla navigazione in grado di visualizzare non solo un modello virtuale del target anatomico ma anche modelli virtuali di strumenti chirurgici e cateteri, la cui posizione e orientamento rispetto al target viene acquisita con precisione in ogni momento – in modo impeccabile – durante la procedura, in modo da consentire la registrazione e la visualizzazione automatizzate in tempo reale di strumenti virtuali all'interno dell'anatomia virtuale. Nel prossimo futuro, ciò rappresenterà un'innovazione dirompente nel campo delle procedure endovascolari, consentendo potenzialmente di evitare l’uso di immagini fluoroscopiche e potenziando la capacità degli interventisti durante l'esecuzione di queste procedure complesse. Ciò è potenzialmente fattibile considerando le crescenti risorse tecnologiche in grado di fornire informazioni tridimensionali (3D) disponibili oggi durante le procedure interventistiche. Queste includono dispositivi intraoperatori in grado di raccogliere dati sulla loro forma e posizione tramite tecnologie di tracciamento del dispositivo come il tracciamento ottico, il rilevamento elettromagnetico e le fibre ottiche. Tuttavia, quando si verificano deformazioni significative a seguito del contatto strumento-tessuto, i modelli anatomici creati dall'imaging preoperatorio non sempre riflettono accuratamente le condizioni dinamiche dell'anatomia reale. In questo scenario, le simulazioni in tempo reale potrebbero aprire la strada a una nuova frontiera della visualizzazione dinamica olografica e della consapevolezza procedurale. Un modello in grado di rendere conto delle deformazioni derivanti sia dalle manipolazioni chirurgiche che dai movimenti fisiologici può infatti essere utilizzato per facilitare la navigazione intraoperatoria fornendo una rappresentazione più realistica degli organi all'interno del paziente riducendo bruscamente l'esposizione a radiazioni dannose in tutte le fasi della procedura . Tutte queste considerazioni devono coesistere in un sistema olografico ideale, in cui non dovrebbe esistere alcuna differenza percepibile tra l'esperienza naturale del mondo dell'utente e la sua esperienza aumentata attraverso l'interfaccia in Mixed Reality. Per questo obiettivo, le condizioni da soddisfare sono duplici: un'accurata registrazione spaziale tra scene del mondo reale e contenuto generato dal visore olografico e interazione in tempo reale tra la scena olografica e i dati reali del paziente. Questa tesi presenta il lavoro per superare le suddette sfide tecniche e cliniche per applicare approcci basati sulla MR per la chirurgia guidata dalle immagini, in particolare applicandole a due campi clinici principali, ovvero la neurochirurgia e i trattamenti endovascolari.
Unveiling the Invisible : mixed reality based approaches for surgical empowerment
Palumbo, Maria Chiara
2022/2023
Abstract
Mixed reality is a blend of the physical and virtual worlds that includes both real and computer-generated objects. The use of this ground-breaking technology has the power to significantly change how surgical procedures can be performed, providing virtual dimensions in exchange for a real impact in the next-generation operating rooms. The overlay of holographic informative content (patient-specific anatomy, images, clinical data) in a non-occlusive way over the surgical field can optimize treatment delivery and safety, with the end goal of improving patient outcomes and quality of care. Patient-specific 3D models generated from preoperative images (e.g., CT, MRI) are superimposed on the real views of the surgical field to provide surgeons with improved visualizations of the anatomical structures and/or to assist and guide them throughout the procedure. The view of MR anatomical structures can be supplemented with different kinds of data such as pre-procedural raw images, clinical data, and real-time acquired information, easily accessible by the surgeon using only gaze, gesture, and vocal commands. Besides the possibility to have a high-fidelity representation of the operative field, which cannot be accessed by direct vision in most of the image-guided-procedures, a Mixed Reality (MR) environment can also include navigation support capable of displaying not only a virtual model of the anatomical target but also virtual models of surgical instruments and catheters, whose position and orientation with respect to the target is precisely acquired at all times – flawlessly – during the procedure, so to allow for real-time automated registration and visualization of the virtual instruments within the virtual anatomy. In the next future, this will represent a disruptive innovation in the field of endovascular procedure potentially allowing the avoidance of fluoroscopic images, and empowering the capability of interventionalists while doing these complex procedures. This is potentially doable considering the increasing sources of Three-Dimensional (3D) information available in interventional procedures today, such as intraoperative devices capable of gathering data about their shape and position via device tracking technologies like optical tracking, electromagnetic sensing, and fiber-optic sensing. However, when significant deformations arise as a result of instrument-tissue contact, anatomical models created from preoperative imaging do not always accurately reflect the dynamic conditions of the real anatomy. In this scenario, real-time simulations could pave the way to a new frontier of holographic dynamic visualization and procedural awareness. A model able to account for the deformations arising from both surgical manipulations and physiological movements can indeed be used to facilitate intra-operative navigation giving a more realistic representation of the organs inside the patient abruptly reducing the exposure to damaging radiations in all phases of the procedure. All these considerations need to coexist in an ideal MR system, where no perceivable difference between the user’s natural experience of the world and his augmented experience through the MR interface should exist. For this goal, the conditions to be satisfied are twofold: an accurate spatial registration between real-world scenes and computer-generated content and real-time interaction between the MR scene and real patient data. This dissertation presents the work towards overcoming the aforementioned technical and clinical challenges to apply MR-based approaches for image-guided surgery, particularly tailoring two main clinical fields, namely neurosurgery and endovascular treatments.| File | Dimensione | Formato | |
|---|---|---|---|
|
MCP_PhdThesis_2023.pdf
Open Access dal 14/01/2024
Descrizione: Doctoral_Dissertation_Palumbo_MariaChiara
Dimensione
155.19 MB
Formato
Adobe PDF
|
155.19 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/195905