Twin-to-twin transfusion syndrome (TTTS) is a serious condition affecting 10-15% of monochorionic diamniotic pregnancies. The pathophysiology of TTTS is not totally clear, but the presence of pathological anastomoses connecting the two circulations plays a major role in its development, as it causes an unbalanced blood flow between the fetuses. The more effective treatment option is fetoscopic laser photo-coagulation of the abnormal vascular connections. This procedure is challenging due to low image quality, limited Field of View (FoV), presence of turbid amniotic fluid, large illumination variability, occlusions, fetal and maternal movements. To overcome the described limitations, this thesis proposes a learning based framework for placental mosaicking from intra-operative videos, with occlusion recovery performed in an end-to-end fashion. Three typical Simultaneous Localization And Mapping (SLAM) tasks, tracking, mapping and relocalization, are performed. Features, descriptors and keypoints are extracted from image pairs through a learning based method. Keypoints are used for mosaicking reconstruction. The combination of feature extraction and matching and mosaicking corresponds to tracking and mapping. Descriptors and features are the inputs for occlusion recovery, which constitutes the relocalization task. The proposed technique for feature extraction is compared with feature based methods, VGG, ResNet and ResNet with a module for dimensionality reduction, and with classical descriptor based approaches, Scale Invariant Feature Transform (SIFT) and Oriented FAST and Rotated BRIEF (ORB). Experiments were conducted on 10 videos (2344 frames) from two different fetal surgery centres. The proposed framework is able to correctly relocalize frames in 93.75% of the conducted experiments, outperforming the other tested approaches. The obtained results show a promising approach to provide computer-assisted intervention during TTTS treatment, broadening the FoV and thus facilitating the anastomoses localization with a consequent decrease in surgical times.
La Sindrome da Trasfusione Feto-Fetale colpisce il 10-15% delle gravidanze monocoriali diamniotiche. La fisiopatologia non è ancora del tutto chiara, ma il suo sviluppo sembra essere legato soprattutto alla presenza di anastomosi patologiche che connettono i flussi sanguigni causando uno squilibrio nell’afflusso ematico dei feti. Il trattamento più efficace è la foto-coagulazione laser delle anastomosi. Bassa qualità delle immagini, campo di vista limitato, presenza di fluido amniotico torbido, ampia variabilità nell’illuminazione, occlusioni, movimenti materni e fetali rendono la procedura complessa. Per superare le difficoltà elencate, questa tesi propone un approccio learning based per la ricostruzione di immagini panoramiche della placenta a partire da video intra-operatori, robusto alle occlusioni e in grado di procedere alla ricostruzione in un unico flusso di esecuzione. Sono state implementate tre funzioni tipiche di Simultaneous Localization And Mapping (SLAM), ovvero tracking, mapping e rilocalizzazione. Features, descrittori e keypoints vengono estratti da coppie di immagini con un approccio learning based. I keypoints vengono utilizzati per la ricostruzione del panorama. La combinazione di estrazione e matching di features e realizzazione del panorama corrisponde alle funzioni di tracking e mapping. Descrittori e features sono gli input per il recupero delle occlusioni, che costituisce la funzione di rilocalizzazione. La tecnica proposta per l’estrazione di features viene confrontata con metodi feature based, VGG, ResNet e ResNet con un metodo per la riduzione di dimensionalità, e con approcci descriptor based, Scale Invariant Feature Transform (SIFT) e Oriented FAST and Rotated BRIEF (ORB). Sono stati condotti esperimenti su 10 video (2344 fotogrammi) provenienti da due centri di chirurgia fetale. La struttura proposta è in grado di effettuare correttamente il recupero delle occlusioni nel 93.75% degli esperimenti condotti, superando i risultati ottenuti dagli altri metodi. I risultati ottenuti mostrano come l’approccio proposto sia promettente per fornire assistenza al chirurgo mediante strumenti informatici avanzati, ampliando il campo di vista e facilitando così la localizzazione delle anastomosi in tempi ridotti.
Towards learning based SLAM framework for occlusion recovery and mosaicking in fetoscopy
LENA, CHIARA
2021/2022
Abstract
Twin-to-twin transfusion syndrome (TTTS) is a serious condition affecting 10-15% of monochorionic diamniotic pregnancies. The pathophysiology of TTTS is not totally clear, but the presence of pathological anastomoses connecting the two circulations plays a major role in its development, as it causes an unbalanced blood flow between the fetuses. The more effective treatment option is fetoscopic laser photo-coagulation of the abnormal vascular connections. This procedure is challenging due to low image quality, limited Field of View (FoV), presence of turbid amniotic fluid, large illumination variability, occlusions, fetal and maternal movements. To overcome the described limitations, this thesis proposes a learning based framework for placental mosaicking from intra-operative videos, with occlusion recovery performed in an end-to-end fashion. Three typical Simultaneous Localization And Mapping (SLAM) tasks, tracking, mapping and relocalization, are performed. Features, descriptors and keypoints are extracted from image pairs through a learning based method. Keypoints are used for mosaicking reconstruction. The combination of feature extraction and matching and mosaicking corresponds to tracking and mapping. Descriptors and features are the inputs for occlusion recovery, which constitutes the relocalization task. The proposed technique for feature extraction is compared with feature based methods, VGG, ResNet and ResNet with a module for dimensionality reduction, and with classical descriptor based approaches, Scale Invariant Feature Transform (SIFT) and Oriented FAST and Rotated BRIEF (ORB). Experiments were conducted on 10 videos (2344 frames) from two different fetal surgery centres. The proposed framework is able to correctly relocalize frames in 93.75% of the conducted experiments, outperforming the other tested approaches. The obtained results show a promising approach to provide computer-assisted intervention during TTTS treatment, broadening the FoV and thus facilitating the anastomoses localization with a consequent decrease in surgical times.| File | Dimensione | Formato | |
|---|---|---|---|
|
Thesis_Summary_Lena_Chiara.pdf
Open Access dal 12/04/2023
Descrizione: File with Executive Summary and Thesis
Dimensione
41.34 MB
Formato
Adobe PDF
|
41.34 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/188432