Robot-Assisted Minimally Invasive Surgery (RAMIS) represents a significant ad- vancement in surgical technology, merging the precision and control of robotic sys- tems with the benefits of minimally invasive techniques. Autonomy in robotic surgery represents a frontier in medical technology, where robotic systems are equipped with advanced algorithms that enable them to perform certain surgical tasks independently or in collaboration with human surgeons. This concept is rooted in the desire to en- hance precision, consistency, and safety in surgical procedures, while also addressing the limitations of human capabilities, such as fatigue, variability, and error. Developing autonomous robotic surgical systems requires complex algorithms capable of real-time decision-making, learning, and adaptation in highly dynamic and unpre- dictable environments. Traditional control theory, often based on predefined models and rules, has been the cornerstone of robotic surgery for years. However, these methods can struggle with the inherent variability and unpredictability of surgical environments. Additionally, safety constraints in RAMIS are critical guidelines that ensure the surgical robot operates within safe boundaries, minimizing the risk of harm to the patient. For example, collision avoidance algorithms to prevent the robot from inadvertently contacting non-target areas and operational safety constraints such as virtual fixtures (VF) during teleoperation. As robotic systems evolve to incorpo- rate higher levels of autonomy, the importance of robust safety mechanisms becomes increasingly paramount. The overall objective of this thesis is to develop a safe and robust intelligent control framework based on the da Vinci robotic surgical system (Intuitive Surgical, CA, USA), with the goal of advancing the level of automation in robot-assisted surgery. The framework fully considers safety constraints within surgical tasks. In particular, the research has been focused on the following topics: * Digital twin of the da Vinci robotic surgical system: the digital twin is designed to accurately replicate the kinematics, dynamics, and control aspects of the da Vinci robotic surgical system. It also integrates a detailed model of the surgical environment, enabling the simulation of various surgical scenarios. The digital twin can supply a virtual environment for surgical training and a learning environment for machine learning such as reinforcement learning (RL). * Digital twin with VFs for surgical training: surgical training is a critical component of preparing surgeons for modern surgical procedures, particularly in robot-assisted surgery. The digital twin introduced above offers a safe, con- trolled, and highly realistic environment for training, which is further enhanced by the incorporation of safety constraints using VFs. Previous research on VFs mainly focused on force calculation methodology, while the guidance paths of VFs need to be manually designed. We employ RL techniques to automatically generate VF guidance paths in dynamic scenarios. The RL-based VF provides real-time assistance to trainees in navigating the surgical robot toward the tar- get anatomy while avoiding collision with risky tissues. * Safe RL for Task Automation: RL methods have shown considerable per- formance in robot autonomous control in complex environments. However, ex- isting RL algorithms for surgical robots do not consider any safety requirements. Safe Experience Reshaping (SER) was proposed to take safety constraints into RL formula. It can be integrated into any offline RL algorithm. Firstly, the method identifies and learns the geometry of constraints. Secondly, a safe ex- perience is obtained by projecting an unsafe action to the tangent space of the learned geometry, which means that the action is in the safe space. Then, the collected safe experiences are used for safe policy training. Results show that our method gets a low rate of constraint violations and a high convergence speed. * Generalization and Sim-to-Real implementation: Although our experi- mental platform is the da Vinci surgical robot, our automation algorithms can run independently on general robotic platforms. In this thesis, we have in- troduced the method for implementing our approach on general robotic arms. Then, we provided a detailed description of the sim-to-real implementation pro- cess. Finally, we validated the effectiveness of our proposed method through three carefully designed tasks on real robots. Specifically, we also extended safe reinforcement learning techniques by proposing a new and more effective method for handling dynamic constraints. we propose a virtual fixture based safe reinforcement learning (VF-SRL) framework to ensure safety constraints in the context of autonomous robotic lymphadenectomy. The framework ensures that the agent, particularly multi-joint robotic manipulator agents, acts within the hard constraints established by the Virtual Fixtures (VF). Subsequently, we transfer the trained safe policy to the real robot. In the transfer phase, we design a visual module to detect safety constraints in the scene to construct the VF. Results show that our framework gets a lower rate of constraint violations and better performance in task success. Supplementary material can be viewed in the videos: Digital Twin, which shows the digital twin of dVRK; Autonomous Grasp1 and Autonomous Grasp2 shows the robot manipulating objects autonomously while avoiding collisions; Human robot interac- tion and Human Robot Collaboration shows the robot trying to grasp targets while avoiding human-operated tools, which aims to improve the human-robot collabora- tion.

L’obiettivo generale di questa tesi è sviluppare un framework di controllo intelligente, sicuro e robusto basato sul sistema chirurgico robotico da Vinci (Intuitive Surgical, CA, USA), con lo scopo di avanzare il livello di automazione nella chirurgia assistita da robot. Il framework prende pienamente in considerazione i vincoli di sicurezza all’interno dei compiti chirurgici e la collaborazione uomo-robot. In particolare, la ricerca si è concentrata sui seguenti temi: * Gemello digitale del sistema chirurgico robotico da Vinci: Il gemello digitale è progettato per replicare accuratamente la cinematica, la dinamica e gli aspetti di controllo del sistema chirurgico robotico da Vinci. Integra inoltre un modello dettagliato dell’ambiente chirurgico, consentendo la simu- lazione di vari scenari chirurgici. Il gemello digitale può fornire un ambiente virtuale per l’addestramento chirurgico e per l’apprendimento automatico, come l’Apprendimento per Rinforzo (RL). * Gemello digitale con VFs per l’addestramento chirurgico: L’addestramento chirurgico è una componente critica nella preparazione dei chirurghi per le pro- cedure chirurgiche moderne, in particolare nella chirurgia assistita da robot. Il gemello digitale descritto sopra offre un ambiente di addestramento sicuro, con- trollato e altamente realistico, ulteriormente migliorato dall’incorporazione di vincoli di sicurezza mediante i VFs. La ricerca precedente sui VFs si è concen- trata principalmente sulla metodologia di calcolo delle forze, mentre i percorsi di guida dei VFs dovevano essere progettati manualmente. Utilizziamo tecniche di RL per generare automaticamente percorsi di guida dei VFs in scenari dinam- ici. Il VF basato su RL fornisce assistenza in tempo reale ai tirocinanti nella navigazione del robot chirurgico verso l’anatomia target, evitando collisioni con tessuti a rischio. * RL sicuro per l’automazione dei compiti: I metodi di RL hanno di- mostrato prestazioni considerevoli nel controllo autonomo dei robot in ambi- enti complessi. Tuttavia, gli algoritmi di RL esistenti per i robot chirurgici non considerano requisiti di sicurezza. È stato proposto il Safe Experience Re- shaping (SER) per integrare i vincoli di sicurezza nella formulazione del RL. Questo può essere integrato in qualsiasi algoritmo di RL offline. In primo lu- ogo, il metodo identifica e apprende la geometria dei vincoli. In secondo lu- ogo, un’esperienza sicura viene ottenuta proiettando un’azione non sicura nello spazio tangente della geometria appresa, il che significa che l’azione si trova nello spazio sicuro. Successivamente, le esperienze sicure raccolte vengono utilizzate per l’addestramento di politiche sicure.

Autonomous object manipulation for robot assisted minimally invasive surgery

Fan, Ke
2024/2025

Abstract

Robot-Assisted Minimally Invasive Surgery (RAMIS) represents a significant ad- vancement in surgical technology, merging the precision and control of robotic sys- tems with the benefits of minimally invasive techniques. Autonomy in robotic surgery represents a frontier in medical technology, where robotic systems are equipped with advanced algorithms that enable them to perform certain surgical tasks independently or in collaboration with human surgeons. This concept is rooted in the desire to en- hance precision, consistency, and safety in surgical procedures, while also addressing the limitations of human capabilities, such as fatigue, variability, and error. Developing autonomous robotic surgical systems requires complex algorithms capable of real-time decision-making, learning, and adaptation in highly dynamic and unpre- dictable environments. Traditional control theory, often based on predefined models and rules, has been the cornerstone of robotic surgery for years. However, these methods can struggle with the inherent variability and unpredictability of surgical environments. Additionally, safety constraints in RAMIS are critical guidelines that ensure the surgical robot operates within safe boundaries, minimizing the risk of harm to the patient. For example, collision avoidance algorithms to prevent the robot from inadvertently contacting non-target areas and operational safety constraints such as virtual fixtures (VF) during teleoperation. As robotic systems evolve to incorpo- rate higher levels of autonomy, the importance of robust safety mechanisms becomes increasingly paramount. The overall objective of this thesis is to develop a safe and robust intelligent control framework based on the da Vinci robotic surgical system (Intuitive Surgical, CA, USA), with the goal of advancing the level of automation in robot-assisted surgery. The framework fully considers safety constraints within surgical tasks. In particular, the research has been focused on the following topics: * Digital twin of the da Vinci robotic surgical system: the digital twin is designed to accurately replicate the kinematics, dynamics, and control aspects of the da Vinci robotic surgical system. It also integrates a detailed model of the surgical environment, enabling the simulation of various surgical scenarios. The digital twin can supply a virtual environment for surgical training and a learning environment for machine learning such as reinforcement learning (RL). * Digital twin with VFs for surgical training: surgical training is a critical component of preparing surgeons for modern surgical procedures, particularly in robot-assisted surgery. The digital twin introduced above offers a safe, con- trolled, and highly realistic environment for training, which is further enhanced by the incorporation of safety constraints using VFs. Previous research on VFs mainly focused on force calculation methodology, while the guidance paths of VFs need to be manually designed. We employ RL techniques to automatically generate VF guidance paths in dynamic scenarios. The RL-based VF provides real-time assistance to trainees in navigating the surgical robot toward the tar- get anatomy while avoiding collision with risky tissues. * Safe RL for Task Automation: RL methods have shown considerable per- formance in robot autonomous control in complex environments. However, ex- isting RL algorithms for surgical robots do not consider any safety requirements. Safe Experience Reshaping (SER) was proposed to take safety constraints into RL formula. It can be integrated into any offline RL algorithm. Firstly, the method identifies and learns the geometry of constraints. Secondly, a safe ex- perience is obtained by projecting an unsafe action to the tangent space of the learned geometry, which means that the action is in the safe space. Then, the collected safe experiences are used for safe policy training. Results show that our method gets a low rate of constraint violations and a high convergence speed. * Generalization and Sim-to-Real implementation: Although our experi- mental platform is the da Vinci surgical robot, our automation algorithms can run independently on general robotic platforms. In this thesis, we have in- troduced the method for implementing our approach on general robotic arms. Then, we provided a detailed description of the sim-to-real implementation pro- cess. Finally, we validated the effectiveness of our proposed method through three carefully designed tasks on real robots. Specifically, we also extended safe reinforcement learning techniques by proposing a new and more effective method for handling dynamic constraints. we propose a virtual fixture based safe reinforcement learning (VF-SRL) framework to ensure safety constraints in the context of autonomous robotic lymphadenectomy. The framework ensures that the agent, particularly multi-joint robotic manipulator agents, acts within the hard constraints established by the Virtual Fixtures (VF). Subsequently, we transfer the trained safe policy to the real robot. In the transfer phase, we design a visual module to detect safety constraints in the scene to construct the VF. Results show that our framework gets a lower rate of constraint violations and better performance in task success. Supplementary material can be viewed in the videos: Digital Twin, which shows the digital twin of dVRK; Autonomous Grasp1 and Autonomous Grasp2 shows the robot manipulating objects autonomously while avoiding collisions; Human robot interac- tion and Human Robot Collaboration shows the robot trying to grasp targets while avoiding human-operated tools, which aims to improve the human-robot collabora- tion.
DUBINI, GABRIELE ANGELO
FERRANTE, SIMONA
DE MOMI, ELENA
5-dic-2024
Autonomous object manipulation for robot assisted minimally invasive surgery
L’obiettivo generale di questa tesi è sviluppare un framework di controllo intelligente, sicuro e robusto basato sul sistema chirurgico robotico da Vinci (Intuitive Surgical, CA, USA), con lo scopo di avanzare il livello di automazione nella chirurgia assistita da robot. Il framework prende pienamente in considerazione i vincoli di sicurezza all’interno dei compiti chirurgici e la collaborazione uomo-robot. In particolare, la ricerca si è concentrata sui seguenti temi: * Gemello digitale del sistema chirurgico robotico da Vinci: Il gemello digitale è progettato per replicare accuratamente la cinematica, la dinamica e gli aspetti di controllo del sistema chirurgico robotico da Vinci. Integra inoltre un modello dettagliato dell’ambiente chirurgico, consentendo la simu- lazione di vari scenari chirurgici. Il gemello digitale può fornire un ambiente virtuale per l’addestramento chirurgico e per l’apprendimento automatico, come l’Apprendimento per Rinforzo (RL). * Gemello digitale con VFs per l’addestramento chirurgico: L’addestramento chirurgico è una componente critica nella preparazione dei chirurghi per le pro- cedure chirurgiche moderne, in particolare nella chirurgia assistita da robot. Il gemello digitale descritto sopra offre un ambiente di addestramento sicuro, con- trollato e altamente realistico, ulteriormente migliorato dall’incorporazione di vincoli di sicurezza mediante i VFs. La ricerca precedente sui VFs si è concen- trata principalmente sulla metodologia di calcolo delle forze, mentre i percorsi di guida dei VFs dovevano essere progettati manualmente. Utilizziamo tecniche di RL per generare automaticamente percorsi di guida dei VFs in scenari dinam- ici. Il VF basato su RL fornisce assistenza in tempo reale ai tirocinanti nella navigazione del robot chirurgico verso l’anatomia target, evitando collisioni con tessuti a rischio. * RL sicuro per l’automazione dei compiti: I metodi di RL hanno di- mostrato prestazioni considerevoli nel controllo autonomo dei robot in ambi- enti complessi. Tuttavia, gli algoritmi di RL esistenti per i robot chirurgici non considerano requisiti di sicurezza. È stato proposto il Safe Experience Re- shaping (SER) per integrare i vincoli di sicurezza nella formulazione del RL. Questo può essere integrato in qualsiasi algoritmo di RL offline. In primo lu- ogo, il metodo identifica e apprende la geometria dei vincoli. In secondo lu- ogo, un’esperienza sicura viene ottenuta proiettando un’azione non sicura nello spazio tangente della geometria appresa, il che significa che l’azione si trova nello spazio sicuro. Successivamente, le esperienze sicure raccolte vengono utilizzate per l’addestramento di politiche sicure.
File allegati
File Dimensione Formato  
thesis_ke_fan.pdf

accessibile in internet per tutti

Dimensione 15.42 MB
Formato Adobe PDF
15.42 MB Adobe PDF Visualizza/Apri

I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10589/232112