Robotics in autism spectrum disorder (ASD) already has a two-decade history. Some individuals with ASD are particularly drawn to robots due to their predictability and repeatability, showing improvements in social, communication and motor capabilities following robot-assisted therapies. However, the simplified external characteristics of robots often extend to their control systems, limiting their applicability in a disorder characterized by its heterogeneity. People with autism exhibit highly varied behaviours across different contexts, days, or even within the same day. Therefore, adaptable control systems are required, but they rely on quantitative measures, which are currently lacking in the autism field. This lack of quantitative measures has also hindered the ability to consolidate evidence from multiple clinical studies, as each relies on its own subjective measures. Consequently, robots are still far from being integrated into standard clinical practice. The main goal of this thesis was to address this gap by developing and testing new protocols and quantitative measures for clinical use. Three sub-goals were established:(i) implementation of protocols that consider the heterogeneity of the disorder; (ii) construction of quantitative measures for evaluation intra and inter-sessions; (iii) design of experimental studies to provide evidence of the system's impact. The contributions of this thesis concern both in new protocols and new quantitative measures. All developments were carried out in close collaboration with clinical institutions specialized in Autism in both Portugal and Italy, where this research took place. Four distinct robotic protocols were developed: one for pre- and post-treatment evaluation (scale protocol); another centred on the therapeutic improvement of the gestural and mirroring skills, including multiple levels to consider the heterogeneity of ASD (hierarchical protocol); and two additional protocols derived from tests conducted in Portugal and Italy using the hierarchical protocol. Interestingly, both of these adaptations evolved into single-level protocols incorporating holistic games to facilitate gesture presentation, making integration into clinical practice easier. In total, 33 children and 11 therapists have been involved in the test of the hierarchical protocol, a relatively large sample size in ASD research. Cameras were used as sensors for quantitative measurement, as children with ASD often prefer non-intrusive devices due to their sensitivity to touch. We developed a new mirroring measure that considered the duration and latency at start of exercises captured differences between neurotypical and ASD children. Additionally, a gesture recognition system was created by combining a kinetic parameter - used to determine the beginning and end of a gesture- with a ResNet architecture, for the classification of the gesture. The integration of this system into one of the protocols led to increased engagement in the child with ASD that tested it. Given the significance of attention in ASD, a neural network was developed for automatic attention classification following the assessment of the scale protocol. Furthermore, an attention biomarker was introduce for use during the therapy sessions, measuring fixation time on different targets of interest using geometrical assumptions. Not only did this system outperform existing methods, but it also revealed a correlation between the obtained fixation time and one of the clinical scales used in our study - an important step toward establishing evidence for the potential impact of robotic therapies in the ASD field. Future work will focus on integrating these metrics in real-time into our robotic protocols to create a personalized system, enhancing engagement in long-term therapy sessions. In addition, we aim to increase parental involvement to facilitate the extension of these therapies into home settings.
La robotica nel disturbo dello spettro autistico (ASD) ha già una storia di due decenni. Alcuni individui con ASD sono Interagiscono facilmente con i robot a causa della loro prevedibilità e ripetibilità, mostrando miglioramenti nelle capacità sociali e motorie dopo terapie assistite da robot. Tuttavia, le caratteristiche esterne semplificate dei robot spesso si estendono ai loro sistemi di controllo, limitando la loro applicabilità in un disturbo caratterizzato dalla sua eterogeneità. Le persone con autismo presentano comportamenti altamente variabili, pertanto, sono necessari sistemi di controllo adattativi. Purtroppo, questi si basano su misure quantitative, che attualmente mancano in questo campo. Questa mancanza ha anche ostacolato la possibilità di consolidare evidenze da diversi studi clinici, poiché ciascuno si basa sulle proprie misure soggettive. Di conseguenza, i robot non sono ancora integrati nella pratica clinica. L'obiettivo principale di questa tesi era colmare questa lacuna sviluppando e testando nuovi protocolli e misure quantitative per l'uso di robot nella terapia dei disturbi nello spettro dell’autismo. Sono stati stabiliti tre sotto-obiettivi: (i) implementazione di protocolli che considerino l'eterogeneità del disturbo; (ii) costruzione di misure quantitative per la valutazione intra e inter-sessioni; (iii) progettazione di studi sperimentali per fornire prove dell'impatto del sistema. Tutti gli sviluppi sono stati realizzati in stretta collaborazione con istituzioni cliniche specializzate in autismo sia in Portogallo che in Italia, dove è stata condotta questa ricerca. Sono stati sviluppati quattro distinti protocolli robotici: uno per la valutazione pre e post-trattamento (protocollo della scala); un altro centrato sul miglioramento terapeutico delle abilità gestuali e di imitazione, con più livelli per considerare l'eterogeneità dell'ASD (protocollo gerarchico); e due protocolli aggiuntivi derivati da test condotti in Portogallo e Italia utilizzando il protocollo gerarchico. Entrambi questi sviluppi si sono evoluti in protocolli a singolo livello che incorporano giochi olistici per facilitare la presentazione dei gesti, rendendo più facile l'integrazione nella pratica clinica. In totale, 33 bambini e 11 terapisti sono stati coinvolti nel test del protocollo gerarchico, un campione relativamente ampio nella ricerca sull'ASD. Per la misurazione quantitativa, sono state utilizzate telecamere come sensori poiché i bambini con ASD spesso preferiscono dispositivi non invasivi a causa della loro sensibilità al tatto. Abbiamo sviluppato una nuova misura di mirroring che considerava la durata e la latenza all'inizio degli esercizi, rilevando differenze tra bambini neurotipici e bambini con ASD. Inoltre, è stato creato un sistema di riconoscimento dei gesti combinando un parametro cinetico - utilizzato per determinare l'inizio e la fine di un gesto - con un'architettura ResNet, per la classificazione del gesto. L'integrazione di questo sistema in uno dei protocolli ha portato a un maggiore coinvolgimento nel bambino con ASD che lo ha testato. Data l'importanza dell'attenzione nell'ASD, è stata sviluppata una rete neurale per la classificazione automatica dell'attenzione dopo la valutazione del protocollo di scala. Inoltre, è stato introdotto un biomarcatore dell'attenzione per l'uso durante le sessioni di terapia, misurando il tempo di fissazione su diversi obiettivi di interesse utilizzando assunzioni geometriche. Non solo questo sistema ha superato i metodi esistenti, ma ha anche rivelato una correlazione tra il tempo di fissazione ottenuto e una delle scale cliniche utilizzate nel nostro studio - un passo importante verso la creazione di prove per il potenziale impatto delle terapie robotiche nel campo dell'ASD. Il lavoro futuro si concentrerà sull'integrazione di queste metriche in tempo reale nei nostri protocolli robotici per creare un sistema personalizzato, migliorando il coinvolgimento nelle sessioni di terapia a lungo termine. Inoltre, intendiamo aumentare il coinvolgimento dei genitori per facilitare l'estensione di queste terapie in ambito domestico.
AnalyseandAct: automatic metrics and innovative protocols in robotics for children with ASD
ESPINOSA FORTES FERREIRA DOS SANTOS, LAURA JOANA
2024/2025
Abstract
Robotics in autism spectrum disorder (ASD) already has a two-decade history. Some individuals with ASD are particularly drawn to robots due to their predictability and repeatability, showing improvements in social, communication and motor capabilities following robot-assisted therapies. However, the simplified external characteristics of robots often extend to their control systems, limiting their applicability in a disorder characterized by its heterogeneity. People with autism exhibit highly varied behaviours across different contexts, days, or even within the same day. Therefore, adaptable control systems are required, but they rely on quantitative measures, which are currently lacking in the autism field. This lack of quantitative measures has also hindered the ability to consolidate evidence from multiple clinical studies, as each relies on its own subjective measures. Consequently, robots are still far from being integrated into standard clinical practice. The main goal of this thesis was to address this gap by developing and testing new protocols and quantitative measures for clinical use. Three sub-goals were established:(i) implementation of protocols that consider the heterogeneity of the disorder; (ii) construction of quantitative measures for evaluation intra and inter-sessions; (iii) design of experimental studies to provide evidence of the system's impact. The contributions of this thesis concern both in new protocols and new quantitative measures. All developments were carried out in close collaboration with clinical institutions specialized in Autism in both Portugal and Italy, where this research took place. Four distinct robotic protocols were developed: one for pre- and post-treatment evaluation (scale protocol); another centred on the therapeutic improvement of the gestural and mirroring skills, including multiple levels to consider the heterogeneity of ASD (hierarchical protocol); and two additional protocols derived from tests conducted in Portugal and Italy using the hierarchical protocol. Interestingly, both of these adaptations evolved into single-level protocols incorporating holistic games to facilitate gesture presentation, making integration into clinical practice easier. In total, 33 children and 11 therapists have been involved in the test of the hierarchical protocol, a relatively large sample size in ASD research. Cameras were used as sensors for quantitative measurement, as children with ASD often prefer non-intrusive devices due to their sensitivity to touch. We developed a new mirroring measure that considered the duration and latency at start of exercises captured differences between neurotypical and ASD children. Additionally, a gesture recognition system was created by combining a kinetic parameter - used to determine the beginning and end of a gesture- with a ResNet architecture, for the classification of the gesture. The integration of this system into one of the protocols led to increased engagement in the child with ASD that tested it. Given the significance of attention in ASD, a neural network was developed for automatic attention classification following the assessment of the scale protocol. Furthermore, an attention biomarker was introduce for use during the therapy sessions, measuring fixation time on different targets of interest using geometrical assumptions. Not only did this system outperform existing methods, but it also revealed a correlation between the obtained fixation time and one of the clinical scales used in our study - an important step toward establishing evidence for the potential impact of robotic therapies in the ASD field. Future work will focus on integrating these metrics in real-time into our robotic protocols to create a personalized system, enhancing engagement in long-term therapy sessions. In addition, we aim to increase parental involvement to facilitate the extension of these therapies into home settings.| File | Dimensione | Formato | |
|---|---|---|---|
|
Thesis_PhD_Santos.pdf
solo utenti autorizzati a partire dal 20/06/2026
Dimensione
32.23 MB
Formato
Adobe PDF
|
32.23 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/239597