Autism spectrum disorder (ASD) has become the fastest growing disability in the United States with current prevalence rates estimated at as many as 1 in 88 children (CDC, 2013) and an urgent need for focused and ongoing intervention. Each child with ASD behaves differently, yet the disorder is characterized by a triad of symptoms related to lack of social interaction, deficits in the acquisition and expression of language, and repetitive patterns of behavior often accompanied by sensorimotor impairments. While cutting-edge learning technologies have transformed communication, connectivity and sensitivity for neurotypicals (non-ASD), the ASD community has been largely left behind. The general context of this research is the development of innovative interactive technologies that can be integrated with both general school classes or personalized therapeutic activities and can be autonomously used by teachers/therapists to promote, through engagement,: social interaction, communication capabilities and motor skills. This work presents a technological ecosystem centered on the brand new notion of Human To Human Computer Interaction, HHCI, where technology is no more the primary actor (as suggested by HCI) but it is just, simply, a support between the human collaboration. This thesis contributes to the current state of the art in the field, by developing an innovative tool, called S’COOL, that for the first time combines motion-based touchless interaction and storytelling approach for autistic children. Storytelling can promote emotional engagement with compelling plot and characters, vivid imagery, and other elements such as repetition of words and phrases. Adding interactive elements and body motion recognition to storytelling creates a more complex activity that involves communication, motor, and problem solving skills while leveraging social connection. The system is designed to run on a screen with the use of the Kinect, a special device which combines an RGB camera, an infrared projector, and a microphone to perform gesture and voice command recognition. The potential of motion based interaction for learning is grounded on theoretical approaches that recognize the relationship between physical activity and cognitive processes, and are supported by a growing body of evidence from psychology and neurobiology. Our initial body of research started from the Lionheart School in Alpharetta, Atlanta, an excellence in the ASD panorama. A participatory design approach with a strong involvement of therapists, psychologists, educators, and children as users has been carried out with the school. So far, this interaction paradigm has been mainly applied to 3 classes of different functional levels and involved 20 children aged from 5 to 13. The empirical study is still being carried through video observation, focus group, semi-structured interviews and log files. Overall children reacted positively to the introduction of the new system and nowadays it is constantly integrated into Lionheart’s activities. Improvements in movement, involvement and engagement with peers were noticed. Furthermore the Occupational Therapist perceived an “unbelievable” progress in motor movements in some kids while educators detected not only an enhancement of socialization but also communication and shared strategy planning. The project is a continuing work, initial development and testing are promising. The ability to easily create or modify learning material around a child's special interest is seen as a major advantage to engaging individuals in social centric activities. Further work is in progress to test the platform’s ability to automatically collect and analyze data and to create a modular architecture that will make it reusable and adaptable for different developers’ and kids’ needs.
Il disturbo dello spettro autistico (DSA) è diventato la disabilità più crescente negli Stati Uniti con tassi di incidenza pari a 1 su 88 nascite (CDC, 2013) e un bisogno urgente di interventi continuativi e focalizzati. Ogni bambino autistico si comporta in modo diverso, eppure il disturbo è caratterizzato da una triade di sintomi: mancanza di interazione e legami sociali, deficit nell’acquisizione e nell’espressione del linguaggio, e pattern ripetitivi spesso accompagnati da menomazioni motorie. Mentre le ultime tecniche di apprendimento hanno trasformato la comunicazione, connettività e sensitività per i neuro-tipici (non-DSA), la comunità autistica è stata largamente abbandonata. Il contesto generale di questa ricerca è lo sviluppo di tecnologie interattive che possano essere integrate sia con attività scolastiche sia autonomamente utilizzate da insegnanti/terapisti per promuovere, attraverso il divertimento, interazione sociale, capacità comunicative e abilità motorie. Questo lavoro presenta un ecosistema tecnologico centrato sulla nuova nozione di Human To Human Computer Interaction, HHCI, dove la tecnologia non è più l’attore primario (come avviene in HCI) ma è solo e semplicemente un supporto tra la collaborazione umana. Questa tesi contribuisce inoltre allo stato dell’arte corrente nel campo, sviluppando uno sistema innovativo, chiamato S’COOL, che per la prima volta combina interazioni basate sul movimento e un approccio di storytelling per bimbi autistici. Lo storytelling può promuovere coinvolgimento emotivo offrendo trama avvincente, immagini condivise e ripetizione di parole. Aggiungendo elementi interattivi e riconoscimento dei gesti allo storytelling tradizionale si crea una attività complessa che involve comunicazione, motricità, e abilità di problem solving facendo leva sulle connessioni sociali. Il sistema è costruito per essere eseguito su uno schermo con l’utilizzo della Kinect che combina una videocamera RGB, un proiettore infrarossi e un microfono per riconoscere gesti del corpo e comandi vocali. Il potenziale dell’interazione basata sui movimenti per poter imparare si fonda su approcci teorici che riconoscono la relazione positiva tra attività fisica e processi cognitivi ed è supportata da un’evidenza psicologica e neurobiologica. La ricerca è partita dalla scuola Lionheart ad Alpharetta, Atlanta, un’eccellenza nel panorama autistico. È stata effettuata una progettazione partecipativa con grande coinvolgimento di terapisti, psicologhi, educatori, e bambini come fruitori di storie. Finora, questo paradigma interattivo è stato applicato a 3 classi di differenti livelli di funzionamento e ha coinvolto 20 bambini di età compresa tra i 5 e i 13 anni. Lo studio empirico è ancora in fase di elaborazione attraverso osservazioni video, focus group, interviste semi-strutturate e file di log. Complessivamente si può affermare che i bambini hanno reagito positivamente all’introduzione del sistema che è diventato in poco tempo parte integrante delle attività della Lionheart. In particolare sono stati notati progressi nel movimento e coinvolgimento tra i compagni. Oltretutto l’ergoterapeuta ha percepito un progresso “incredibile” in alcuni bambini mentre gli educatori hanno confermato un miglioramento della socializzazione e della comunicazione con presenza di pianificazione strategica condivisa mai avvenuta prima. Il progetto è in continua espansione: sviluppi e test iniziali sono promettenti. L’abilità di creare o modificare il materiale didattico personalizzandolo in base alle abilità e agli interessi del bambino è visto come un vantaggio enorme per poterlo coinvolgere nelle attività sociali. Ulteriori sviluppi sono in progresso per testare la capacità della piattaforma di raccogliere e analizzare dati in maniera automatica e per creare un’architettura modulare che permetterà di avere un sistema riusabile e adattabile alle esigenze di sviluppatori e in particolare dei bambini.
Motion based interactive storytelling for children with ASD
GELSOMINI, MIRKO
2013/2014
Abstract
Autism spectrum disorder (ASD) has become the fastest growing disability in the United States with current prevalence rates estimated at as many as 1 in 88 children (CDC, 2013) and an urgent need for focused and ongoing intervention. Each child with ASD behaves differently, yet the disorder is characterized by a triad of symptoms related to lack of social interaction, deficits in the acquisition and expression of language, and repetitive patterns of behavior often accompanied by sensorimotor impairments. While cutting-edge learning technologies have transformed communication, connectivity and sensitivity for neurotypicals (non-ASD), the ASD community has been largely left behind. The general context of this research is the development of innovative interactive technologies that can be integrated with both general school classes or personalized therapeutic activities and can be autonomously used by teachers/therapists to promote, through engagement,: social interaction, communication capabilities and motor skills. This work presents a technological ecosystem centered on the brand new notion of Human To Human Computer Interaction, HHCI, where technology is no more the primary actor (as suggested by HCI) but it is just, simply, a support between the human collaboration. This thesis contributes to the current state of the art in the field, by developing an innovative tool, called S’COOL, that for the first time combines motion-based touchless interaction and storytelling approach for autistic children. Storytelling can promote emotional engagement with compelling plot and characters, vivid imagery, and other elements such as repetition of words and phrases. Adding interactive elements and body motion recognition to storytelling creates a more complex activity that involves communication, motor, and problem solving skills while leveraging social connection. The system is designed to run on a screen with the use of the Kinect, a special device which combines an RGB camera, an infrared projector, and a microphone to perform gesture and voice command recognition. The potential of motion based interaction for learning is grounded on theoretical approaches that recognize the relationship between physical activity and cognitive processes, and are supported by a growing body of evidence from psychology and neurobiology. Our initial body of research started from the Lionheart School in Alpharetta, Atlanta, an excellence in the ASD panorama. A participatory design approach with a strong involvement of therapists, psychologists, educators, and children as users has been carried out with the school. So far, this interaction paradigm has been mainly applied to 3 classes of different functional levels and involved 20 children aged from 5 to 13. The empirical study is still being carried through video observation, focus group, semi-structured interviews and log files. Overall children reacted positively to the introduction of the new system and nowadays it is constantly integrated into Lionheart’s activities. Improvements in movement, involvement and engagement with peers were noticed. Furthermore the Occupational Therapist perceived an “unbelievable” progress in motor movements in some kids while educators detected not only an enhancement of socialization but also communication and shared strategy planning. The project is a continuing work, initial development and testing are promising. The ability to easily create or modify learning material around a child's special interest is seen as a major advantage to engaging individuals in social centric activities. Further work is in progress to test the platform’s ability to automatically collect and analyze data and to create a modular architecture that will make it reusable and adaptable for different developers’ and kids’ needs.| File | Dimensione | Formato | |
|---|---|---|---|
|
tesi.pdf
accessibile in internet per tutti
Descrizione: Thesis text
Dimensione
15.15 MB
Formato
Adobe PDF
|
15.15 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/86126