About 7.5% of children worldwide are diagnosed with language impairments. The traditional linguistic therapeutic interventions usually lack attractiveness and engagement, highlight children’s weaknesses rather than their strengths, and focus on stimulating linguistic skills instead of spurring them alongside the extra-linguistic ones, e.g., cognitive aspect and social skills. Recent studies have pointed out the potential of adopting innovative technologies into speech-language therapy to overcome those limitations. Specifically, conversational agents have emerged as promising tools for children with language impairments. State of the art in conversational technologies for this population has several open challenges, such as (1) providing empirical evidence of the benefits of using conversational agents in therapeutic interventions; (2) understanding how user’s performance and perception of conversational interactions are affected by different design features, particularly those related to extit{embodiment}, i.e., the tangible or visible form for the agent’s representation; (3) grounding therapy-oriented conversational agents on psycholinguistic theories and approaches underlying the treatment of children with language impairments; and (4) making these agents scalable and portable to multiple platforms. This thesis explores the following research questions to address the above gaps: “Can conversational agents help children with language impairments to improve their linguistic capabilities? What is the most suitable form of embodiment for these agents? What are psycholinguistic methods appropriate to inform their design? How can the development process be improved to facilitate reuse, portability, and scalability?”. This Ph.D. thesis sought to answer the research questions as follows. First, we ran four empirical studies involving a total of 47 children with and without language impairments who interacted with a conversational agent represented by different forms of embodiment. The results suggested that the conversational agent reified by a socially assistive robot (SAR) achieved the best results in enhancing linguistic skills, improving social interaction in the child-agent-therapist triad, and boosting verbal and non-verbal communication. Second, in collaboration with researchers in speech-language therapy, we identified a psycholinguistics method that we translated into a set of design patterns for a therapy-oriented conversational agent. Such patterns, named Activity Patterns, were defined at multiple levels of abstraction - from task flow and interaction design to implementation architecture - and provide reusable solutions to facilitate the development process. Their usability and expressive power were validated in an exploratory study that involved four computer scientists and five psycholinguistics professionals. Finally, we designed and implemented a modular, composable, pattern-oriented, and robot-agnostic ROS-based framework for SAR-based conversational interactions, namely HARMONI (Human and Robot Modular Open Interaction). Conversational interactions implemented with HARMONI exploit a variety of behavioral patterns (including the ones mentioned above), are portable on different robotic platforms and can be easily extended/modified to address the requirements of different therapeutic scenarios. A final study involved five novice robotic developers in testing the usability of HARMONI, leading to improvements of the framework based on users’ feedback.
A circa il 7,5% dei bambini in tutto il mondo è diagnosticato un disturbo del linguaggio. Gli attuali interventi terapeutici linguistici risultano poco attrattivi e coinvolgenti per i bambini, sottolineano le loro debolezze piuttosto che i loro punti di forza e si concentrano sulla stimolazione delle abilità linguistiche senza tener conto di quelle extralinguistiche (ad es. aspetto cognitivo, abilità sociali). Recenti studi hanno evidenziato il potenziale dell'adozione di tecnologie innovative nella logopedia per superare tali limiti. In particolare, gli agenti conversazionali sono emersi come strumenti promettenti per i bambini con disturbi del linguaggio. In letteratura ci sono ancora molti problemi non risolti riguardo l'utilizzo di tecnologie conversazionali per questa popolazione, come: (1) fornire prove empiriche dei benefici dell'utilizzo di agenti conversazionali negli interventi terapeutici; (2) comprendere come le prestazioni e la percezione del bambino durante l' interazione conversazionale siano influenzate da diversi fattori di design, in particolare quelli relative all' embodiment, ovvero la rappresentazione o espressione dell'agente in una forma tangibile o visibile; (3) progettare agenti conversazionali per la terapia che siano basati su teorie e approcci psicolinguistici per il trattamento dei bambini con disabilità linguistiche; e (4) rendere questi agenti scalabili e portabili su più piattaforme. Questa tesi esplora le seguenti domande di ricerca per risolvere i sopracitati problemi: “Gli agenti conversazionali possono aiutare i bambini con disabilità del linguaggio a migliorare le loro capacità linguistiche? Qual è la forma di embodiment più adatta per questi agenti? Quali sono i metodi psicolinguistici appropriati per informare il loro design? Come si può migliorare il processo di sviluppo per facilitare il riutilizzo, la portabilità e la scalabilità?". Questa tesi ha risposto alle domande di ricerca come segue. In primo luogo, abbiamo condotto quattro studi empirici che hanno coinvolto un totale di 47 bambini con e senza disabilità linguistiche che hanno interagito con un agente conversazionale rappresentato da diverse forme di embodiment. I nostri risultati hanno suggerito che l'agente conversazionale rappresentato da un robot sociale assistivo (SAR) ha ottenuto i migliori risultati nel potenziamento delle abilità linguistiche, nel miglioramento dell'interazione sociale nella triade bambino-agente-terapeuta e nel potenziamento della comunicazione verbale e non verbale. Come secondo contributo, in collaborazione con ricercatori in logopedia, abbiamo identificato un metodo psicolinguistico che abbiamo tradotto in una serie di design patterns per un agente conversazionale terapeutico. Tali modelli, denominati Activity Patterns, sono stati definiti a più livelli di astrazione - dal flusso delle attività e dal design dell'interazione all'architettura di implementazione - e forniscono soluzioni riutilizzabili per facilitare il processo di sviluppo. La loro usabilità e potenza espressiva sono state convalidate in uno studio esplorativo che ha coinvolto quattro informatici e cinque professionisti della psicolinguistica. Infine, abbiamo progettato e implementato un framework basato su ROS che è modulare, componibile, basato sui patterns e indipendente dalla piattaforma robotica per agenti conversazionali robotici, chiamato HARMONI (Human and Robot Modular Open Interaction). Gli agenti conversazionali robotici implementati con HARMONI sfruttano una varietà di modelli comportamentali (compresi quelli sopra menzionati), sono portatili su diverse piattaforme robotiche e possono essere facilmente estesi/modificati per soddisfare le esigenze di diversi scenari terapeutici. Uno studio finale ha coinvolto cinque sviluppatori robotici alle prime armi per testare l'usabilità di HARMONI, portando a miglioramenti del framework in base al feedback degli utenti.
Conversational agents for children with language impairments: design and technology
Spitale, Micol
2021/2022
Abstract
About 7.5% of children worldwide are diagnosed with language impairments. The traditional linguistic therapeutic interventions usually lack attractiveness and engagement, highlight children’s weaknesses rather than their strengths, and focus on stimulating linguistic skills instead of spurring them alongside the extra-linguistic ones, e.g., cognitive aspect and social skills. Recent studies have pointed out the potential of adopting innovative technologies into speech-language therapy to overcome those limitations. Specifically, conversational agents have emerged as promising tools for children with language impairments. State of the art in conversational technologies for this population has several open challenges, such as (1) providing empirical evidence of the benefits of using conversational agents in therapeutic interventions; (2) understanding how user’s performance and perception of conversational interactions are affected by different design features, particularly those related to extit{embodiment}, i.e., the tangible or visible form for the agent’s representation; (3) grounding therapy-oriented conversational agents on psycholinguistic theories and approaches underlying the treatment of children with language impairments; and (4) making these agents scalable and portable to multiple platforms. This thesis explores the following research questions to address the above gaps: “Can conversational agents help children with language impairments to improve their linguistic capabilities? What is the most suitable form of embodiment for these agents? What are psycholinguistic methods appropriate to inform their design? How can the development process be improved to facilitate reuse, portability, and scalability?”. This Ph.D. thesis sought to answer the research questions as follows. First, we ran four empirical studies involving a total of 47 children with and without language impairments who interacted with a conversational agent represented by different forms of embodiment. The results suggested that the conversational agent reified by a socially assistive robot (SAR) achieved the best results in enhancing linguistic skills, improving social interaction in the child-agent-therapist triad, and boosting verbal and non-verbal communication. Second, in collaboration with researchers in speech-language therapy, we identified a psycholinguistics method that we translated into a set of design patterns for a therapy-oriented conversational agent. Such patterns, named Activity Patterns, were defined at multiple levels of abstraction - from task flow and interaction design to implementation architecture - and provide reusable solutions to facilitate the development process. Their usability and expressive power were validated in an exploratory study that involved four computer scientists and five psycholinguistics professionals. Finally, we designed and implemented a modular, composable, pattern-oriented, and robot-agnostic ROS-based framework for SAR-based conversational interactions, namely HARMONI (Human and Robot Modular Open Interaction). Conversational interactions implemented with HARMONI exploit a variety of behavioral patterns (including the ones mentioned above), are portable on different robotic platforms and can be easily extended/modified to address the requirements of different therapeutic scenarios. A final study involved five novice robotic developers in testing the usability of HARMONI, leading to improvements of the framework based on users’ feedback.| File | Dimensione | Formato | |
|---|---|---|---|
|
Micol_Spitale_Thesis.pdf
accessibile in internet solo dagli utenti autorizzati
Dimensione
29.82 MB
Formato
Adobe PDF
|
29.82 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/187658