Test procedures involving aeronautical cockpits are generally interpreted and executed by human operators, making the process repetitive and time-consuming. Robotic automation therefore represents a possible solution to overcome these limitations. However, since technical instructions are written in natural language, they must be converted into a representation that can be understood by the robotic system. The objective of this thesis is the development and validation of a pipeline capable of automatically transforming technical procedures written in natural language into structured robotic commands. The proposed system processes technical instructions through a sequence of modules. First, complex sentences containing multiple actions are decomposed into elementary steps through a strategy in which a Large Language Model (LLM) is used as support to derive stable deterministic splitting rules. Each step is then associated with the most suitable robotic primitive among those available in the database. Each primitive requires specific operational parameters, which are extracted through a semantic parsing approach based on the GLiNER model or through the use of an LLM. Finally, the system can generate two types of output: a structured JSON message to be sent to an external orchestrator, or an automatically generated Behavior Tree in XML format. A graphical user interface was also developed to support non-specialized users in using the system. The validation confirmed the effectiveness of the proposed approach. The final splitting strategy achieved an accuracy of 98.43%, primitive identification reached an accuracy of 98.00%, and LLM-based parameter extraction achieved 100.00% on the considered dataset, outperforming the GLiNER-based solution. In conclusion, this work demonstrates that the developed modules are able to ensure reliability and controllability in the process of transforming technical procedures, resulting in a robust and modular solution for converting aeronautical test procedures into structured robotic commands.
Le procedure di test sui cockpit aeronautici sono generalmente interpretate ed eseguite da operatori umani, rendendo il processo ripetitivo e dispendioso in termini di tempo. L’automazione robotica rappresenta quindi una possibile soluzione per superare questi limiti. Tuttavia, poiché le istruzioni tecniche sono scritte in linguaggio naturale, esse devono essere convertite in una rappresentazione comprensibile dal sistema robotico. L’obiettivo di questa tesi è lo sviluppo e la validazione di una pipeline capace di trasformare automaticamente procedure tecniche scritte in linguaggio naturale in comandi robotici strutturati. Il sistema proposto elabora le istruzioni tecniche attraverso una sequenza di moduli. In primo luogo, le frasi complesse contenenti più azioni vengono scomposte in step elementari tramite una strategia in cui un Large Language Model (LLM) è utilizzato come supporto per derivare regole deterministiche di splitting stabili. Ogni step viene poi associato alla primitiva robotica più adatta tra quelle disponibili nel database. Ciascuna primitiva richiede specifici parametri operativi, che vengono estratti tramite un approccio di parsing semantico basato sul modello GLiNER oppure attraverso l’utilizzo di un LLM. Infine, il sistema può generare due tipi di output: un messaggio JSON strutturato da inviare a un orchestratore esterno oppure un Behavior Tree in formato XML generato automaticamente. È stata inoltre sviluppata un’interfaccia grafica per supportare utenti non specializzati nell’utilizzo del sistema. La validazione ha confermato l’efficacia dell’approccio proposto. La strategia finale di splitting ha raggiunto un’accuratezza del 98,43%, l’identificazione della primitiva ha ottenuto un’accuratezza del 98,00% e l’estrazione dei parametri tramite LLM ha raggiunto il 100,00% sul dataset considerato, superando la soluzione basata su GLiNER. In conclusione, questo lavoro dimostra che i moduli sviluppati sono in grado di garantire affidabilità e controllabilità nel processo di trasformazione delle procedure tecniche, dando vita a una soluzione robusta e modulare per convertire procedure di test aeronautiche in comandi robotici strutturati.
Automatic generation of behavior trees from natural language instructions for aeronautical cockpit test automation
BONINI, NICOLÒ
2025/2026
Abstract
Test procedures involving aeronautical cockpits are generally interpreted and executed by human operators, making the process repetitive and time-consuming. Robotic automation therefore represents a possible solution to overcome these limitations. However, since technical instructions are written in natural language, they must be converted into a representation that can be understood by the robotic system. The objective of this thesis is the development and validation of a pipeline capable of automatically transforming technical procedures written in natural language into structured robotic commands. The proposed system processes technical instructions through a sequence of modules. First, complex sentences containing multiple actions are decomposed into elementary steps through a strategy in which a Large Language Model (LLM) is used as support to derive stable deterministic splitting rules. Each step is then associated with the most suitable robotic primitive among those available in the database. Each primitive requires specific operational parameters, which are extracted through a semantic parsing approach based on the GLiNER model or through the use of an LLM. Finally, the system can generate two types of output: a structured JSON message to be sent to an external orchestrator, or an automatically generated Behavior Tree in XML format. A graphical user interface was also developed to support non-specialized users in using the system. The validation confirmed the effectiveness of the proposed approach. The final splitting strategy achieved an accuracy of 98.43%, primitive identification reached an accuracy of 98.00%, and LLM-based parameter extraction achieved 100.00% on the considered dataset, outperforming the GLiNER-based solution. In conclusion, this work demonstrates that the developed modules are able to ensure reliability and controllability in the process of transforming technical procedures, resulting in a robust and modular solution for converting aeronautical test procedures into structured robotic commands.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026_07_Bonini_Tesi_01.pdf
non accessibile
Descrizione: Testo della tesi
Dimensione
8.8 MB
Formato
Adobe PDF
|
8.8 MB | Adobe PDF | Visualizza/Apri |
|
2026_07_Bonini_Executive_Summary_02.pdf
non accessibile
Descrizione: Executive Summary
Dimensione
2.09 MB
Formato
Adobe PDF
|
2.09 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/261183