Augmented Reality (AR) technologies are transforming industry by merging digital and physical information, yet their use in non-inertial environments, such as moving vehicles, has not been fully explored. This thesis develops methods and algorithms to enable reliable AR in moving vehicles, particularly on Head-Mounted Displays (HMDs). The research targets two main goals: estimating the six-degree-of-freedom (6-DoF) pose of an HMD within a moving vehicle, and localizing the vehicle with accuracy sufficient for geo-referenced AR visualization. For HMD pose estimation, a visual-inertial fusion architecture is introduced to compute motion relative to the cockpit, compensating for vehicle dynamics. Regarding the visual component, two novel computer-vision algorithms are presented, leveraging both monocular and stereo cameras installed in modern HMDs. For vehicle localization, two pipelines are introduced: one fuses GPS-RTK and attitude data for precise on-track mapping and positioning, while the other uses stereo vision for GPS-denied environments such as fields or vineyards. A reprojection-based validation method evaluates localization accuracy without ground truth. To demonstrate the validity of the approach in heterogeneous scenarios, sample AR applications are presented, leveraging the proposed algorithms. These include a coaching tool for on-track experiences, a smart-farming system to improve the productivity of in-field operations, and an automatic dataset generation pipeline for neural network training that exploits AR for generating and labeling synthetic images. All the work proposed in this thesis has been tested in collaboration with HMDrive s.r.l., and most of the algorithms have been integrated into their AR software stack, demonstrating the industrial impact of this work.
La Realtà Aumentata (AR) sta trasformando il mondo industriale integrando la visualizzazione di informazioni virtuali e fisiche. Tuttavia, il suo utilizzo in ambienti non inerziali, come i veicoli in movimento, è stato finora poco esplorato. Questa tesi sviluppa metodi e algoritmi per rendere possibile l'impiego di tali tecnologie anche in contesti dinamici, in particolare su visori Head-Mounted Display (HMD). La ricerca si concentra su due obiettivi principali: stimare la posa di un HMD all'interno di un veicolo in movimento e localizzare il veicolo stesso con un'accuratezza sufficiente a consentire la visualizzazione di elementi virtuali georeferenziati. Per la stima della posa dell'HMD viene proposto un algoritmo di fusione visivo-inerziale capace di compensare le dinamiche del veicolo. Sul piano visivo, vengono presentati due nuovi algoritmi che sfruttano le telecamere integrate nei moderni HMD. Per la localizzazione del veicolo sono state sviluppate due procedure: la prima integra dati GPS e una mappa dell'ambiente per stimare la posizione in pista, mentre la seconda utilizza telecamere per localizzare il veicolo in ambienti privi di segnale GPS, come campi o vigneti. È stato inoltre introdotto un metodo di validazione basato sulla riproiezione di feature AR, utile per valutare l'accuratezza in assenza di ground truth. Per dimostrare la validità degli approcci proposti, vengono presentate alcune applicazioni AR: uno strumento di coaching per esperienze su pista, un sistema di smart farming per rendere più efficienti le operazioni in campo e una pipeline automatica per la generazione di dataset per l'addestramento di reti neurali. Tutto il lavoro è stato testato in collaborazione con HMDrive s.r.l., e la maggior parte degli algoritmi sviluppati è stata integrata nel loro software AR, a dimostrazione dell'impatto industriale dei risultati ottenuti.
Analysis and development of augmented reality and computer vision algorithms for automotive applications
GAMBAROTTO, LUCA
2025/2026
Abstract
Augmented Reality (AR) technologies are transforming industry by merging digital and physical information, yet their use in non-inertial environments, such as moving vehicles, has not been fully explored. This thesis develops methods and algorithms to enable reliable AR in moving vehicles, particularly on Head-Mounted Displays (HMDs). The research targets two main goals: estimating the six-degree-of-freedom (6-DoF) pose of an HMD within a moving vehicle, and localizing the vehicle with accuracy sufficient for geo-referenced AR visualization. For HMD pose estimation, a visual-inertial fusion architecture is introduced to compute motion relative to the cockpit, compensating for vehicle dynamics. Regarding the visual component, two novel computer-vision algorithms are presented, leveraging both monocular and stereo cameras installed in modern HMDs. For vehicle localization, two pipelines are introduced: one fuses GPS-RTK and attitude data for precise on-track mapping and positioning, while the other uses stereo vision for GPS-denied environments such as fields or vineyards. A reprojection-based validation method evaluates localization accuracy without ground truth. To demonstrate the validity of the approach in heterogeneous scenarios, sample AR applications are presented, leveraging the proposed algorithms. These include a coaching tool for on-track experiences, a smart-farming system to improve the productivity of in-field operations, and an automatic dataset generation pipeline for neural network training that exploits AR for generating and labeling synthetic images. All the work proposed in this thesis has been tested in collaboration with HMDrive s.r.l., and most of the algorithms have been integrated into their AR software stack, demonstrating the industrial impact of this work.| File | Dimensione | Formato | |
|---|---|---|---|
|
thesis_LG_compressed.pdf
non accessibile
Descrizione: PhD thesis
Dimensione
38.16 MB
Formato
Adobe PDF
|
38.16 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/254097