Dependability Engineering Innovation for Cyber-Physical Systems (DEIS) is an ICT project funded by the European Commission within the Horizon 2020 program. The DEIS project addresses important and unsolved challenges in assuring dependability within complex Cyber-Physical-Systems (CPS). New paradigms for dependable systems, such as Digital Dependability Identities (DDIs), will be developed and validated in several use cases, one of which is in the automotive field. POLIMI SPAD-lab participates to this project developing a Single-Photon Avalanche-Diode (SPAD) camera to perform 3D measurements inside the vehicle and also monitor biometric parameters of the driver, thus developing algorithms for remote Photo Plethysmography (rPPG) measurements. PPG is based on the principle that blood absorbs light more than surrounding tissues so variations in blood volume can be detected considering changes in the light intensity backscattered by the human skin. In rPPG a camera is used to acquire a video, typically of the subject face. Main focuses of this thesis are the estimation of Heart Rate (HR), Heart Rate Variability (HRV) and Respiration Rate (RR) of the drivers using the SPAD camera. Different devices were used to validate the results: a commercial RGB camera to measure rPPG with a more standard camera, portable ECG (Faros 180) in order to have a ground truth for heart signals, breaths resistive sensor to measure RR. Several measurements were performed to optimize the experimental setup. The first acquisitions with AC illumination (from a neon lamp) returned very low quality results. Thus further measurements have been acquired with an active DC illuminator and optical filters in front of the SPAD camera, with the purpose to compare the contribution of different wavelengths on the signal modulation. Measurements were performed with 10 optical filters from 400 nm to 850 nm (40 nm filter width, 50 nm steps) on five volunteers. The acquisitions lasted 10 minutes with the subject still in front of the SPAD camera, wearing a portable ECG for validation. Heart rate, tachogram and its spectrum were computed in post processing and compared to the same parameters estimated from the ECG signal. The highest accuracy was achieved with wavelengths centered in 550 nm and 850 nm (which typically is preferred for eye safety reasons). Other 10 minutes video acquisitions on three subjects at 550 nm wavelength were recorded also with the breath sensors and RGB camera, obtaining a very high accuracy over short (30 seconds) and long (10 minutes) time windows for HR, HRV and RR. Two LED illuminators at 550 nm and 850 nm were developed and validated with experimental measurements. Final measurements have been performed installing the SPAD camera and illuminator within a car cockpit. Issues related to movement artifacts and ambient light modulations have not been addressed in detail and will be object of future developments. In particular, face detection and tracking algorithms and adaptive filters will be applied to cope with the aforementioned issues, opening the possibility to exploit the developed system in automotive environment.
Dependability Engineering Innovation for Cyber-Physical Systems (DEIS) è un progetto ICT finanziato dalla commissione europea all'interno del programma Horizon 2020. Il progetto DEIS si occupa di questioni irrisolte nell'ambito dell'ffidabilità all'interno di Sistemi Ciberfisici (CPS) complessi. Verranno sviluppati nuovi paradigmi per l'affidabilità di sistemi complessi, ovvero Digital Dependability Identities (DDIs), che saranno validati su diversi casi pratici di impiego, uno dei quali nel campo automotive. Il laboratorio SPAD-lab del Politecnico di Milano, partecipa al progetto sviluppando una camera in grado di rivelare singoli fotoni (camera SPAD) per effettuare immagini 3D all'interno del veicolo e monitorare le condizioni di salute del guidatore, sviluppando algoritmi per misure di foto pletismografia da remoto (rPPG). La PPG si basa sull'assorbimento di luce, maggiore nel sangue ossigenato rispetto ai tessuti circostanti. Le variazioni di volume sanguigno possono essere misurate considerando i cambiamenti dell'intensità della luce riflessa dalla pelle. Nella foto pletismografia da remoto (rPPG) tipicamente si inquadra il volto del soggetto. Gli obiettivi di questa tesi sono la stima della frequenza cardiaca, della variabilità cardiaca, e della frequenza respiratoria del guidatore utilizzando la camera a singolo fotone. Per validare i risultati ottenuti sono stati utilizzati diversi dispositivi: una camera RGB commerciale, per avere un confronto sulle misure ottenute con una tipologia camera più standard, un elettrocardiografo portatile (FAROS 180), per avere un gold standard di confronto per il segnale cardiaco e un sensore resistivo per misurare il respiro. Sono state effettuate diverse misure per arrivare a definire un setup sperimentale ottimizzato. Le prime acquisizioni con illuminazione AC (data da lampade al neon) hanno fornito scarsi risultati, per questo motivo le successive acquisizioni sono state effettuate con un illuminatore attivo alimentato in DC e dei filtri ottici sull'obiettivo della camera SPAD, con l'obbiettivo di confrontare i contributi delle diverse lunghezze d'onda sul segnale cardiaco estratto. Le misure sono state effettuate utilizzando 10 filtri ottici tra 400 nm e 850 nm (con 40 nm di larghezza spettrale), con un passo di 50 nm, su cinque volontari. La durata delle acquisizioni è di 10 minuti, con soggetto fermo e monitorato con elettrocardiografo portatile per la successiva validazione dei risultati. Dal segnale estratto dal video sono stati calcolati frequenza cardiaca, variabilità cardiaca e relativo spettro e sono stati confrontati con i rispettivi dati estratti dal tracciato ECG. La migliore accuratezza è stata ottenuta dal segnale con lunghezza d'onda centrata a 550 nm e 850 nm (consigliata per motivi di eye-safety). Ulteriori acquisizioni di 10 minuti sono state effettuate su 3 soggetti a 550 nm, insieme a misure di respiro e al video della camera RGB, ottenendo elevata accuratezza per frequenza cardiaca, variabilità cardiaca e frequenza respiratoria sia su finestre temporali di breve durata (30 secondi) sia di lunga durata (10 minuti). Sono stati sviluppati e testati con misure sperimentali due illuminatori, uno a 550 nm e uno a 850 nm. Le ultime misure sono state effettuate all'interno di un'automobile installando la camera SPAD e gli illuminatori all'interno dell'abitacolo. Problematiche relative a artefatti da movimento e modulazione della luce naturale non sono state affrontate in dettaglio e saranno oggetto di sviluppi futuri. In particolare, algoritmi di riconoscimento del volto e tracking in aggiunta a filtri adattivi possono essere sviluppati per risolvere questi problemi, aprendo questa tecnologia a nuove possibilità nel mondo automotive.
Biometric signal estimation using single photon camera
MARCHESI, LORENZO
2017/2018
Abstract
Dependability Engineering Innovation for Cyber-Physical Systems (DEIS) is an ICT project funded by the European Commission within the Horizon 2020 program. The DEIS project addresses important and unsolved challenges in assuring dependability within complex Cyber-Physical-Systems (CPS). New paradigms for dependable systems, such as Digital Dependability Identities (DDIs), will be developed and validated in several use cases, one of which is in the automotive field. POLIMI SPAD-lab participates to this project developing a Single-Photon Avalanche-Diode (SPAD) camera to perform 3D measurements inside the vehicle and also monitor biometric parameters of the driver, thus developing algorithms for remote Photo Plethysmography (rPPG) measurements. PPG is based on the principle that blood absorbs light more than surrounding tissues so variations in blood volume can be detected considering changes in the light intensity backscattered by the human skin. In rPPG a camera is used to acquire a video, typically of the subject face. Main focuses of this thesis are the estimation of Heart Rate (HR), Heart Rate Variability (HRV) and Respiration Rate (RR) of the drivers using the SPAD camera. Different devices were used to validate the results: a commercial RGB camera to measure rPPG with a more standard camera, portable ECG (Faros 180) in order to have a ground truth for heart signals, breaths resistive sensor to measure RR. Several measurements were performed to optimize the experimental setup. The first acquisitions with AC illumination (from a neon lamp) returned very low quality results. Thus further measurements have been acquired with an active DC illuminator and optical filters in front of the SPAD camera, with the purpose to compare the contribution of different wavelengths on the signal modulation. Measurements were performed with 10 optical filters from 400 nm to 850 nm (40 nm filter width, 50 nm steps) on five volunteers. The acquisitions lasted 10 minutes with the subject still in front of the SPAD camera, wearing a portable ECG for validation. Heart rate, tachogram and its spectrum were computed in post processing and compared to the same parameters estimated from the ECG signal. The highest accuracy was achieved with wavelengths centered in 550 nm and 850 nm (which typically is preferred for eye safety reasons). Other 10 minutes video acquisitions on three subjects at 550 nm wavelength were recorded also with the breath sensors and RGB camera, obtaining a very high accuracy over short (30 seconds) and long (10 minutes) time windows for HR, HRV and RR. Two LED illuminators at 550 nm and 850 nm were developed and validated with experimental measurements. Final measurements have been performed installing the SPAD camera and illuminator within a car cockpit. Issues related to movement artifacts and ambient light modulations have not been addressed in detail and will be object of future developments. In particular, face detection and tracking algorithms and adaptive filters will be applied to cope with the aforementioned issues, opening the possibility to exploit the developed system in automotive environment.| File | Dimensione | Formato | |
|---|---|---|---|
|
2018_10_MARCHESI.pdf
accessibile in internet per tutti
Descrizione: Testo della tesi
Dimensione
16.28 MB
Formato
Adobe PDF
|
16.28 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/142646