Event-based cameras are a new type of vision sensors that asynchronously measure pixels’ intensity changes, resulting in a stream of events, or spikes, which encode time, location, and polarity of brightness changes. They have a very high dynamic range (140 dB versus 60 dB), latency on the order of microseconds and drastically reduce the problem of motion blur. The remarkable characteristics of this technology open up the opportunity to develop new useful systems for industrial applications, since they marry well with their requirements. Nevertheless, current works on computer vision methods based on events are still in early stages and are mainly research projects. In this work we will focus on the usage of event-based cameras for monitoring of manual industrial processes, a key task when dealing with operators performance assessment and safety issues. Our main objective is to suggest a feasible and complete workflow for monitoring using this new technology: starting from the definition of the problem and of industrial requirements for the target application, we will determine which computer vision methods are necessary for performing a complete Human Activity Recognition (HAR). Then, we will collect a plethora of different techniques for building blocks of interest and develop our own evaluation framework for their analysis, establishing some useful qualitative KPIs for assessing their accuracy, robustness and feasibility. In addition, we will perform some experiments on open-source codes in order to understand their ease-of-use and their performances on event-based videos recorded by our own. On the one hand, our study reveals that existing approaches’ Technology Readiness Level (TRL) is in between level 4 and 5, because most of them are research works tested in lab conditions and they should be experimented on more complex environments. On the other hand, the majority of them prove to retain all events benefits, being able to process very fast movements with almost no motion blur, to avoid data redundancy and to work with very low latency, enabling real-time implementation. With further development of the proposed workflow, there is therefore potential for developing a complete, feasible and marketable system for monitoring using event-based cameras.
Le telecamere event-based sono un nuovo tipo di sensori di visione che misurano in modo asincrono i cambiamenti di intensità dei pixel, risultando in un flusso di eventi, o spikes, che codificano il tempo, la posizione e la polarità dei cambiamenti di luminosità. Hanno una gamma dinamica molto alta (140 dB contro 60 dB), una latenza dell'ordine dei microsecondi e riducono drasticamente il problema della sfocatura del movimento. Le notevoli caratteristiche di questa tecnologia aprono l'opportunità di sviluppare nuovi sistemi utili per le applicazioni industriali, poiché si sposano bene con le loro esigenze. Tuttavia, gli attuali lavori sui metodi di computer vision basati sugli eventi sono ancora in fase iniziale e sono principalmente progetti di ricerca. In questo lavoro ci concentreremo sull'uso di telecamere event-based per il monitoraggio di processi industriali manuali, un compito chiave quando si tratta di valutare le prestazioni degli operatori e le questioni di sicurezza. Il nostro obiettivo principale è quello di suggerire un flusso di lavoro fattibile e completo per il monitoraggio, utilizzando questa nuova tecnologia: partendo dalla definizione del problema e dei requisiti industriali per l'applicazione target, determineremo quali metodi di computer vision sono necessari per eseguire un completo Human Activity Recognition (HAR). In seguito, raccoglieremo una pletora di diverse tecniche per ogni elemento costitutivo di interesse e svilupperemo un nostro quadro di valutazione per la loro analisi, stabilendo alcuni KPIs qualitativi utili per valutare la loro accuratezza, robustezza e fattibilità. Inoltre, eseguiremo alcuni esperimenti su codici open-source per capire la loro facilità d'uso e le loro prestazioni su video basati su eventi registrati da noi stesse. Da un lato, il nostro studio rivela che il Technology Readiness Level (TRL) degli approcci esistenti è tra i livelli 4 e 5, in quanto la maggior parte di essi sono lavori di ricerca testati in condizioni di laboratorio, e dovrebbero essere sperimentati in ambienti più complessi. D'altra parte, la maggior parte di essi dimostra di riuscire a mantenere tutti i vantaggi degli eventi, essendo in grado di elaborare movimenti molto veloci con quasi nessuna sfocatura del movimento, di evitare la ridondanza dei dati e di lavorare con una latenza molto bassa, permettendo un'implementazione in tempo reale. Con un ulteriore sviluppo del flusso di lavoro proposto, c'è quindi il potenziale per sviluppare un sistema completo, fattibile e commerciabile per il monitoraggio utilizzando telecamere event-based.
Assessment of event-based cameras technology for monitoring of manual industrial processes
CONSOLARO, GRETA;KALALI, YASMINE
2020/2021
Abstract
Event-based cameras are a new type of vision sensors that asynchronously measure pixels’ intensity changes, resulting in a stream of events, or spikes, which encode time, location, and polarity of brightness changes. They have a very high dynamic range (140 dB versus 60 dB), latency on the order of microseconds and drastically reduce the problem of motion blur. The remarkable characteristics of this technology open up the opportunity to develop new useful systems for industrial applications, since they marry well with their requirements. Nevertheless, current works on computer vision methods based on events are still in early stages and are mainly research projects. In this work we will focus on the usage of event-based cameras for monitoring of manual industrial processes, a key task when dealing with operators performance assessment and safety issues. Our main objective is to suggest a feasible and complete workflow for monitoring using this new technology: starting from the definition of the problem and of industrial requirements for the target application, we will determine which computer vision methods are necessary for performing a complete Human Activity Recognition (HAR). Then, we will collect a plethora of different techniques for building blocks of interest and develop our own evaluation framework for their analysis, establishing some useful qualitative KPIs for assessing their accuracy, robustness and feasibility. In addition, we will perform some experiments on open-source codes in order to understand their ease-of-use and their performances on event-based videos recorded by our own. On the one hand, our study reveals that existing approaches’ Technology Readiness Level (TRL) is in between level 4 and 5, because most of them are research works tested in lab conditions and they should be experimented on more complex environments. On the other hand, the majority of them prove to retain all events benefits, being able to process very fast movements with almost no motion blur, to avoid data redundancy and to work with very low latency, enabling real-time implementation. With further development of the proposed workflow, there is therefore potential for developing a complete, feasible and marketable system for monitoring using event-based cameras.| File | Dimensione | Formato | |
|---|---|---|---|
|
Master Thesis - Consolaro, Kalali.pdf
non accessibile
Descrizione: Master Thesis - Consolaro, Kalali
Dimensione
4.91 MB
Formato
Adobe PDF
|
4.91 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/186413