The increasing demands of the Industrial Internet of Things (IIoT) bring significant challenges, emphasizing the importance of effectively scheduling and managing mixed criticality information. This master thesis incorporates deep reinforcement learning (DRL) in mixed-criticality applications with functions that have different timing requirements, i.e., hard real-time (HRT), soft real-time (SRT), and functions that are not time-critical (NC) to determine the traffic type of each message. Time-Sensitive Networking (TSN) supports the convergence of multiple traffic types, i.e., critical, real-time, and regular "best-effort" traffic within a single network: Time-triggered (TT), where the messages are transmitted based on static schedule tables, Audio-video Bridging (AVB), for dynamically scheduled messages with a guaranteed bandwidth and bounded delays, and Best Effort (BE), for which no timing guarantees are provided. The HRT messages have deadlines, whereas we capture the quality-of-service for the SRT messages using "utility functions." We propose a Soft Actor-Critic-based(SAC-based) scheduler to determine the traffic type of each message, such that as many as the HRT messages are schedulable and the total utility for the SRT messages is maximized. The proposed proof-of-concept tool has been evaluated using several benchmarks, including two realistic test cases. The results indi cate at least a 12% improvement in HRT schedule and at least a 6.58% improvement in the SRT schedule.
Le crescenti esigenze dell’Industrial Internet of Things (IIoT) comportano sfide significa tive, sottolineando l’importanza di pianificare e gestire in modo efficace le informazioni a criticità mista. Questa tesi di master incorpora l’apprendimento per rinforzo profondo (DRL) in applicazioni a criticità mista con funzioni che hanno requisiti temporali diversi, ad esempio hard real-time (HRT), soft real-time (SRT) e funzioni che non sono time critical ( NC) per determinare il tipo di traffico di ciascun messaggio. Time-Sensitive Networking (TSN) supporta la convergenza di più tipi di traffico, ovvero traffico critico, in tempo reale e regolare "best-effort" all’interno di una singola rete: Time-triggered (TT), in cui i messaggi vengono trasmessi in base a tabelle di schedulazione statiche, Audio video Bridging (AVB), per messaggi schedulati dinamicamente con larghezza di banda garantita e ritardi limitati, e Best Effort (BE), per i quali non sono previste garanzie tem porali. I messaggi HRT hanno delle scadenze, mentre acquisiamo la qualità del servizio per i messaggi SRT utilizzando "funzioni di utilità". Proponiamo uno scheduler basato su Soft Actor-Critic (basato su SAC) per determinare il tipo di traffico di ciascun messaggio, in modo tale che tanti messaggi HRT siano programmabili e l’utilità totale per i messaggi SRT sia massimizzata. Lo strumento di prova proposto è stato valutato utilizzando di versi parametri di riferimento, inclusi due casi di test realistici. I risultati indicano un miglioramento di almeno il 12% nel programma di TOS e un miglioramento di almeno il 6,58% nel programma di SRT.
Traffic-type assignment using deep reinforcement learning in TSN
YU, ZHEN
2023/2024
Abstract
The increasing demands of the Industrial Internet of Things (IIoT) bring significant challenges, emphasizing the importance of effectively scheduling and managing mixed criticality information. This master thesis incorporates deep reinforcement learning (DRL) in mixed-criticality applications with functions that have different timing requirements, i.e., hard real-time (HRT), soft real-time (SRT), and functions that are not time-critical (NC) to determine the traffic type of each message. Time-Sensitive Networking (TSN) supports the convergence of multiple traffic types, i.e., critical, real-time, and regular "best-effort" traffic within a single network: Time-triggered (TT), where the messages are transmitted based on static schedule tables, Audio-video Bridging (AVB), for dynamically scheduled messages with a guaranteed bandwidth and bounded delays, and Best Effort (BE), for which no timing guarantees are provided. The HRT messages have deadlines, whereas we capture the quality-of-service for the SRT messages using "utility functions." We propose a Soft Actor-Critic-based(SAC-based) scheduler to determine the traffic type of each message, such that as many as the HRT messages are schedulable and the total utility for the SRT messages is maximized. The proposed proof-of-concept tool has been evaluated using several benchmarks, including two realistic test cases. The results indi cate at least a 12% improvement in HRT schedule and at least a 6.58% improvement in the SRT schedule.| File | Dimensione | Formato | |
|---|---|---|---|
|
2024_07_Yu_Thesis.pdf
accessibile in internet solo dagli utenti autorizzati
Dimensione
2.32 MB
Formato
Adobe PDF
|
2.32 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/222710