Time division multiplexing passive optical networks (TDM-PON), with their massive deployment worldwide and the constantly decreasing energy and cost expenses of their deployment and implementation, are seen as an integral part of future communication networks not only for delivery of traditional broadband services, but also for supporting latency-sensitive and real-time services. Traditional reactive-based DBA methods adopt a polling mechanism to learn the ONUs buffers status incurring high queuing delay, especially for bursts arriving just after the ONU report is sent, and violating the requirements of latency-sensitive services. In this work, we propose a new predictive-based dynamic bandwidth allocation (DBA) mechanism to extend the deployment of XGS-PON beyond the FTTx broadband services by meeting the stringent latency requirements of emerging latency-sensitive services. We propose the first reinforcement learning-based DBA mechanism to overcome the main shortcoming of polling-based DBAs. Our solution employs, on the OLT side, an RL model in order to predict the ingress buffer occupancy of ONUs in the next DBA cycle. Then, the predicted values are used to proactively allocate bandwidth to ONUs reducing the queuing delay significantly. Further, to improve the prediction Accuracy and accelerate the RL training process, we use an independent RL agent for each traffic class (i.e., T-CONT). The proposed RL model was trained using PPO algorithm with an LSTM network policy. We compared the proposed mechanism with state-of-the-art DBAs in a simulated XGS-PON network with three T-CONTs, which belong to three different QoS requirements, using self-similar traffic. Results show that the proposed RL method outlines other alternatives in terms of upstream delay and jitter while maintaining similar frame loss ratio. Moreover, the proposed RL mechanism reduces T-CONT 2 upstream delay by 32% and 88% compared with IBU and GIANT DBAs.
Le reti ottiche passive multiplexing a divisione di tempo (TDM-PON), con la loro massiccia diffusione in tutto il mondo e la costante diminuzione dei costi energetici e dei costi della loro distribuzione e implementazione, sono viste come parte integrante delle future reti di comunicazione non solo per la fornitura di servizi tradizionali a banda larga, ma anche per supportare servizi sensibili alla latenza e in tempo reale. I tradizionali metodi DBA basati su reattività adottano un meccanismo di polling per apprendere lo stato dei buffer delle ONU che subiscono un elevato ritardo di accodamento, in particolare per i burst che arrivano subito dopo l'invio del rapporto ONU e violano i requisiti dei servizi sensibili alla latenza. In questo lavoro, proponiamo un nuovo meccanismo di allocazione dinamica della larghezza di banda (DBA) basato su predittivo per estendere l'implementazione di XGS-PON oltre i servizi a banda larga FTTx soddisfacendo i severi requisiti di latenza dei servizi sensibili alla latenza emergenti. Proponiamo il primo meccanismo DBA basato sull'apprendimento per rinforzo per superare il principale difetto dei DBA basati su polling. La nostra soluzione impiega, sul lato OLT, un modello RL per prevedere l'occupazione del buffer di ingresso delle ONU nel successivo ciclo DBA. Quindi, i valori previsti vengono utilizzati per allocare in modo proattivo la larghezza di banda alle ONU riducendo significativamente il ritardo di coda. Inoltre, per migliorare l'accuratezza della previsione e accelerare il processo di addestramento RL, utilizziamo un agente RL indipendente per ogni classe di traffico (cioè, T-CONT). Il modello RL proposto è stato addestrato utilizzando l'algoritmo PPO con una politica di rete LSTM. Abbiamo confrontato il meccanismo proposto con DBA all'avanguardia in una rete XGS-PON simulata con tre T-CONT, che appartengono a tre diversi requisiti di QoS, utilizzando traffico auto-simile. I risultati mostrano che il metodo RL proposto delinea altre alternative in termini di ritardo a monte e jitter mantenendo un rapporto di perdita di frame simile. Inoltre, il meccanismo RL proposto riduce il ritardo a monte di T-CONT 2 del 32% e dell'88% rispetto ai DBA IBU e GIANT.
A reinforcement learning-based dynamic bandwidth allocation for XGS-PON networks
Quran, Abdullah Moh'D Saleh
2021/2022
Abstract
Time division multiplexing passive optical networks (TDM-PON), with their massive deployment worldwide and the constantly decreasing energy and cost expenses of their deployment and implementation, are seen as an integral part of future communication networks not only for delivery of traditional broadband services, but also for supporting latency-sensitive and real-time services. Traditional reactive-based DBA methods adopt a polling mechanism to learn the ONUs buffers status incurring high queuing delay, especially for bursts arriving just after the ONU report is sent, and violating the requirements of latency-sensitive services. In this work, we propose a new predictive-based dynamic bandwidth allocation (DBA) mechanism to extend the deployment of XGS-PON beyond the FTTx broadband services by meeting the stringent latency requirements of emerging latency-sensitive services. We propose the first reinforcement learning-based DBA mechanism to overcome the main shortcoming of polling-based DBAs. Our solution employs, on the OLT side, an RL model in order to predict the ingress buffer occupancy of ONUs in the next DBA cycle. Then, the predicted values are used to proactively allocate bandwidth to ONUs reducing the queuing delay significantly. Further, to improve the prediction Accuracy and accelerate the RL training process, we use an independent RL agent for each traffic class (i.e., T-CONT). The proposed RL model was trained using PPO algorithm with an LSTM network policy. We compared the proposed mechanism with state-of-the-art DBAs in a simulated XGS-PON network with three T-CONTs, which belong to three different QoS requirements, using self-similar traffic. Results show that the proposed RL method outlines other alternatives in terms of upstream delay and jitter while maintaining similar frame loss ratio. Moreover, the proposed RL mechanism reduces T-CONT 2 upstream delay by 32% and 88% compared with IBU and GIANT DBAs.| File | Dimensione | Formato | |
|---|---|---|---|
|
Abdullah_Quran_Thesis.pdf
embargo fino al 29/11/2024
Dimensione
3.43 MB
Formato
Adobe PDF
|
3.43 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/182474