Hardware-aware neural architecture search (NAS) for graph neural networks (GNNs) predominantly assumes that operation costs are deterministic quantities derived from static look-up tables, an approximation increasingly inadequate as deployment targets shift toward Non-Volatile Memory (NVM)-based accelerators, where cell conductance variability and read noise make latency and energy genuinely stochastic. This thesis investigates whether replacing deterministic cost proxies with stochastic distributions changes the architectures selected by hardware-aware GNN-NAS. We design and implement a complete GNNNAS system over the ogbn-arxiv node classification benchmark, built around a shared-weight one-shot supernet trained with the sandwich rule across a 331,776- architecture search space. Architecture search is performed using an evolutionary algorithm under six objectives: accuracy, deterministic latency, deterministic energy, NVM weight robustness, and two CVaR-penalised stochastic objectives for uncertain latency and uncertain energy. The sandwich rule supports multi-width search at virtually no accuracy cost: the accuracy-optimal architecture achieves 62.75% validation accuracy, within 0.03 percentage points of the standalone GCN- 256 baseline. All hardware-aware objectives consistently select the minimum hidden dimension (dh = 64), confirming that width is the dominant hardware design knob. The evolutionary searches for uncertain and deterministic latency find different local optima (A4: att-att-mean vs. A2: att-mean-att); cross-evaluation reveals the two architectures score nearly identically on both objectives, since they share the same operation composition and the proxy cost model assigns variance proportionally to mean, preserving cost ordering under the CVaR correction. The uncertain energy objective converges to the same winner as its deterministic counterpart. These results show that a GNN-NAS framework with stochastic hardware cost objectives and CVaR-based risk penalisation is feasible and successfully executed over a 331,776-architecture space. Under the proxy cost model the stochastic and deterministic objectives converge to the same narrow architecture class, consistent with the proportional variance structure of the proxy. Genuine divergence between deterministic and risk-averse objectives is expected to emerge with empirically measured NVM hardware cost tables, where operation cost variance can be decoupled from the mean.
La ricerca di architetture neurali (NAS) hardware-consapevole per le reti neurali su grafi (GNN) assume prevalentemente che i costi operativi siano quantità deterministiche derivate da tabelle di look-up statiche, un’approssimazione sempre meno adeguata man mano che gli obiettivi di deployment si spostano verso gli acceleratori basati su memoria non volatile (NVM), nei quali la variabilità della conduttanza delle celle e il rumore di lettura rendono la latenza e l’energia genuinamente stocastiche. Questa tesi studia se sostituire i proxy di costo deterministici con distribuzioni stocastiche modifichi le architetture selezionate dalla GNN-NAS hardware-consapevole. Viene progettato e implementato un sistema GNN-NAS completo sul benchmark di classificazione dei nodi ogbn-arxiv, costruito attorno a una supernet one-shot a pesi condivisi addestrata con la sandwich rule su uno spazio di ricerca di 331.776 architetture. La ricerca avviene tramite un algoritmo evolutivo su sei obiettivi: accuratezza, latenza e energia deterministiche, robustezza ai pesi NVM, e due obiettivi stocastici penalizzati con CVaR per latenza ed energia incerte. La sandwich rule supporta la ricerca multi-larghezza a un costo di accuratezza quasi nullo: l’architettura ottimale per l’accuratezza raggiunge il 62,75% di accuratezza di validazione, a soli 0,03 punti percentuali dalla baseline GCN-256 addestrata in modo autonomo. Tutti gli obiettivi hardware-consapevoli selezionano la dimensione nascosta minima (dh = 64), confermando che la larghezza è il principale parametro di progettazione hardware. Le ricerche evolutive per latenza incerta e deterministica trovano ottimi locali diversi (A4: att-att-mean vs. A2: att-mean-att); la cross-valutazione rivela che le due architetture ottengono punteggi quasi identici su entrambi gli obiettivi, poiché condividono la stessa composizione operativa e il modello di costo proxy assegna la varianza proporzionalmente alla media, preservando l’ordinamento dei costi sotto la correzione CVaR. L’obiettivo di energia incerta converge allo stesso vincitore della controparte deterministica. Questi risultati mostrano che un framework GNN-NAS con obiettivi stocastici di costo hardware e penalizzazione basata su CVaR è fattibile ed efficiente su uno spazio di 331.776 architetture. Con il modello di costo proxy, gli obiettivi stocastici e deterministici convergono alla stessa classe architetturale ristretta, risultato coerente con la struttura proporzionale della varianza del proxy. Una divergenza genuina tra obiettivi deterministici e risk-averse è attesa con tabelle di costo hardware NVM misurate empiricamente, dove la varianza del costo operativo può essere disaccoppiata dalla media.
Hardware-aware GNN Neural Architecture Search under uncertain cost models
HAGHIGHAT, MIKAEIL
2025/2026
Abstract
Hardware-aware neural architecture search (NAS) for graph neural networks (GNNs) predominantly assumes that operation costs are deterministic quantities derived from static look-up tables, an approximation increasingly inadequate as deployment targets shift toward Non-Volatile Memory (NVM)-based accelerators, where cell conductance variability and read noise make latency and energy genuinely stochastic. This thesis investigates whether replacing deterministic cost proxies with stochastic distributions changes the architectures selected by hardware-aware GNN-NAS. We design and implement a complete GNNNAS system over the ogbn-arxiv node classification benchmark, built around a shared-weight one-shot supernet trained with the sandwich rule across a 331,776- architecture search space. Architecture search is performed using an evolutionary algorithm under six objectives: accuracy, deterministic latency, deterministic energy, NVM weight robustness, and two CVaR-penalised stochastic objectives for uncertain latency and uncertain energy. The sandwich rule supports multi-width search at virtually no accuracy cost: the accuracy-optimal architecture achieves 62.75% validation accuracy, within 0.03 percentage points of the standalone GCN- 256 baseline. All hardware-aware objectives consistently select the minimum hidden dimension (dh = 64), confirming that width is the dominant hardware design knob. The evolutionary searches for uncertain and deterministic latency find different local optima (A4: att-att-mean vs. A2: att-mean-att); cross-evaluation reveals the two architectures score nearly identically on both objectives, since they share the same operation composition and the proxy cost model assigns variance proportionally to mean, preserving cost ordering under the CVaR correction. The uncertain energy objective converges to the same winner as its deterministic counterpart. These results show that a GNN-NAS framework with stochastic hardware cost objectives and CVaR-based risk penalisation is feasible and successfully executed over a 331,776-architecture space. Under the proxy cost model the stochastic and deterministic objectives converge to the same narrow architecture class, consistent with the proportional variance structure of the proxy. Genuine divergence between deterministic and risk-averse objectives is expected to emerge with empirically measured NVM hardware cost tables, where operation cost variance can be decoupled from the mean.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026_07_Haghighat.pdf
accessibile in internet solo dagli utenti autorizzati
Dimensione
1.26 MB
Formato
Adobe PDF
|
1.26 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/260654