The present Thesis focuses on the assessment and improvement of a training set design procedure for Machine Learning (ML) application in the approximation of first-principles kinetic models. Machine learning and, in particular, the ExtraTrees algorithm have been recently regarded as promising tools to enable the coupling between computationally expensive kinetic schemes (mean-field and kMC) and reactor model, since they are conceived to deal with highly dimensional dataset and can learn complex relations. The generation of the training dataset is at the very core of this process, since it has to properly describe the system trend and behavior to enable the learning procedure of the artificial intelligence algorithms. Hence, specific attention has to be addressed to its generation to effectively reproduce the system complex behavior with a certain level of accuracy. Moreover, the generation of the dataset may be computationally expensive since it requires solving multiple times the full kinetic model to quantify the system outputs given a set of input conditions. In this view, the effective design of the dataset become extremely relevant to minimize the computational cost without any penalty on the approximation level. In this work, a dedicated procedure for the generation of the dataset, already proposed in the literature, has been analyzed, assessed and extended to properly deal with multi-output systems. The proposed procedure iteratively generates training datasets by adding points only on the most relevant directions and regions of the compositional space. The ExtraTrees algorithm intrinsically provides a score known as variable importance which quantifies the influence of each input variable on the output function. Directions with a high score are selected for point addition. New training points are placed where the target function shows the steepest variations, to accurately describe the output function where the approximation is more critical. This information is provided by computing first derivative. In this work, the use of the second derivate is implemented to discriminate minima and maxima from flat regions, improving the overall performances of the approach. The point addition terminates when the desired accuracy is reached. In this work, the procedure has been specifically extended to treat multi-outputs functions which is extremely relevant in the context of complex systems characterized by more than one stoichiometry. Moreover, a parametrical analysis of the effect of the machine learning and procedure features has been carried out. In particular, the optimal range of these parameters to minimize the dataset size is assessed in several different conditions providing guidelines for the usage of the procedure. Then, the procedure has been employed in three different kinetic systems characterized by an increasingly complexity in terms of number of variables to quantify the performances in terms of approximation accuracy and dataset size reduction. The resulting training set has been proved to effectively approximate the real system in all showcases by reducing the training data size of almost 40% on average with respect to an evenly spaced grid with the same accuracy. In doing so, the coupling between CFD and fist-principles kinetics is feasible and to demonstrate that the ExtraTrees trained on the resulting dataset has been implemented in a CFD simulation. The calyticFOAM solver has been employed and a numerical interface has been developed in this work to link the solver with the ML algorithm. The use of the ExtraTrees algorithm to approximate the reaction rates revealed to enable the coupling between complex kinetic models and CFD simulations. This result paves the way for the inclusion of computationally-intensive kMC model into reactor simulations. In conclusions, the procedure improved and analyzed has been proved successful in reducing the training set size while still providing an accurate description of the target function. The general structure of the code and the excellent performances on different kinetic systems highlight the potential of the procedure, which could be a possible breakthrough in the efficient generation of training set for the approximation of first-principles kinetic models.
Il presente lavoro di tesi si focalizza sulla stima e sul miglioramento di una procedura per progettare un training set per applicazioni Machine Learining (ML) all’approssimazione di modelli cinetici first-principles. Il Machine Learning e in particolare l’algoritmo Extra Trees, sono stati recentemente considerati strumenti promettenti per consentire il coupling tra schemi cinetici costosi dal punto di vista computazionale (mean-field e kMC) e macroscala, in quanto sono concepiti per gestire grandi quantità di dati e possono imparare relazioni complesse. La generazione del training dataset è il cuore del processo perché deve descrivere adeguatamente il trend e il comportamento del sistema consentendo la procedura di apprendimento dell’intelligenza artificiale. Quindi, particolare attenzione deve essere data alla sua generazione per riprodurre efficacemente sistemi complessi con un certo livello di accuratezza. Inoltre, la generazione del dataset può essere costosa dal punto di vista computazionale in quanto richiedere di risolvere il modello cinetico più volte per quantificare gli output del sistema data una serie di condizioni di input. In questa ottica, un design efficacie del dataset diventerebbe estremamente importante per minimizzare il costo computazionale, senza penalizzare l’accuratezza del modello. In questa tesi, una procedura dedicata alla generazione del dataset, già proposta in letteratura, è stata analizzata ed estesa per gestire sistemi multi-output. La procedura proposta genera training dataset iterativamente, aggiungendo punti solo nelle direzioni e nelle regioni più rilevanti del dominio. L’algoritmo Extra Trees fornisce intrinsecamente un punteggio, noto come variable importance in grado di quantificare l’importanza di una variable di input rispetto alla funzione di ouput. Le direzioni con il punteggio maggiore, sono scelte per l’aggiunta di punti. I nuovi punti vengono posizionati dove la funzione obiettivo mostra le variazioni maggiori, al fine di descrivere accuratamente la funzione dove l’approssimazione è più critica. Questa informazione è ottenuta calcolando la derivata prima. Nel presente lavoro, la derivata seconda è stata introdotta per distinguere massimi e minimi dalle regioni piatte, migliorando nel complesso le performance del metodo. L’aggiunta del punti termina una volta raggiunta l’accuratezza desiderata. In questo lavoro, la procedura è stata estesa per trattare sistemi multioutput, aspetto di estrema importanza per sistemi complessi, caratterizzati da più di una stechiometria. Inoltre, è stata effettuata un’analisi parametrica sull’effetto dei parametri dell’algoritmo di Machine Learning e della procedura stessa. Nello specifico, gli intervalli ottimali di tali parametri per minimizzare la dimensione del dataset sono stati stimati in diverse condizioni, fornendo linee guida per l’uso della procedura. In seguito, la procedura è stata applicata in tre diversi schemi cinetici caratterizzati da una complessità crescente in termini di numero di variabili al fine di valutare le performance rispetto alla qualità di approssimazione e alla riduzione delle dimensioni del dataset. Il dataset ottenuto dalla procedura si è rivelato efficacie nell’approssimare il sistema reale in tutti i casi d’esame, riducendo in media la dimensione del dataset del 40% rispetto ad una griglia uniforme con la stessa accuratezza. Questo approccio rende possibile il coupling tra CFD e modelli first-principles e per mostrare ciò l’ExtraTrees allenato sul dataset ottimale è stato implementato in una simulazione CFD. La soluzione è stata ottenuta con catalyticFOAM e nel presente lavoro un’interfaccia numerica è stata sviluppata per legare il solver CFD con l’algoritmo ML. L’utilizzo dell’algoritmo ExtraTrees per approssimare la velocità di reazione ha reso possibile il coupling tra modelli cinetici complessi e simulazioni CFD. Questo risultato apre la strada per l’inclusione di modelli kMC, complessi dal punto di vista computazionale, in simulazioni a scala di reattore. In conclusione, la procedura migliorata e analizzata si è rivelata efficacie nel ridurre la dimensione del training dataset fornendo contemporaneamente un’accurata descrizione della funzione obiettivo. La struttura generale del codice e gli ottimi risultati su schemi cinetici diversi sottolineano le potenzialità del metodo, che potrebbe essere una possibile svolta nella generazione efficiente dei training set per approssimare modelli cinetici first-principles.
Assessment and analysis of a training set design procedure for machine learning applied to the approximation of first-principles kinetic models
Tomasoni, Alessia
2019/2020
Abstract
The present Thesis focuses on the assessment and improvement of a training set design procedure for Machine Learning (ML) application in the approximation of first-principles kinetic models. Machine learning and, in particular, the ExtraTrees algorithm have been recently regarded as promising tools to enable the coupling between computationally expensive kinetic schemes (mean-field and kMC) and reactor model, since they are conceived to deal with highly dimensional dataset and can learn complex relations. The generation of the training dataset is at the very core of this process, since it has to properly describe the system trend and behavior to enable the learning procedure of the artificial intelligence algorithms. Hence, specific attention has to be addressed to its generation to effectively reproduce the system complex behavior with a certain level of accuracy. Moreover, the generation of the dataset may be computationally expensive since it requires solving multiple times the full kinetic model to quantify the system outputs given a set of input conditions. In this view, the effective design of the dataset become extremely relevant to minimize the computational cost without any penalty on the approximation level. In this work, a dedicated procedure for the generation of the dataset, already proposed in the literature, has been analyzed, assessed and extended to properly deal with multi-output systems. The proposed procedure iteratively generates training datasets by adding points only on the most relevant directions and regions of the compositional space. The ExtraTrees algorithm intrinsically provides a score known as variable importance which quantifies the influence of each input variable on the output function. Directions with a high score are selected for point addition. New training points are placed where the target function shows the steepest variations, to accurately describe the output function where the approximation is more critical. This information is provided by computing first derivative. In this work, the use of the second derivate is implemented to discriminate minima and maxima from flat regions, improving the overall performances of the approach. The point addition terminates when the desired accuracy is reached. In this work, the procedure has been specifically extended to treat multi-outputs functions which is extremely relevant in the context of complex systems characterized by more than one stoichiometry. Moreover, a parametrical analysis of the effect of the machine learning and procedure features has been carried out. In particular, the optimal range of these parameters to minimize the dataset size is assessed in several different conditions providing guidelines for the usage of the procedure. Then, the procedure has been employed in three different kinetic systems characterized by an increasingly complexity in terms of number of variables to quantify the performances in terms of approximation accuracy and dataset size reduction. The resulting training set has been proved to effectively approximate the real system in all showcases by reducing the training data size of almost 40% on average with respect to an evenly spaced grid with the same accuracy. In doing so, the coupling between CFD and fist-principles kinetics is feasible and to demonstrate that the ExtraTrees trained on the resulting dataset has been implemented in a CFD simulation. The calyticFOAM solver has been employed and a numerical interface has been developed in this work to link the solver with the ML algorithm. The use of the ExtraTrees algorithm to approximate the reaction rates revealed to enable the coupling between complex kinetic models and CFD simulations. This result paves the way for the inclusion of computationally-intensive kMC model into reactor simulations. In conclusions, the procedure improved and analyzed has been proved successful in reducing the training set size while still providing an accurate description of the target function. The general structure of the code and the excellent performances on different kinetic systems highlight the potential of the procedure, which could be a possible breakthrough in the efficient generation of training set for the approximation of first-principles kinetic models.| File | Dimensione | Formato | |
|---|---|---|---|
|
2020_10_Tomasoni.pdf
non accessibile
Dimensione
4.9 MB
Formato
Adobe PDF
|
4.9 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/166402