The rapid growth of platforms such as Hugging Face has made hundreds of thousands of pre-trained models readily available to researchers, making the development of new ma- chine learning applications significantly faster. However this abundance of models has also introduced a challenge: identifying the most suitable model for a specific task has become increasingly difficult. Even though repositories provide extensive metadata covering com- plete information, it is often heterogeneous, fragmented across multiple repository compo- nents and with different levels of completeness, limiting its effectiveness for proper model selection. This thesis addresses these challenges by investigating how metadata can be organized and extracted to better support large scale ML repositories inside the emerging model lake paradigm. A hierarchical metadata framework is proposed to organize reposi- tory information according to its functional role in the model selection process, prioritizing metadata based on its contribution to discovery, technical evaluation, provenance, perfor- mance assessment, governance and contextual understanding. Following a multi source metadata extraction pipeline is designed to consolidate heterogeneous information into a unified, machine readable metadata representation. The proposed approach is validated through an empirical analysis of 104 Hugging Face repositories covering 53 distinct ma- chine learning tasks, with the experimental results showing that metadata availability varies significantly across categories: discovery and technical metadata are consistently well represented through structured repository components, when governance, contextual and relational metadata remain considerably less available and more difficult to extract automatically. The evaluation also demonstrates that metadata importance and metadata availability are not necessarily correlated, highlighting the need for metadata management strategies that balance decision making value with practical extractability. The proposed work provide a structured foundation for organizing heterogeneous metadata inside ma- chine learning repositories and demonstrate how information distributed across multiple repository components can be consolidated into a coherent representation suitable for large scale model discovery and selection. Besides the specific case of Hugging Face, the results contribute to the broader development of metadata driven infrastructures capable of supporting scalable, interoperable and efficient model lake environments.
La crescita di piattaforme come Hugging Face ha reso disponibili a ricercatori centinaia di migliaia di modelli di ML pre-addestrati, accelerando significativamente lo sviluppo di nuove applicazioni di intelligenza artificiale. Tuttavia, una nuova sfida è stata introdotta: individuare il modello più adatto a uno specifico caso d’uso è diventato un processo sempre più complesso. Nonostante ci sia una grande quantità di metadati relativi ai dei modelli, risultano spesso eterogenee, distribuite tra diverse componenti e documentate con dif- ferenti livelli di completezza, limitandone l’efficacia. Questa tesi tratta come i metadati possano essere organizzati ed estratti per supportare in modo più efficace cartelle ML di grandi dimensioni all’interno del paradigma emergente dei Model Lake. Per questo mo- tivo viene proposta una gerarchia dei metadati che organizza le informazioni in funzione del loro ruolo nel processo di selezione dei modelli, attribuendo priorità ai metadati rela- tivi alla scoperta del modello, alla valutazione tecnica, alla provenienza, alle prestazioni, agli aspetti di governance e alle informazioni contestuali. Sulla base di questa gerarchia viene inoltre progettata un proccesso di estrazione per integrare informazioni eterogenee provenienti da differenti componenti delle cartelle in una rappresentazione unificata e in- terpretabile. Il lavoro è stato validato attraverso un’analisi condotta su 104 modelli di Hugging Face appartenenti a 53 task di ML ed i risultati mostrano che la disponibilità dei metadati varia significativamente tra le diverse categorie: i metadati relativi alla scoperta del modello e agli aspetti tecnici risultano ampiamente rappresentati mediante compo- nenti strutturate, mentre i metadati di governance, contestuali e relazionali presentano una disponibilità inferiore e risultano più complessi da estrarre automaticamente. L’analisi evidenzia inoltre come l’importanza di un metadato non sia necessariamente correlata alla sua disponibilità, sottolineando la necessità di strategie di gestione dei metadati capaci di bilanciare il valore informativo con la reale estraibilità. Il lavoro proposto fornisce una base per l’organizzazione dei metadati all’interno delle cartelle di ML e dimostra come in- formazioni eterogenee possano essere consolidate in una rappresentazione coerente, adatta a supportare attività di ricerca e selezione dei modelli su larga scala. Sebbene il caso stu- dio di Hugging Face, i risultati ottenuti contribuiscono più in generale allo sviluppo di infrastrutture basate sui metadati, per renderle scalabili, interoperabili ed efficienti.
A metadata hierarchy and multi source extraction framework for model lake architectures
SOUSSANE, FIRAS
2025/2026
Abstract
The rapid growth of platforms such as Hugging Face has made hundreds of thousands of pre-trained models readily available to researchers, making the development of new ma- chine learning applications significantly faster. However this abundance of models has also introduced a challenge: identifying the most suitable model for a specific task has become increasingly difficult. Even though repositories provide extensive metadata covering com- plete information, it is often heterogeneous, fragmented across multiple repository compo- nents and with different levels of completeness, limiting its effectiveness for proper model selection. This thesis addresses these challenges by investigating how metadata can be organized and extracted to better support large scale ML repositories inside the emerging model lake paradigm. A hierarchical metadata framework is proposed to organize reposi- tory information according to its functional role in the model selection process, prioritizing metadata based on its contribution to discovery, technical evaluation, provenance, perfor- mance assessment, governance and contextual understanding. Following a multi source metadata extraction pipeline is designed to consolidate heterogeneous information into a unified, machine readable metadata representation. The proposed approach is validated through an empirical analysis of 104 Hugging Face repositories covering 53 distinct ma- chine learning tasks, with the experimental results showing that metadata availability varies significantly across categories: discovery and technical metadata are consistently well represented through structured repository components, when governance, contextual and relational metadata remain considerably less available and more difficult to extract automatically. The evaluation also demonstrates that metadata importance and metadata availability are not necessarily correlated, highlighting the need for metadata management strategies that balance decision making value with practical extractability. The proposed work provide a structured foundation for organizing heterogeneous metadata inside ma- chine learning repositories and demonstrate how information distributed across multiple repository components can be consolidated into a coherent representation suitable for large scale model discovery and selection. Besides the specific case of Hugging Face, the results contribute to the broader development of metadata driven infrastructures capable of supporting scalable, interoperable and efficient model lake environments.| File | Dimensione | Formato | |
|---|---|---|---|
|
Tesi_Firas_Soussane.pdf
solo utenti autorizzati a partire dal 30/06/2027
Descrizione: tesi aggiornata
Dimensione
643.79 kB
Formato
Adobe PDF
|
643.79 kB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/261576