This thesis presents one of the first engineering steps of the Sound of Bees project, aimed at developing an operational multimodal framework for monitoring honeybee communication in observation hives, with particular focus on the stop signal. The study of the bees’ stop signal—a short, vibrational-acoustic event associated with a characteristic head-butting behaviour—is challenging because candidate events are rare, noisy, and not reliably identifiable from audio alone. Since visual validation hence remains necessary, stop-signal research still relies heavily on manual observation and thus struggles to produce large-scale quantitative datasets. The long-term goal is to automatically extract plausible stop-signal candidates and present them as synchronized audio-video clips, reducing manual monitoring and enabling a stronger focus on biological interpretation. Within this context, the main contribution of this thesis is the development of an end-to-end multimodal framework combining hardware, acquisition software, and post-processing tools. The implemented setup is based on an observation hive instrumented for continuous multichannel audio and synchronized video recording. In addition, a software layer was developed to orchestrate audio-video recording and manage storage, file organisation, and data transfer. On the analysis side, a post-processing workflow was implemented to extract acoustically plausible candidates, enrich them with relevant audio features, including fundamental frequency estimated through cepstral analysis, and automatically generate synchronized audio-video clips centred on each detected event. For audio-based candidate extraction, we compared two approaches: a classical digital signal processing (DSP) pipeline based on harmonic-to-noise ratio, acoustic feature extraction, and rule-based filtering; and a deep-learning-based approach, in which Animal2vec, a self-supervised model for rare-event bioacoustics, was adapted to honeybee recordings through pretraining and fine-tuning on beehive audio from the previous season. While the Animal2vec-based approach achieved encouraging performance on the validation split of the labelled dataset, it generalized poorly to the new recording campaign, producing a high number of false positives and therefore lacking practical reliability. By contrast, the classical DSP pipeline proved more effective as a conservative high-precision candidate-retrieval tool: although it was not suitable as a fully exhaustive detector, it retained a useful proportion of plausible candidates while keeping manual validation manageable. This thesis represents a concrete first step towards large-scale automated monitoring of honeybee acoustic communication, while also clarifying the main technical challenges that remain open, including precise localisation, robust inter-channel synchronization, and improved adaptation of Animal2vec to beehive recordings. Although the thesis does not yet provide a definitive solution to automated stop-signal detection, it successfully established a complete and operational multimodal framework for continuous hive monitoring and the generation of synchronized candidate-centred audio-video clips.
Questa tesi presenta uno dei primi passi ingegneristici del progetto The Sound of Bees, finalizzato allo sviluppo di un framework multimodale operativo per il monitoraggio della comunicazione delle api mellifere in alveari di osservazione, con particolare attenzione allo stop signal. Lo studio dello stop signal delle api --- un breve evento vibro-acustico associato a un caratteristico comportamento di head-butting --- è complesso perché gli eventi candidati sono rari, rumorosi e non identificabili in modo affidabile a partire dal solo audio. Poiché la validazione visiva rimane quindi necessaria, la ricerca sullo stop signal dipende ancora fortemente dall’osservazione manuale e fatica quindi a produrre dataset quantitativi su larga scala. L’obiettivo a lungo termine è estrarre automaticamente candidati plausibili di stop signal e presentarli come clip audio-video sincronizzate, riducendo il monitoraggio manuale e consentendo una maggiore focalizzazione sull’interpretazione biologica. In questo contesto, il contributo principale di questa tesi consiste nello sviluppo di un framework multimodale end-to-end che combina hardware, software di acquisizione e strumenti di post-processing. Il setup implementato è basato su un alveare di osservazione strumentato per la registrazione continua di audio multicanale e video sincronizzato. Inoltre, è stato sviluppato un livello software per orchestrare la registrazione audio-video e gestire storage, organizzazione dei file e trasferimento dei dati. Sul versante dell’analisi, è stato implementato un workflow di post-processing per estrarre candidati acusticamente plausibili, arricchirli con caratteristiche audio rilevanti, inclusa la frequenza fondamentale stimata mediante analisi cepstrale, e generare automaticamente clip audio-video sincronizzate centrate su ciascun evento rilevato. Per la fase di estrazione dei candidati basata sull’audio, sono stati esplorati due approcci alternativi: il primo, una pipeline basata su tecniche classiche di DSP, fondata su rilevamento dell’armonicità basato su HNR (Harmonic to Noise Ratio), estrazione di caratteristiche acustiche e filtraggio rule-based; il secondo, basato su Animal2vec, un framework di deep learning self-supervised per la bioacustica di eventi rari, adattato al dominio delle api mellifere mediante pretraining e fine-tuning su registrazioni di alveare della stagione precedente. Sebbene l’approccio basato su Animal2vec abbia ottenuto prestazioni incoraggianti sul validation split del dataset etichettato, ha generalizzato male sulle registrazioni acquisite durante la nuova stagione, producendo troppi falsi positivi per poter essere considerato affidabile in pratica. Al contrario, la pipeline DSP classica si è dimostrata più efficace come strumento conservativo di candidate retrieval ad alta precisione: sebbene non fosse adatta come rilevatore completamente esaustivo, è riuscita a mantenere una quota utile di candidati plausibili, mantenendo al contempo gestibile la validazione manuale. Questa tesi rappresenta un primo passo concreto verso il monitoraggio automatico su larga scala della comunicazione acustica delle api mellifere, chiarendo al contempo le principali sfide tecniche ancora aperte, tra cui la localizzazione precisa, la sincronizzazione robusta inter-channel e l’ulteriore adattamento di Animal2vec al dominio di interesse, migliorandone la capacità di generalizzazione su nuove registrazioni. Sebbene la tesi non offra ancora una soluzione definitiva al riconoscimento automatico dello stop signal, è riuscita a stabilire un framework multimodale completo e operativo per il monitoraggio continuo dell’alveare e la generazione di clip audio-video sincronizzate centrate sui candidati.
The Sound of Bees : stop signal detection and automatic audio and video monitoring in beehives
Montanari, Mattia
2025/2026
Abstract
This thesis presents one of the first engineering steps of the Sound of Bees project, aimed at developing an operational multimodal framework for monitoring honeybee communication in observation hives, with particular focus on the stop signal. The study of the bees’ stop signal—a short, vibrational-acoustic event associated with a characteristic head-butting behaviour—is challenging because candidate events are rare, noisy, and not reliably identifiable from audio alone. Since visual validation hence remains necessary, stop-signal research still relies heavily on manual observation and thus struggles to produce large-scale quantitative datasets. The long-term goal is to automatically extract plausible stop-signal candidates and present them as synchronized audio-video clips, reducing manual monitoring and enabling a stronger focus on biological interpretation. Within this context, the main contribution of this thesis is the development of an end-to-end multimodal framework combining hardware, acquisition software, and post-processing tools. The implemented setup is based on an observation hive instrumented for continuous multichannel audio and synchronized video recording. In addition, a software layer was developed to orchestrate audio-video recording and manage storage, file organisation, and data transfer. On the analysis side, a post-processing workflow was implemented to extract acoustically plausible candidates, enrich them with relevant audio features, including fundamental frequency estimated through cepstral analysis, and automatically generate synchronized audio-video clips centred on each detected event. For audio-based candidate extraction, we compared two approaches: a classical digital signal processing (DSP) pipeline based on harmonic-to-noise ratio, acoustic feature extraction, and rule-based filtering; and a deep-learning-based approach, in which Animal2vec, a self-supervised model for rare-event bioacoustics, was adapted to honeybee recordings through pretraining and fine-tuning on beehive audio from the previous season. While the Animal2vec-based approach achieved encouraging performance on the validation split of the labelled dataset, it generalized poorly to the new recording campaign, producing a high number of false positives and therefore lacking practical reliability. By contrast, the classical DSP pipeline proved more effective as a conservative high-precision candidate-retrieval tool: although it was not suitable as a fully exhaustive detector, it retained a useful proportion of plausible candidates while keeping manual validation manageable. This thesis represents a concrete first step towards large-scale automated monitoring of honeybee acoustic communication, while also clarifying the main technical challenges that remain open, including precise localisation, robust inter-channel synchronization, and improved adaptation of Animal2vec to beehive recordings. Although the thesis does not yet provide a definitive solution to automated stop-signal detection, it successfully established a complete and operational multimodal framework for continuous hive monitoring and the generation of synchronized candidate-centred audio-video clips.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026_07_Montanari_executive_summary.pdf
accessibile in internet per tutti
Descrizione: The Sound of Bees, executive summary
Dimensione
1.38 MB
Formato
Adobe PDF
|
1.38 MB | Adobe PDF | Visualizza/Apri |
|
2026_07_Montanari_tesi.pdf
accessibile in internet per tutti
Descrizione: The Sound of Bees, thesis
Dimensione
37.13 MB
Formato
Adobe PDF
|
37.13 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/261857