This thesis investigates how adaptive and behaviourally informed design can improve the first-time user experience [FTUE] in web-based educational applications. Using Flashka.ai, an AI-powered study platform, as a case study, it explores how usability principles, personalised onboarding, and transparency mechanisms influence early activation, retention, and engagement. The research combines qualitative and quantitative methods, including CES evaluation, a cognitive walkthrough and user testing with first-time learners. From these analyses, nine design requirements were defined and translated into a complete redesign of Flashka’s onboarding, flashcard generation, and study flows. Each intervention was evaluated through measurable behavioural outcomes across two user cohorts: before and after implementation, via A/B testing on product data. The results show that adaptive onboarding and usability refinements can meaningfully shape user behaviour. Retention and monetisation improved substantially: Day-1 retention +12.6%, Week-1 retention +24.1%, and paid conversion +45%. Targeted A/B tests confirmed that clearer interface logic, smoother UX and adaptive branching increased meaningful study actions and reduced friction. At the same time, the outcomes highlight the delicate balance between autonomy and guidance, personalisation can strengthen motivation but requires careful scaffolding to avoid fragmentation. From these findings, four design principles emerged: always direct effort toward meaningful early wins; provide adaptive guidance that balances freedom with direction; treat credit systems transparency as a dynamic hypothesis to iterate on; and view onboarding as a continuous, testable process rather than a static entry flow. Together, these contributions extend existing FTUE literature by demonstrating how adaptive, data-driven design can foster trust, comprehension, and long-term engagement in educational web platforms. Beyond Flashka, the results outline a replicable framework for evaluating and improving first-run experiences in AI-enhanced consumer applications.
Questa tesi esplora come un design adattivo e basato sui principi comportamentali possa migliorare la First-Time User Experience [FTUE] in applicazioni web educative. Attraverso il caso studio di Flashka.ai, piattaforma di studio che migliora la preparazione di esami universitari con IA, il lavoro analizza in che modo usabilità, onboarding personalizzato e meccanismi di trasparenza influenzano l’attivazione iniziale, la early-retention e il coinvolgimento dell’utente. La ricerca adotta un approccio misto qualitativo e quantitativo, includendo valutazioni CES, cognitive walkthrough e user test con studenti alla prima esperienza con la piattaforma. Dalle evidenze emergono nove requisiti di design, tradotti in una completa riprogettazione dei flussi di onboarding, generazione delle flashcard e studio. Le modifiche sono state valutate tramite metriche comportamentali confrontando due gruppi: pre-implementazione e post-implementazione, attraverso esperimenti A/B su utenti reali. I risultati mostrano che personalizzazione e chiarezza dell’interfaccia incidono in modo significativo sul comportamento: Day-1 retention +12.6%, Week-1 retention +24.1%, conversione a utente pagante +45%. Tuttavia, emerge la necessità di bilanciare autonomia e guida per evitare frammentazioni dell’esperienza. Dall’analisi derivano quattro principi di progetto per FTUE efficaci: orientare lo sforzo verso early first wins, fornire una guida adattiva alle preferenze dell’utente, trattare la trasparenza di sistemi di crediti IA come un’ ipotesi dinamica da verificare nel tempo e continuare a considerare l’onboarding come processo continuo e sperimentabile. Nel complesso, la tesi contribuisce alla letteratura sull’onboarding dimostrando come un design adattivo, informato dai dati, possa rafforzare fiducia, comprensione e coinvolgimento a lungo termine nelle piattaforme web in ambito educativo.
Designing and evaluating first-time user experiences: the case study of Flashka.ai, a web-based educational platform
DE MARCHI, SIMONE
2024/2025
Abstract
This thesis investigates how adaptive and behaviourally informed design can improve the first-time user experience [FTUE] in web-based educational applications. Using Flashka.ai, an AI-powered study platform, as a case study, it explores how usability principles, personalised onboarding, and transparency mechanisms influence early activation, retention, and engagement. The research combines qualitative and quantitative methods, including CES evaluation, a cognitive walkthrough and user testing with first-time learners. From these analyses, nine design requirements were defined and translated into a complete redesign of Flashka’s onboarding, flashcard generation, and study flows. Each intervention was evaluated through measurable behavioural outcomes across two user cohorts: before and after implementation, via A/B testing on product data. The results show that adaptive onboarding and usability refinements can meaningfully shape user behaviour. Retention and monetisation improved substantially: Day-1 retention +12.6%, Week-1 retention +24.1%, and paid conversion +45%. Targeted A/B tests confirmed that clearer interface logic, smoother UX and adaptive branching increased meaningful study actions and reduced friction. At the same time, the outcomes highlight the delicate balance between autonomy and guidance, personalisation can strengthen motivation but requires careful scaffolding to avoid fragmentation. From these findings, four design principles emerged: always direct effort toward meaningful early wins; provide adaptive guidance that balances freedom with direction; treat credit systems transparency as a dynamic hypothesis to iterate on; and view onboarding as a continuous, testable process rather than a static entry flow. Together, these contributions extend existing FTUE literature by demonstrating how adaptive, data-driven design can foster trust, comprehension, and long-term engagement in educational web platforms. Beyond Flashka, the results outline a replicable framework for evaluating and improving first-run experiences in AI-enhanced consumer applications.| File | Dimensione | Formato | |
|---|---|---|---|
|
Designing & Evaluating_Adaptive First-Time_User_Experiences.pdf
accessibile in internet per tutti a partire dal 18/11/2026
Dimensione
15.85 MB
Formato
Adobe PDF
|
15.85 MB | Adobe PDF | Visualizza/Apri |
I documenti in POLITesi sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/10589/247466