Live Improvisation with Fine-Tuned Generative AI: A Musical Metacreation Approach

Misagh Azimi; Mo H. Zareei

Live Improvisation with Fine-Tuned Generative AI: A Musical Metacreation Approach

Abstract:

This paper presents a pipeline to integrate a fine-tuned open-source text-to-audio latent diffusion model into a workflow with Ableton Live for the improvisation of contemporary electronic music. The system generates audio fragments based on text prompts provided in real time by the performer, enabling dynamic interaction. Guided by Musical Metacreation as a framework, this case study reframes generative AI as a co-creative agent rather than a mere style imitator. By fine-tuning Stable Audio Open on a dataset of the first author’s compositions and field recordings, this approach demonstrates the ethical and practical benefits of open-source solutions. Beyond showcasing the model’s creative potential, this study highlights the model’s significant challenges and the need for democratized tools with real-world applications.