In this episode you can expect to learn about how AI can be used in live improvisation and the frontiers of musical AI tools such as Dicy2. 

Jérôme Nika is a researcher and electronic musician specializing in generative technologies that foster creative interactions between humans and machines. With an educational background from Télécom ParisTech and ENSTA ParisTech, he’s studied acoustics, signal processing, and computer science applied to music at Sorbonne University. Awarded for his innovative contributions during his PhD at IRCAM, he blends technology and art. As a permanent researcher at IRCAM since 2020, Nika collaborates on numerous artistic projects, continuously pushing the boundaries of digital music creation.

[Automatic captions by Autotekst using OpenAI Whisper V3. May contain recognition errors.]

[SPEAKER_00]: Welcome to the podcast Artificial Art.

[SPEAKER_00]: My name is Steinar Jeffs.

[SPEAKER_00]: I'm a musician and a music teacher.

[SPEAKER_00]: And in this podcast, I'll be interviewing guests about technology and creativity.

[SPEAKER_01]: So Jerome Nika is a researcher and electronic musician specializing in generative technologies that foster creative interactions between humans and machines.

[SPEAKER_01]: With an educational background from Telekom ParisTech and Ensta ParisTech, he studied acoustic signal processing and computer science applied to music at Sorbonne University.

[SPEAKER_01]: Awarded for his innovative contributions during his PhD at IRKAM, he blends technology and art.

[SPEAKER_01]: As a permanent researcher at IRKAM since 2020, Nika collaborates on numerous artistic projects, continuously pushing the boundaries of digital music creation.

[SPEAKER_01]: Did that sound okay?

[SPEAKER_01]: It's totally okay.

[SPEAKER_01]: Thank you.

[SPEAKER_01]: Nice.

[SPEAKER_01]: So my first question to you is what inspired you to delve into the field of generative music technologies and how has your journey shaped your artistic and professional outlook?

[SPEAKER_02]: So I think that what motivates me is the finality, the goal is to try to invent new practices and in particular practices that involves several people playing music together and what are the instruments, the new generation of instruments that we can create, that we can invent, that would enable persons to

[SPEAKER_02]: play music together and to have two human musicians playing the first one using an acoustic instrument for instance for example and the other one using a computer and to have a third musical discourse that will that would be created by this interaction so uh for me technology and ai in particular is really uh

[SPEAKER_02]: a tool to be able to, or something that we can use to be able to create this kind of process.

[SPEAKER_02]: So my goal is not to use technology to make music, but the goal is what can be done to emphasize and to empower interaction between two human musicians.

[SPEAKER_02]: And I think that with these kind of technologies and tools, we have a lot of direction that can be explored.

[SPEAKER_01]: Did you start out playing music or did you start out with the data side of things?

[SPEAKER_02]: I studied both computer science and signal processing on one side and music on the other side in conservatoire and I studied mostly composition so I was not that much into playing instruments and it may sound surprising because

[SPEAKER_02]: A large part of the musical project that I was involved in was about improvisation.

[SPEAKER_02]: But this is not that paradoxical because what I like about this AI and technological tools to create music is precisely the compositional aspect of it, even when we work in an improvised context.

[SPEAKER_02]: When we create music with musicians,

[SPEAKER_02]: We have these moments on stage when there is improvisation and interaction of course, but the most interesting part for me, using this kind of tools, is the whole process of composing the instrument, composing the behavior of the instrument that we do before in studio.

[SPEAKER_02]: And it underlines another aspect of what I'm really interested about in this field of research is the reflexivity that music with technological tools involves.

[SPEAKER_02]: Because when you want to communicate to a machine, to a system, what are the musical dimensions, directions that you want to explore, it makes you think and formalize about

[SPEAKER_02]: What is your music about?

[SPEAKER_02]: What is your project about?

[SPEAKER_02]: And this is the opposite of trying to find something that would be plug and play.

[SPEAKER_02]: When you want to use this kind of tools, you need to work even more because part of the job is to build a new instrument.

[SPEAKER_01]: And during your work, you have made such a tool or multiple tools even.

[SPEAKER_01]: And one of these tools are called Dicey2.

[SPEAKER_01]: Could you explain why you made it and also how it works?

[SPEAKER_02]: Yeah, so DAISY 2 is the current state of a long genealogy of tools for creating music with computers that was born in IRCAM around

[SPEAKER_02]: 2004 2005 something like that so the very first step of this research was just to create an engine a first system that would be able to be fed with music and then generate new musical contents that would be both

[SPEAKER_02]: novel that wouldn't be the same that the one it was fed with but keeping something of the internal logics of the music that was used at the corpus to train the system so this was great it was possible to generate something that was both new and keeping something of the aesthetics of what it

[SPEAKER_02]: learned on but then the challenge was how to turn this into an instrument how to control this generativity and how to metaphorically speaking how to add keys to this instrument so that on one side composers wanting to

[SPEAKER_02]: specify temporal evolutions to generate material or on the other side performers that would need something that would react to what they play could guide the way you travel through this

[SPEAKER_02]: musical memory.

[SPEAKER_02]: So the reason why DIC2 was born was how to control, how to, once again, to add the keys to these kind of generative engines.

[SPEAKER_02]: And the way it works is that the main, let's say the main, uh, uh,

[SPEAKER_02]: The core notion of the IC2 is the articulation between a memory, that is to say, a model that is built from musical sequences that we fed the system with on one side, and a scenario.

[SPEAKER_02]: What I call a scenario is a temporal evolution, a temporal specification of the music we want that we meet the...

[SPEAKER_02]: It's a temple specification of the music that we want the system to generate.

[SPEAKER_02]: And if we provide, I don't know,

[SPEAKER_02]: a long-term curve or a chord progression, a fixed scenario, then it can be used in studio for composers to generate things that will match the scenario using the memory.

[SPEAKER_02]: And if we put it into time, then the idea would be

[SPEAKER_02]: how to infer scenarios from the live analysis of musicians playing with us.

[SPEAKER_02]: So what is interesting in the interactive or reactive part

[SPEAKER_02]: is the fact that each time a musician is playing, the system thinks that, okay, this is what I listen from the musician.

[SPEAKER_02]: So what I should generate for the next 10 seconds is this, but it will never unfold the 10 seconds because then it would listen to something else from the musician.

[SPEAKER_02]: And the idea is to spend time

[SPEAKER_02]: your time refining and rewriting anticipations so that we have this directionality that music needs to be involving.

[SPEAKER_01]: Yeah, so it sounds like it's something that's a bit inspired by how the human brain and creativity in musicians work.

[SPEAKER_01]: At least it's a common thing for music teachers to say to the students that

[SPEAKER_01]: In order to create a sound and be creative, you should copy the music that you love and make that into your bag of tricks, sort of.

[SPEAKER_01]: And that becomes your musical memory, which you then lean upon to make your own stuff.

[SPEAKER_01]: Would you say that's a bit analogous to how Dicey 2 works?

[SPEAKER_01]: And you could use kind of a short-term memory as well in live performances.

[SPEAKER_02]: Yeah, totally.

[SPEAKER_02]: We never claim that we model anything in the human behavior, but we talk a lot with musicians.

[SPEAKER_02]: There is a very important part of this work that is

[SPEAKER_02]: even more about ethnomusicology with interviews, feedback, etc.

[SPEAKER_02]: And let's say that what we get from the musicians when discussing with them and the kind of thing you were talking about when a teacher talks about how to create music with the students, this is not things that we claim that we model, but that we use as metaphors, at least, to build the system.

[SPEAKER_02]: So

[SPEAKER_02]: As you said, we do not, with DIC2, model what a short-term memory is, we do not model how the anticipation works, but the fact that we address these issues comes from a metaphorical inspiration that we derive from the discussion that we have with the musicians.

[SPEAKER_01]: And so far in your experience with the Dicey 2, what has been the most interesting musical results in your personal opinion?

[SPEAKER_01]: What kinds of things?

[SPEAKER_02]: Yeah, the kinds of things would be the projects that

[SPEAKER_02]: use this tool to be able to walk on the thin frontier between idiomatic music and non-idiomatic music.

[SPEAKER_02]: I mean music that respects, that has to respect some musical language because it is jazz, it is baroque music, it is electronic music.

[SPEAKER_02]: but that uses it to be at the frontier between the aesthetics of this genre while making it strange, making it bizarre.

[SPEAKER_02]: And this is exactly the musical scope that we want to address.

[SPEAKER_02]: Because why...

[SPEAKER_02]: would a musician use a new type of tool, a new type of instrument?

[SPEAKER_02]: Bernard Luba, with a famous French musician I worked a lot with during my PhD thesis, always said, if it's about playing with credible and normal piano sound, piano, piano,

[SPEAKER_02]: I should just call a pianist that I know and why bother spending time with researchers and electronic musicians to get something that is, even if it's virtuosic or whatever, why spend time playing with electronic musicians and researchers?

[SPEAKER_02]: And on the opposite, of course, it's a kind of provocation what I say, but if it's totally random and you cannot make it fit a musical aesthetics that you want to address,

[SPEAKER_02]: it won't be of use either so when the projects that i'm i'm passionate about i'm thinking about things that i've done with remy fox who is a saxophonist and composer and from paris and steve leeman who is also play saxophone and composer from los angeles and in the work that i did with them uh with the same tools

[SPEAKER_02]: uh we you have it starts with a regular beat and something that is that belongs to the jazz or electronic music genre and then it digresses and it's still jazz and it's

[SPEAKER_02]: still acceptable regarding the idioms but becoming strange and going back and forth experiment and experimental aesthetics while being okay with the with the genre we're in and this is what I like about it why should we use these instruments

[SPEAKER_02]: to address this kind of frontier?

[SPEAKER_02]: I think my answer was way too long.

[SPEAKER_01]: I thought it was a good answer.

[SPEAKER_01]: A really short summary might be that why use a tool that sounds normal or something you could get from just playing a regular instrument?

[SPEAKER_01]: And on the other side, why would you use a tool that just sounds weird and random?

[SPEAKER_01]: So the coolest projects are in the intersection between those two or the thin frontier, as you said.

[SPEAKER_01]: Do you see a way for this tool to be used in popular music in more of a not experimental way?

[SPEAKER_02]: This is exactly the thing that I'm addressing in my current research right now.

[SPEAKER_02]: This is a long-term research process and the main musical application that we had for the moment was about contemporary music

[SPEAKER_02]: free improvisation because first we wanted to address the discursive aspects of music how to interact how to how to how to take the lead how to react when the person you're playing with is doing this or that in terms of of dialogue without focusing on that much on

[SPEAKER_02]: harmony melody beat because this is something that can be addressed using other techniques than machine learning and stuff and since now i think we got to a interesting point on these aspects this is precisely what i'm addressing right now and a lot of projects that are ongoing are about uh

[SPEAKER_02]: popular electronic music, rock music, with songwriters or hip-hop musicians.

[SPEAKER_02]: And then this is a very interesting challenge, that is to say, and it is linked to what I was saying previously, what can be done

[SPEAKER_02]: if it's not about creating a polite bass line that works with the piano chords, or creating any other example.

[SPEAKER_02]: So yes, this is exactly the current goal.

[SPEAKER_02]: And it can already be used in this way, but there are a lot of things to explore to make it even more interesting.

[SPEAKER_01]: And in Dicey 2, I've seen some videos on YouTube demonstrating it with the saxophonist you were talking about, Steve Lehman, and Remy Fox, was it?

[SPEAKER_01]: And it seems it has different modes you could put it in to either react in terms of chords or in terms of sound or in terms of noise, intensity.

[SPEAKER_01]: Stuff like that.

[SPEAKER_01]: Could you explain a bit about that?

[SPEAKER_02]: Yes, this is what I call composing the behavior of the instruments because, so to be just a bit technical, we have two versions of Dicey 2, the one that is a plugin for Ableton Live and the one that is a library for more advanced users.

[SPEAKER_02]: And with more or less details, the process to use it is the following.

[SPEAKER_02]: What is the material that I want to use to create music?

[SPEAKER_02]: So the thing that I want to fit the system with.

[SPEAKER_02]: How do I want the system to listen to the live environment?

[SPEAKER_02]: Do I want to listen and to react to each note that my fellow musician will play?

[SPEAKER_02]: Or to react to an average feeling of what the musician plays?

[SPEAKER_02]: Then what are the musical dimensions that I'm interested in?

[SPEAKER_02]: Do I want to listen, to react to melody, to harmony, to energy?

[SPEAKER_02]: And then from this perception, what is my musical intention?

[SPEAKER_02]: Do I want to match?

[SPEAKER_02]: Do I want to match sometimes and digress sometimes too?

[SPEAKER_02]: So these four or five

[SPEAKER_02]: parameters that I just mentioned are the things that you can compose when using the IC2.

[SPEAKER_02]: And this way, when you work with it, first, you spend time thinking about it, formalizing your musical idea.

[SPEAKER_02]: So let's say, for instance, I want to use singing voices and I want to listen to the pitch and to the harmony that the saxophone will play.

[SPEAKER_02]: And I want that

[SPEAKER_02]: from a very sparse playing from the musician that will just give one note or two, then the system unfolds the singing voice around the same type of note.

[SPEAKER_02]: So this is the main process, composing the behavior and then play with it on stage.

[SPEAKER_01]: So that means if I'm a saxophone player and I use that example as you were talking about, I can get singing voices as kind of an accompaniment to my playing, which is based on the context of which notes I'm playing?

[SPEAKER_02]: Yeah, exactly.

[SPEAKER_02]: And you can also, with the same voice,

[SPEAKER_02]: during the same performance or at another moment, you can decide that you want to use this voice not as an accompaniment anymore, but as a second musical discourse, like to have a duet with a virtual singer.

[SPEAKER_02]: So this way you would change the parameters so that the system is not matching strictly

[SPEAKER_02]: or following strictly what you are doing so that the system could digress and be a little bit more autonomous and be inspired by what you play with your saxophone while being able to propose new directions and with other set of parameters you can also have something that is much more simple and easy

[SPEAKER_02]: That is something like an audio effect, like each time I play this kind of note, I just have a shallow halo of voice around the same note and that's all.

[SPEAKER_02]: The idea is to be able, with the same material and the same type of reaction, to navigate with a continuous cursor between a duet with the electronics, an accompaniment by the electronics, or an audio effect, let's say.

[SPEAKER_02]: And what is interesting to explore, musically speaking, of course, is all the states that are in between these points.

[SPEAKER_01]: Yeah.

[SPEAKER_01]: Another example I was just thinking about is if I play a chord progression and then I could get a suggestion for a continuation of the chord progression I'm playing.

[SPEAKER_01]: from Dicey and yeah what happens in between might then be that I have like three or four chords I play then I get another three chords and then that inspires me to make another three and then that inspires Dicey to make another three and there's a continuous loop of inspiration going

[SPEAKER_02]: Yeah, this is the kind of process, the kind of instrument that can be built using this toolbox, yeah.

[SPEAKER_01]: Sounds really cool.

[SPEAKER_01]: I have to learn Dicey 2 ASAP.

[SPEAKER_01]: So, one of your projects is with a group called Ex Machina, which is... This is the title of the project, yeah.

[SPEAKER_01]: It's a large jazz ensemble, I guess you could call it, which incorporates the use of this tool, the Dicey 2 tool.

[SPEAKER_01]: And what unique elements do generative technologies bring to the live performance of this band?

[SPEAKER_02]: So this project, so the title of the project is Ex Machina and it's carried by the French National Jazz Orchestra.

[SPEAKER_02]: and Steve Lehman, we talked about previously, who is a composer and musician from Los Angeles.

[SPEAKER_02]: And in this project, what we wanted to achieve was to use the same system and the same mechanisms, both for orchestration, that is to say, really to create spectral textures within the band, so some work that is more about composition,

[SPEAKER_02]: and really interactive and real-time interactions with improviders.

[SPEAKER_02]: So on the composition orchestration side,

[SPEAKER_02]: The thing that was brought by this technology was a shift of musical practice of orchestration.

[SPEAKER_02]: What I mean is the goal of Steve Layman and Frederic Morin, I created the electronic whizz, was to be able to add to the very rich sound of the orchestra

[SPEAKER_02]: some very precise spectral evolutions to match the perfect idea that they have of how the whole sound should be.

[SPEAKER_02]: So the process was for them to give me musical material

[SPEAKER_02]: on one side that they wanted to use as a repertoire to search and retrieve things and to give me parts of segments of scores on the other side so that we could articulate the score that they wrote with the sound that they wanted to add to this score.

[SPEAKER_02]: So this was the first part of electronics that was more about composition orchestration.

[SPEAKER_02]: The second part of electronic was the interactive part.

[SPEAKER_02]: So in this project, I was linked to six or seven soloists.

[SPEAKER_02]: And quite often in the piece as interludes or as solos during the, in the middle of a piece, the idea was to have the musician improvising.

[SPEAKER_02]: So it was really about improvisation, but improvising with a musical agent that was composed in terms of behavior beforehand.

[SPEAKER_02]: So for instance, I can,

[SPEAKER_02]: give the example of Fanny Menegos who plays flute and as the introduction of one piece by Steve Lehman, she first improvised this with a very huge orchestral mass that is

[SPEAKER_02]: used to fit the system and with her flute she is trying to find to to fight against this huge sound sometimes this sound listens to her sometimes it does not and she has the control over this

[SPEAKER_02]: sing but that is quite autonomous sometimes and then it derives on a more rhythmic interaction with a drum track that she controls with her flute and this is the introduction of the piece after something that is a solo that can last for five six minutes so a solo from her but a duo with the the system

[SPEAKER_02]: And this is an example of this interactive setup that was composed for the project.

[SPEAKER_01]: So in the start you mentioned that the composers had ideas for sounds or sound spectra they wanted in different parts of the score.

[SPEAKER_01]: What kind of words or metaphors did they use to describe the kind of sound they wanted?

[SPEAKER_02]: And they didn't need to find words because what I like to do with this kind of process is just for them to basically give me sound files that correspond to the texture that they want.

[SPEAKER_02]: So things that they recorded or improvisation that they made in other contexts or separate tracks from previous works of them.

[SPEAKER_02]: And the way the system can be used is precisely to use this thing that they give me just to give me a hint of what flavor it should have.

[SPEAKER_02]: to use this material and to put it into time to slice it to rearrange it so that it fits the new story let's say that they want to tell in this project so this is what is interesting too with this kind of practice and something that i will explore even more deeper in the coming months is precisely to find this situation

[SPEAKER_02]: where you can teach the system something that is really precise about the aesthetics that you want to get, but without having to formalize explicitly, I want this type of sound, etc., just giving an example.

[SPEAKER_02]: And this is a great way to work with musicians, because, of course, even when music is really formalized

[SPEAKER_02]: and is thought to be upstream, you can have a strong idea of what you want.

[SPEAKER_02]: But giving an example, saying, you know, in this part of the piece, I would like to have something that would follow this temporal evolution, but with the sound that would be like that.

[SPEAKER_02]: And providing a tool that allows you to work with this like that is what we try to do.

[SPEAKER_01]: So I'm thinking about in terms of the perspective of a pop producer, how one might use that tool and what the competitors are for that kind of tool in the AI market.

[SPEAKER_01]: I guess one obvious contender would be using like Suno or Udio, some generative software, and then prompt it to make something that sounds like,

[SPEAKER_01]: a song that you heard before.

[SPEAKER_01]: Let's say you're producing a song for a new artist and you figure out that the bridge of this song should sound like Bob Dylan or something.

[SPEAKER_01]: And then you would generate some content in the video and then sample that content and put it in your own production or maybe just re-record it.

[SPEAKER_01]: So could you see Dicey being used instead and also in a cooler way than what I just described?

[SPEAKER_02]: Let's say I don't think so because I don't want to.

[SPEAKER_02]: It's half a joke, half reality.

[SPEAKER_02]: I will just explain why.

[SPEAKER_02]: It's because the...

[SPEAKER_02]: the choice that we made was that we want to develop tools for musicians and musicians that have an

[SPEAKER_02]: part of exploration of research in their process and it does not mean that the music that they make at the end has to be experimental you have a lot of people working in what we can call mainstream pop music whose process is experimental who like to try things and experiment things

[SPEAKER_02]: And by experience, if you give tools such as Suno to these people, they won't be afraid, they won't say, oh my god, AI is killing the musicians, etc.

[SPEAKER_02]: But they just...

[SPEAKER_02]: and i'm talking about all the musicians but the typology i'm talking about they will just say okay this is great but i can't use it because there are not enough keys on these instruments i can prompt a thing but what i want to do is to be the author of my work

[SPEAKER_02]: And what we want to offer them is a new way to make music.

[SPEAKER_02]: So to give you an example of things that I'm doing right now with hip hop producers or pop song producers is to be able to say, OK, give me on one side a very basic drum part that is not interesting in terms of

[SPEAKER_02]: sound in terms of articulation but that has the basic rhythmic that you want and on the other side a much more complex sound musical structure etc and

[SPEAKER_02]: Let's have a look at the results that we could get using the first one as a structure and the other one as a texture.

[SPEAKER_02]: Or on the other side, something that is once again going towards the idea of shifting the musical practice.

[SPEAKER_02]: Let's say that you want to generate a bass line.

[SPEAKER_02]: OK, great.

[SPEAKER_02]: But thinking

[SPEAKER_02]: I'm trying to generate a baseline is one thing, but thinking, OK, I have

[SPEAKER_02]: the voice from the singer and I want to generate a bass line that would derive from, in a way, this voice from the singer that would analyze things from the singer and given some parameters that I'm giving, would create a bass.

[SPEAKER_02]: And this way, it creates a new kind of practice that is like, okay, what I want to get

[SPEAKER_02]: is a bass, but what I describe is the voice it should match with.

[SPEAKER_02]: And this is why I think the AC-2 won't be used as a concurrent of Suno or things like that, because what we want to explore is exactly this.

[SPEAKER_02]: I don't know if I'm clear about my answer.

[SPEAKER_01]: Yeah, I think so.

[SPEAKER_01]: You kind of make a distinction between AI as an engine instead of an instrument.

[SPEAKER_01]: And you kind of favor the instrument part of it.

[SPEAKER_01]: So it has more keys, as you say, so you can interact with it.

[SPEAKER_02]: And also, and once again, I'm just talking about the musical collaboration aspect.

[SPEAKER_02]: I made, and I'm not talking for the whole musicians, and I'm not saying music is like that, musicians work like that in general.

[SPEAKER_02]: But for this typology of artists that we're talking about, that is to say, persons that like to explore, to create new setups, et cetera, the prompt thing is not of use because

[SPEAKER_02]: the need or the want descriptions that are unfolding through time that are sequential descriptions saying I want further that than this and the temporal granularity that they need is at the level of let's say the beat or the measure but

[SPEAKER_02]: let's say the zoom is way too far from the music that will be generated at the end if it's about prompting in even with long sentences the temporal evolution so the main drawback for these kind of people once again and only for them is that prompting does not allow them to specify the time with a

[SPEAKER_02]: a grain that is fine enough.

[SPEAKER_01]: Yeah.

[SPEAKER_01]: With assume that's too far out.

[SPEAKER_01]: That was a nice way of putting it, I think.

[SPEAKER_01]: And that probably puts you on one side of the argument when it comes to the philosophical debate of AI replacing human creativity versus augmenting it, I guess.

[SPEAKER_02]: I don't know, because I think that this question is not precise enough and that something is missing.

[SPEAKER_02]: And it's not your question, of course, because this is the question.

[SPEAKER_02]: I'm not saying that your question is not the right one, but precisely the one that you underline.

[SPEAKER_02]: It's because AI does not want anything, the technology does not have any goal, and something that is striking for me is that we forget a variable in the equation being AI does not want to replace or won't replace if it's not something that the people developing it have in mind.

[SPEAKER_02]: In the community I work in, the idea is to build instruments to empower musicians and offer them new practices.

[SPEAKER_02]: So on this point of view, we do not want to do something that would be credible and that could address tasks that were previously handled by human musicians because we want to give them new tools.

[SPEAKER_02]: on the other side you have indeed companies or research labs whose goal is to be able to produce music in the style of this or that or to produce some designs in the style of this or that and this is indeed something that is right now done by

[SPEAKER_02]: musician, composers, and that is sometimes something that is important in their life, just in terms of incomes.

[SPEAKER_02]: And do I think that these things, that is to say,

[SPEAKER_02]: something that we can call algorithmic composition that was previously made by humans with stronger specifications.

[SPEAKER_02]: I think, indeed, and I fear, indeed, that these tasks will be, in the next future, will be achieved by these kind of systems.

[SPEAKER_02]: Yes.

[SPEAKER_01]: Do you think it will alter how musicians work and how they make a living?

[SPEAKER_02]: I think it won't alter the creativity, the disruption, because it's not about it, but I fear that it will alter indeed their way of living because maybe it's totally out of scope of the topic we're discussing, but musicians, most of them, do not only work

[SPEAKER_02]: on music and art they're passionate about, because it's hard to make one's living only exploring the music you want to explore.

[SPEAKER_02]: So you have to, as a musician, work on projects that are not yours, et cetera, or to compose music for this or this event.

[SPEAKER_02]: And I fear, indeed, that this part

[SPEAKER_02]: of this aspect of the life of some musician will be transferred to people using this kind of system without any musical expertise.

[SPEAKER_02]: So I don't know if it will be a lack in terms of creativity.

[SPEAKER_02]: I don't think so, because this is not where creativity lies.

[SPEAKER_02]: But yeah, I think that's, sorry to be so pragmatical, but that indeed this thing will, there will be less jobs

[SPEAKER_02]: or in this particular scope.

[SPEAKER_01]: I think so as well.

[SPEAKER_01]: I recently heard a story from a friend of mine who told that a musician he knows has a side gig, as you were talking about, where he makes lo-fi chill beats for studying.

[SPEAKER_01]: And he said he made like 7,000 euros a month on these lo-fi chill beats.

[SPEAKER_01]: And my initial thought was, in five years, I don't think that is a thing anymore.

[SPEAKER_01]: It's definitely going to be generated by large companies themselves, so they don't have to pay royalty anymore.

[SPEAKER_01]: That's interesting, as you said, that the parts of the...

[SPEAKER_01]: the job of being a musician which not necessarily are the most passionate the kind of side hustle kind of jobs those are the types of jobs that are first to go away so it will be interesting to see what musicians will make money doing

[SPEAKER_02]: And I hope that the companies or institutions that are developing or will develop these tools will have a strong ethics and will not use things that do not belong to them

[SPEAKER_02]: or I think created by musicians to train their models to be able to generate this and that.

[SPEAKER_02]: If they do, what I hope,

[SPEAKER_02]: uh we can hope that they hire musicians to i don't know to curate databases or to create them to create some examples that could be used to train their models uh i don't know if it will be the if it is the direction that is that is uh the one that is being taken right now but this is something at least that we can hope for them

[SPEAKER_01]: Yeah, and all the major music labels are suing the generative softwares at the moment.

[SPEAKER_01]: They're suing Suno and Udo.

[SPEAKER_01]: So I guess the outcome of those lawsuits will probably be important in dictating the future.

[SPEAKER_02]: Yeah, I hope, yeah.

[SPEAKER_01]: Well, yeah.

[SPEAKER_01]: I'll take us back on topic for a bit now.

[SPEAKER_01]: Because you do also have some music information retrieval projects, which I'm really interested about.

[SPEAKER_01]: So I was hoping you could tell us something about that.

[SPEAKER_02]: yeah so uh if the idea of our instruments is to be able to be composed in terms of behavior in terms of listening in terms of reaction to a live musician playing of course there is one very crucial topic which is the

[SPEAKER_02]: perception or how our instruments derive information from the audio that we receive from the saxophone playing with us for instance so the idea is to be able to analyze the life playing of the musician in a way that is interesting to them

[SPEAKER_02]: send queries to the system to generate something.

[SPEAKER_02]: Of course, we already have, and these are things that exist since 20, 30 years, some audio features, audio descriptors that we can use.

[SPEAKER_02]: It is totally known now how to find the pitch in an audio sound, how to find the

[SPEAKER_02]: harmony, the energy, etc.

[SPEAKER_02]: But maybe what we want to do is to go beyond matching.

[SPEAKER_02]: What I mean is if I listen to a musician playing with my machine and I know that this musician is playing this type of notes, this type of energy, etc.

[SPEAKER_02]: okay I can use this information to ask the system find something in your memory that matches this information or maybe we want to teach our system to

[SPEAKER_02]: be able to know that in a database that you give to it, maybe the best thing to do is not to find something that matches exactly what you listen to in real time, but to find something that

[SPEAKER_02]: works with it, that is complementary, that has the same properties on one side but is different on another musical aspect.

[SPEAKER_02]: So all the work in music information retrieval that is related on this topic is precisely about what are the relevant information that we can extract

[SPEAKER_02]: from the musician that is playing and is connected to the system?

[SPEAKER_02]: And also, what are the different hierarchical levels in terms of temporality that we can extract?

[SPEAKER_02]: Because it's important to know that

[SPEAKER_02]: I don't know that right now the musician is playing C sharp very loud, but maybe it's important to use information that are not only about the note that the musician is playing right now, but the long term evolution in which this particular note belongs.

[SPEAKER_02]: So let's say that all the work

[SPEAKER_02]: of music information retrieval is about what type of information is relevant to then translate this perception into an intention that can be used to specify to the machine what we want to generate.

[SPEAKER_01]: So it sounds a bit like ear training.

[SPEAKER_02]: in a way an ear training and since the thing that we have in mind each time we work on a module is not only how to achieve that but how to achieve that and provide a workflow to compose the way that could be achieved the main question is okay how to do here training and also how to propose to a musician a workflow

[SPEAKER_02]: that will allow the musician, by giving some audiophiles or by demonstration, that would allow the musician to tune the way the system would train its ear.

[SPEAKER_01]: What kind of applications could this be used to?

[SPEAKER_01]: I'm not sure I get your question.

[SPEAKER_01]: Yeah, the music information retrieval you're developing.

[SPEAKER_01]: Do you use that in, for instance, Dicey 2?

[SPEAKER_02]: So what is implemented in Dicey 2 now is more traditional, let's say, single audio processing.

[SPEAKER_02]: That is to say, the main information that we use from the outside world that the system listens to are traditional audio features that then we

[SPEAKER_02]: we use in specific ways of course, but the thing I was mentioning just before is one current research topic

[SPEAKER_02]: And just to give you an example, we have a PhD student that will start on September about this question of what can be achieved in terms of tuning your instruments if instead of providing

[SPEAKER_02]: one sound file or one database that would be the memory of your system if we provided two one that would be your memory the other one that could be a kind of annotating data sets so that in addition to learning how to generate music from your memory the system could learn

[SPEAKER_02]: what are the relationships between two musical discourses that you want to get.

[SPEAKER_02]: To give you a very basic example,

[SPEAKER_02]: For the moment with Dicey 2, if I want to play a piano that is controlled by the saxophone, the saxophone is playing, I try to find information in the live saxophone playing that I can find also in the piano and then generate something from it.

[SPEAKER_02]: The idea would be, now if you want to generate piano controlled by your saxophone, you provide

[SPEAKER_02]: a piano track, but also a saxophone track, so that the system could learn in an informal way what are the associations, the relationship that the saxophone and the piano should have.

[SPEAKER_02]: This way, when your saxophone will play, you would not seek into your piano something that matches what the saxophone plays, but something that has the same kind of relationship that the one you demonstrate

[SPEAKER_02]: while giving your two tracks?

[SPEAKER_01]: So, potentially could that mean that I could analyze the relationship between Tony Williams, the drummer, and Miles Davis, the trumpet player, and use the way Tony Williams is reacting to Miles Davis' playing to inform some other aesthetic?

[SPEAKER_02]: you explain it way better than I do.

[SPEAKER_02]: This is exactly the thing that we want to achieve.

[SPEAKER_02]: And for us, it's a way to add another layer of composition and not another layer of autonomy from the system.

[SPEAKER_02]: Because we're thinking about the user that would use it and that would have this other option for composing the musical discourse that would be, oh,

[SPEAKER_02]: I want the relationship between this instrument and this instrument having the same flavors, the same spirit that the one that is in these couple of tracks or etc.

[SPEAKER_01]: That has at least been my dream scenario when it comes to working with AI software.

[SPEAKER_01]: I could then play with dead heroes or alive heroes that I might not have contact with.

[SPEAKER_01]: And also use the decision making to guide other kind of aesthetic choices in the music.

[SPEAKER_01]: That would be really awesome.

[SPEAKER_01]: Do you have any dreams of where this kind of research could go in the future?

[SPEAKER_01]: I mean, in five, let's say five or 10 years or something, do you see yourself working on something in your dreams?

[SPEAKER_02]: I will be totally honest with you.

[SPEAKER_02]: I never have ideas all alone by myself.

[SPEAKER_02]: I really need and this is also something that I claim and this is what I want my work process to be.

[SPEAKER_02]: I don't

[SPEAKER_02]: want to have long-term directions that I invent all alone in my lab and then try to force it in musical productions.

[SPEAKER_02]: 99% of the motivations or the directions that I developed, they come from

[SPEAKER_02]: okay we are working for this concert on next week we know that we cannot achieve this because we only have five days but what would be great for the next concert that we may do in two months would be to have this or that and all the instruments we discussed previously

[SPEAKER_02]: The motivations and the keys, once again, to add to these instruments, they come from this performance-led research, this practice-based research.

[SPEAKER_02]: So, no, I don't have this long-term plan.

[SPEAKER_02]: sorry if this is not a very satisfying answer but what is sure is that I want to dig deeply into this even more into this question of working at the frontier between idiomatic and non-idiomatic music and to find a way to be able to address with

[SPEAKER_02]: the same models uh free improvisation and contemporary creation on one side and um yeah more mainstream as we say so just keep digging deeper and we'll see what happens i guess

[SPEAKER_01]: yeah to be honest that's nice so I have one last question for you Jerome which I ask all my guests basically and that is what advice would you give to to young people or emerging artists who are interested in integrating technology into their creative processes the advice would be

[SPEAKER_02]: don't try not to try to make everything with these with your with the instrument that you're trying to build what i mean is that there is no moral and there is no ethics it's just about creating music and if you're engaged in an artistic process that involves an instrument that you're building

[SPEAKER_02]: Maybe for the good of this musical project, at the end of the day, this instrument will be used in one section of two minutes and not on the one hour long creation.

[SPEAKER_02]: And to be clear and honest about that and not trying to

[SPEAKER_02]: to change the project and to change the direction so that you can explore all the possibilities of your instrument for the entire hour would be on in the long term much more interesting because and this is something that is sometimes quite long to understand I know that

[SPEAKER_02]: On my side, it's already happened once that I spent one night thinking, okay, I don't know how to do with this tool to be able to play this exact sample at this exact moment.

[SPEAKER_02]: I don't want it to be changed or whatever.

[SPEAKER_02]: And just to realize in the morning that if I want to do that, to play a sequence that already exists,

[SPEAKER_02]: I don't have to use my system I can just use a sampler and you know it may sound stupid but this is about be clear about what is the practice that you want to create or what is the playing modes that you want the new playing modes that you want to offer and being clear about this is the best way to do something that will be of interest I think

[SPEAKER_01]: So it's like when a guitar player or bass player gets a new effect pedal and they suddenly start to use that on every song.

[SPEAKER_02]: Exactly.

[SPEAKER_02]: This is the exact same thing.

[SPEAKER_01]: So it's been a great honor to have you on the podcast, Jerome.

[SPEAKER_02]: Thank you for the invitation.

[SPEAKER_02]: I was really happy.

Thank you.