AI-generated music is already an innovative enough concept, but Riffusion takes it to another level with a clever, weird approach that produces weird and compelling music using not audio but images of audio.
Both sounds and are odd. But if it functions, it functions. And it does function! Sort of.
A machine learning method for creating pictures called diffusion has recently propelled the field of artificial intelligence.
The two most well-known models that function by progressively substituting visual noise with what the AI believes a prompt should look like are DALL-E 2 and Stable Diffusion.
The approach has shown to be effective in a variety of situations and is particularly adaptable to fine-tuning, which involves feeding the heavily trained model a lot of a particular sort of information to have
It focuses on creating additional instances of the material. If you adjusted it for watercolours or automotive shots, for example, it would be more successful at duplicating those kind of images.
For their side project Riffusion, Seth Forsgren and Hayk Martiros improved Stable Diffusion on spectrograms.
"Hayk and I are in a little band together, and we began the project only because we love music and were unsure if Stable Diffusion would be able to produce a spectrogram picture.
According to Forsgren, the conversion to audio had sufficient quality. The possibilities have continually astounded us along the process, as one thought feeds into the next.
According to Forsgren, the conversion to audio had sufficient quality. The possibilities have continually astounded us along the process, as one thought feeds into the next.
'Riffusion' is a computer programme that creates music by picturing it.