Made a walkthrough vid for Magenta RealTime “Audio Injection”! The notebook takes ~10m to spin up, but totally worth it for the surreal experience 🎤💻🎧⁉️

We built a web app that lets you fly a spaceship through a 3D constellation of music - powered by our Lyria RealTime model. 🎶 Space DJ is an interactive visualization where every star represents a different music genre. As you explore, your path is translated into prompts for

We’re just getting started with live music models. Can’t wait to see the possibilities for musicians of any skill level to craft, shape, layer their sound in real time.

Would love to see this combined with fine-tuned models for more personalized results.

Or sometimes it’s like a looper pedal with a mind of its own? Hard to describe, so try it out yourself! https://t.co/OfpAWnjz7w

Audio Injection lets you steer Magenta RT using *live* audio input. Doesn’t feel like jamming with a human. More like jamming along with the radio... but then the radio starts incorporating elements of your sound??

Everyone talks about real-time AI for speech but sleeps on it for music Real-time is nice to have for conversations, but essential for co-jamming

@jesseengel Hours and hours of fun and inspiration. Tiny excerpt: https://t.co/nz7fKxjwh4

🔊 Easy to lose hours to live jamming w/ Magenta RT. Feels like a glimpse at something magical Try it yourself! Some setup involved (we're working on improving that!) but worth it: https://t.co/eD7rK2lsfz

In 2018, we wrote a paper titled "the challenge of realistic music generation". A few years ago, this challenge was essentially solved: it is now a thing that exists🤯 Realtime is the next frontier, it opens up so many more possibilities. There's a tech report + open weights!🙏

I’m happy to have donated my voice to this incredible new open model and instrument! It is a new, active, approach to spawning music, made for real time jamming and exploration. More like a sampler than a jukebox, you can fine tune it on your own sounds and soon run it locally!

We've worked with @hollyherndon and @matdryhurst to finetune Magenta RT on Holly's vocals and the result is a really cool model that keeps the base model's ability to respond to wide-ranging controls while infusing outputs with the unique sonic qualities of the finetuning domain

Realtime interactive generative models FTW! Announcing a new 🌊 of details and features for Magenta RealTime, the open weights live music AI model from GDM! * Live Jamming with audio input 🎤🎸🎵 * Personalize your own models 🔧 * Tech report 📜 Links below in the 🧵...

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 Like other Lyria RT demos, you can mix together text prompts and other controls to steer the model in real-time. But now with a VST plugin you can feed audio

We’ve just released Magenta RealTime, an open-weights live music model that lets you craft sounds in real time by exploring the latent space through text and audio! 🤗 Model: https://t.co/MLBTuoXQYm… 🧑‍💻Code: https://t.co/WcQHDyHLuv 📝Blog post:

🎸🥁🎹🎵 New open weights music model from Magenta for real-time generation. Excited to see what cool instruments you all can make with it 🙂

Introducing Magenta RealTime, our open weights live music model! Like Lyria RealTime, this model provides the user continuous control of the musical flow 🌊. We will be sharing some of our favorite ways to extend these models over the next few weeks. Can't wait to see what you

Excited to announce 🎵Magenta RealTime, the first open weights music generation model capable of real-time audio generation with real-time control. 👋 **Try Magenta RT on Colab TPUs**: https://t.co/SofwwmWlxC 👀 Blog post: https://t.co/QTypsxNHvK 🧵 below

On the occasion of returning to Magenta's roots at @sonarplusd, we're dusting off the blog to share news and insights about what we're working on at @GoogleDeepMind on the Lyria Team. https://t.co/2yoqM5Pni4 Our latest post is about the Lyria RealTime API, providing access to

Pleasantly surprised with how easy/fun it is to "vibe code" new musical experiences in AI Studio. I'm not much of a JS guy, but had Gemini help me make a "Kaoss Pad" emulator using the Lyria RealTime API to explore musical latent spaces in realtime.