Been seeing Cat Olympics recently, but there was no story. So I added GenAI sound effects. SFX are not just there to “match” what you see. They craft a story: change the sound, and drama becomes comedy. AI sound design with creative controls = you control the storytelling 🥇

Would love to see your own creations in this thread 👇. Here’s one from @karenxcheng I thought was super cool.

Can GenAI keep the performative aspect of sound design? @Adobe Firefly lets you perform with your voice and turn it into any sound effect!. Try it out: Model by our SODA group at @AdobeResearch : @pseetharaman @urinieto & our incredible intern @hugggof

To learn more please check out our ICML'25 paper: "FLAM: Frame-Wise Language-Audio Modeling". Big congratulations to @wuyusongwys, @tsirigoc and everyone on the team. So excited to share this work, which solves a long standing problem in audio research!.

FLAM is trained jointly on instance (global) and frame-wise (local) objectives. The secret sauce: A memory-efficient and calibrated frame-wise objective with logit adjustment to address spurious correlations, such as event dependencies and label imbalances during training

Enter FLAM: Frame-Wise Language-Audio Modeling. A model trained to produce a calibrated likelihood for *any* text prompt. FLAM outperforms prior self-supervised models on both closed-set and open-set SED, while preserving strong retrieval and zero-shot classification accuracy

Our goal is for the model to detect *any* sound via free form text queries. "So use CLAP", some of you will say. The problem is its output likelihoods are not calibrated for different prompts :(. That's ok ranked retrieval, but for detection it's a no go.

Sound Event Detection models, ie finding sounds in audio/video recordings, are typically constrained to a predefined "closed" set of sounds, like in this (old!) model below for urban sound detection. It has some applications, but it doesn't address general purpose sound search.

I think we finally cracked it? FLAM can detect *any* sound via text prompts. arXiv (ICML'25): demos: @AdobeResearch+@MIT+@Mila_Quebec led by @wuyusongwys w/@tsirigoc @Kotentorothy @huangcza @AaronCourville @urinieto @pseetharaman

Generative Extend in Premiere Pro just won *five* awards at NAB 2025, including the @NABShow Product of the Year award! SODA, our group, created the audio GenAI model in charge of audio extensions in the feature. Couldn't be more proud of the team!. w/@urinieto @pseetharaman

RT @AdobeResearch: Generative Extend just launched in Premiere Pro—use #GenAI to extend video and audio for perfectly timed edits! This was….

Generative Extend just released in Premiere Pro! Use GenAI to extend your video *and audio* clips for a perfectly timed edit. The audio model was built by our team, the Sound Design AI (SODA) group at @AdobeResearch w/ @pseetharaman and @urinieto 🙌 .

We didn't expect this. our Sketch2Sound demo video has gone viral on IG with more than 5.2 million views 🤯. Amazing job @hugggof @pseetharaman @urinieto . I should've done my hair.

Really cool to see our AI for bioacoustics work in the MIT Technology Review! BirdVox was a fantastic project that beyond posing a fascinating research problem, introduced me to the world of bird watching and deepened my appreciation of the natural world.

Sketch2Sound is out!. Takes a text prompt + vocal (or sonic) imitation and generates sound effects that perfectly match the energy and dynamics of your voice. It's an extremely intuitive (and fun!) way to create SFX that are perfectly timed to your video. Led by @hugggof 👏.

I'm hiring a Research Engineer for my team at @AdobeResearch . Full details in the link I shared in the post below. DM me if interested.

More from our group: MultiFoley, a Video-to-Audio model that generates perfectly synced audio for video at 48 kHz and supports multimodal conditioning. You can define the generated sound via a text prompt, an example SFX, or audio to extend. More here:.

📢 Audio AI Job opportunity at Adobe!. The Sound Design AI Group (SODA) is looking for an exceptional research engineer to join us in building the future of AI-assisted audio and video creation. Strong ML background, GenAI experience a plus. Details:

Webpage: Paper: . w/ @CzyangChen, @pseetharaman, Bryan Russell, @urinieto, David Bourgin, @andrewhowens.

New model from the SODA group @AdobeResearch and UMich!. MultiFoley generates perfectly synced audio for video at 48 kHz and supports multimodal conditioning. Define the generated sound via a text prompt, an example SFX, or audio to extend. Led by our intern @CzyangChen 👇.