Neutone SDK: An Open Source Framework for Neural Audio Processing We’ve finally published a paper for the Neutone SDK which I presented at AES AIMLA 2025 a few weeks ago! arXiv: https://t.co/Vj4opVDyr4 code: https://t.co/L5E3J5Cb9g discord: https://t.co/s2OfqjyxCh @neutone_ai

Neutone、ついにハードウェア化しました！ @neutone_ai 昨日、Rolandさんと新プロジェクト Project LYDIA を発表。AIモデルを学習して載せられる DIYエフェクトボックス です。Morphoのリアルタイム音色変換を搭載し、AIを“触れる楽器”に! Roland Future Design Labの皆さんに感謝します。

New VST3/AU plugin drop to kick off the week! 🚀 Stable Audio Open Small by @StabilityAI Demucs stem separation LoopGAN for generating audio loops Many more models to come All running locally on-device! Opensource SDK and free plugin, enjoy! https://t.co/ndSnXbcA96

Check out the paper, plugins, and code for more details, and join the Discord server to stay up to date. Finally, a huge thank you to my collaborators at Neutone for the amazing work they’re doing!

At a time when many AI companies are competing with artists and training on their work without permission, the SDK democratizes this technology and provides a foundation for AI tools that enhance rather than replace human creativity. https://t.co/aClnV72fY0 (8/8)

To date, the SDK has powered a wide variety of applications such as neural audio effects, timbre transfer, sample generation, and stem separation, as well as seen adoption by researchers, educators, industry, and artists alike. https://t.co/AIHahigZeX (7/8)

Since early 2022, Neutone FX has made the latest realtime neural audio models accessible to artists around the world. It includes a model browser that allows one to search for and download user models that have been shared and uploaded to the Neutone servers via the SDK. (6/8)

We provide a technical overview of the interfaces needed to accomplish this, as well as the corresponding SDK implementations. Personally, I love prototyping neural audio models in Python with the SDK, and listening to the results in the DAW seconds later after exporting. (5/8)

By encapsulating common challenges like variable buffer sizes, sample rate conversion, and delay compensation within a model-agnostic interface, our framework enables seamless interoperability between neural models and host plugins while allowing users to work entirely in Python.

The Neutone SDK is an open source framework that streamlines the deployment of PyTorch neural audio models for both real-time and offline applications. It enables researchers to wrap their own PyTorch models and run them in the DAW using our free host plugins FX and Gen. (3/8)

We’re also releasing the beta version of Neutone Gen, the counterpart to Neutone FX that continues to bridge the gap between audio researchers and artists. Now, you can export heavy-weight, non-realtime models using the SDK and run them in the DAW via the free Gen plugin. (2/8)

We’re excited to release Neutone Gen, the counter-part of Neutone FX that continues to bridge the gap between AI Audio researchers and artists. Now, you can export heavy-weight non-real-time models using our SDK and run them in the DAW through the Gen plugin. ✨ More info ↓

Lastly, a huge thank you to my collaborator Hao Hao (@GoodGood014) and supervisor Josh (@IntelSoundEng) for their help and contributions!

Our code is open source ( https://t.co/eADt4ZbeWV) and the trained synths are available as VST plugins via the @neutone_ai platform and SDK. Listening samples, visualizations, plugins, and more can be found at https://t.co/Skt5L37HqR (7/7)

We evaluate our modulation discovery framework on unseen real-world modulation curves, highly modulated synthetic and real-world audio, and on white-box, gray-box, and black-box synth architectures. (6/7)

We investigate three modulation signal parameterizations: • Framewise (Frame) • Low-pass filtered (LPF) • Piecewise 2D Bézier curves (Spline) We find that LPF and Spline yield human-readable curves that trade sound-matching accuracy for interpretability. (5/7)

We apply our approach to a differentiable synthesizer inspired by the popular soft synths Serum and Vital with wavetable, filter, and envelope modulations. We also demonstrate its ability to generalize to other DDSP synth architectures. (4/7)

We propose a self-supervised neural sound-matching approach that leverages modulation extraction, constrained control signal parameterizations, and differentiable digital signal processing (DDSP) to discover the modulations present in a sound. (3/7)

Modulations are a critical part of sound design, enabling the creation of complex, evolving audio. However, finding the modulations in a sound is difficult and typical sound-matching / parameter estimation systems don’t consider the structure or routing of underlying modulations.

Modulation Discovery with Differentiable Digital Signal Processing This week I’ll be at @IEEE_WASPAA presenting our work on discovering synthesizer modulation signals in arbitrary audio. arXiv: https://t.co/qFNPVZdr9Y web: https://t.co/Skt5L37HqR @GoodGood014 @IntelSoundEng