We show that existing policy gradients introduce bias in the random time horizon setting. Our corrected formulas fix this, leading to better convergence and improved performance in practice.

In many real-world reinforcement learning tasks - like robotics, dialogue, or trading - episode lengths vary randomly. Yet, most policy gradient methods assume a fixed and deterministic or infinite time horizon. We close this gap with a principled derivation for random times.

We derive policy gradients for reinforcement learning with random time horizons in While arguably being a typical setting in applications, it has been largely overlooked in the literature. Our adjusted formulas offer significant numerical improvements.

I'm especially looking forward to meeting my fantastic collaborators @julberner, @DenBless94, @junhua_c and of course many other friends.

On my way to #ICLR2025 in Singapore. We will present.• Sequential controlled Langevin diffusions, Thu, 10am,.• Underdamped diffusion bridges with applications to sampling, Sat, 10am + oral at FPI workshop, Mon, 10am. Let's meet!

Our new work extends the theory of diffusion bridges to degenerate noise settings, including underdamped Langevin dynamics (with @DenBless94, @julberner). This enables more efficient diffusion-based sampling with substantially fewer discretization steps.

My friend @daspantom is looking for interns working on generative modeling for proteins. Let him know in case you are interested.

RT @papercopilot: Also on open/close peer review, I analyzed the AI/ML community's interest by analyzing user activity on open statistics a….

Thanks to @FelineAutomaton, @julberner, @MarcinSendera, @jarridrb for the great collaboration!.

Our new work studies connections between discrete and continuous time diffusion samplers - after all, most things are very much related (GFlowNets, optimal control, path measures, PDEs). This allows us to reach faster convergence by randomized time steps.

Check out our new paper on diffusion-based sampling, combining diffusion models with Sequential Monte Carlo in a principled way, It improves sampling quality and leads to more robust training. Thanks @junhua_c, @julberner, @DenBless94 for the great work!

Our work from last year on solving high-dimensional PDEs with tensor trains via robust BSDE-based methods has just appeared in @JmlrOrg. It's a follow-up of and suggests a new loss that is robust and explicit, i.e. fast.

@icmlconf You can find the paper at

Come to our poster tomorrow, Thursday, 11:30, #411, at @icmlconf, where we present to you the 𝐄𝐡𝐫𝐞𝐧𝐟𝐞𝐬𝐭 𝐩𝐫𝐨𝐜𝐞𝐬𝐬, which for the first time allows to directly link discrete and continuous state-space diffusion models. See our recorded video:

And back to Vienna again. I will present our work on time-continuous discrete-space diffusion models at @icmlconf (with @ludiXIVwinkler, Thu, 11:30). For the first time, we can connect those models to score-based generative modeling, see Ping me!

I gave a talk on our latest work on the connections between dynamical systems, PDEs, control and path space measures for sampling from densities at the @FieldsInstitute in Toronto last week (with @julberner, @Jeff_J_Sun). You can find the recording here:

I will give at talk about our latest work on PINN-based sampling from densities via SDEs and ODEs (with @julberner and @Jeff_J_Sun) at the Fields Institute in Toronto today. Check out the paper at

I gave a presentation on our latest diffusion-based sampling work virtually at @turinginst yesterday. The work has been mostly done jointly with @julberner and Nik Nüsken. You can find the recording here:

For the first time, we can directly link state-discrete continuous-time diffusion models to their time- and space-continuous (SDE-based) counterparts, i.e. score-based generative modeling. Credits go to the Ehrenfest process (+ @ludiXIVwinkler & Manfred):

Especially looking forward to meet my collaborators @julberner, @daspantom, @ludiXIVwinkler, @LVaitl, @guanhorng_liu and proud to represent @ZuseInstitute and @dida_ML.