@tw_killian 😀 Happy to hear that you like it! ODE - base case. A continuous time version of a ResNet. CDE - when you add in a time-varying input. A continuous time version of an RNN. SDE - when you want a generative model; think of this like a GAN. Noise goes in, sample comes out. Tweet added by Patrick Kidger @PatrickKidger

Patrick Kidger

2 years

@tw_killian 😀 Happy to hear that you like it! ODE - base case. A continuous time version of a ResNet. CDE - when you add in a time-varying input. A continuous time version of an RNN. SDE - when you want a generative model; think of this like a GAN. Noise goes in, sample comes out.

1

6

39

Taylor W. Killian

@tw_killian

2 years

I’ve been slowly and surely making my way through @PatrickKidger ’s thesis (it’s remarkable btw): While I’ve been learning a lot I’ve found it hard to know which type of NDE I want to use. Does anyone know of a clear taxonomy between ODEs, SDEs and CDEs?

On Neural Differential Equations

The conjoining of dynamical systems and deep learning has become a topic of great interest. In particular, neural differential equations (NDEs) demonstrate that neural networks and differential...

arxiv.org

6

28

157

Patrick Kidger

@PatrickKidger

2 years

@tw_killian So for example, a CDE may be used to model a function of a time series / path. SDEs can be used to model the distribution time series/ path itself. Here's another explanation I like giving. Between ODEs amd SDEs there are actually ... 2/n

1

0

5

Patrick Kidger

@PatrickKidger

2 years

@tw_killian ...three differences: (1) a control; your diffeq changed its output based on the Brownian input. (2) roughness: Brownian motion is not a bounded variation process. (3) stochasticity: your input Brownian path is random. This is despite only (3) being reflected in the name! 3/n

1

0

3

Replies