Just released a new blog post on the recent approach that will likely supersede GANs in the soon future: Score Matching with Langevin Sampling by @YSongStanford. I explain how the approach works and its pros and cons. Tweet added by Alexia Jolicoeur-Martineau @jm_alexia

Alexia Jolicoeur-Martineau

4 years

Just released a new blog post on the recent approach that will likely supersede GANs in the soon future: Score Matching with Langevin Sampling by @YSongStanford . I explain how the approach works and its pros and cons.

The new contender to GANs: score matching with Langevin Sampling

Last year, Yang Song, a graduate student at Stanford, showed an entirely new way of generating data based on denoising score matching with Annealed Langevin sampling (DSM-ALS). The paper showed tha…

ajolicoeur.ca

8

111

504

S

@NakramR

4 years

@jm_alexia @YSongStanford Your blog doesn't mention actual training/generation times. What are they like? For hobbyists, wouldn't CPU also be an option? Sure you'd increase training time by a factor 2 or 3, but that's still doable, right?

1

0

4

Alexia Jolicoeur-Martineau

@jm_alexia

4 years

@NakramR @YSongStanford I have no clue about CPU. CIFAR-10 can be done on 1 GPU. 64x64 on 2 GPUs. 256x256 on 8+ GPUs. Training time is 2-5 days depending on the image size. Generation time is 5min-30min depending on the image size and quality-time trade-off (more iterations = better samples).

0

7

Arthur B. 🌮

@ArthurB

4 years

@jm_alexia @YSongStanford You mention that you lose the pyramidal structure. Can you recover it by learning the conditional distribution of each upsampling? For instance, at each resolution, the denoising autoencoder only adds noise to 3/4th of the pixels.

0

1