From GAN to WGAN (2017)

#1

Post by rouli » Sun, Aug 26, 2018, 5:02 AM UTC

From GAN to WGAN (2017)
lilianweng.github.io

Re: From GAN to WGAN (2017)

#2

Post by nl » Mon, Aug 27, 2018, 6:16 AM UTC

If anyone is looking for a good project, alternate Wasserstein distance/EMD solvers would be a good place to start.

AFAIK everyone (eg, pyemd, gensim, textacy) uses wrappers around the EMD solver from http://ofirpele.droppages.com/, which is a zip file from some time in 2008. The limits on performance mean it can't practically be used in things like interactive nearest neighbor calculations (FAISS, nmslib, annoy etc)

Re: From GAN to WGAN (2017)

#3

Post by Uberphallus » Mon, Aug 27, 2018, 11:02 AM UTC

nl wrote: post #2

If anyone is looking for a good project, alternate Wasserstein distance/EMD solvers would be a good place to start. AFAIK everyone (eg, pyemd, gensim, textacy) uses wrappers around the EMD solver from http://ofirpele.droppages.com/ , which is a zip file from some time in 2008. The limits on performance mean it can't practically be used in things like interactive nearest neighbor calculations (FAISS, nmslib, annoy etc…

Looks like a complex topic, but the code is small enough to help remove the rust from my math gears. I guess everybody would want it ported to CUDA/OpenCL/Vulkan?

Re: From GAN to WGAN (2017)

#4

Post by currymj » Mon, Aug 27, 2018, 2:24 PM UTC

nl wrote: post #2

If anyone is looking for a good project, alternate Wasserstein distance/EMD solvers would be a good place to start. AFAIK everyone (eg, pyemd, gensim, textacy) uses wrappers around the EMD solver from http://ofirpele.droppages.com/ , which is a zip file from some time in 2008. The limits on performance mean it can't practically be used in things like interactive nearest neighbor calculations (FAISS, nmslib, annoy etc…

it is weird that everyone uses this relatively old EMD solver.

in this case the WGAN doesn’t actually compute the discrete EMD like that. instead it uses some constraints in the optimization process (gradient clipping), which it can be argued make the training objective equivalent in the limit to minimizing continuous Wasserstein distance (between probability distributions).

Re: From GAN to WGAN (2017)

#5

Post by blt » Mon, Aug 27, 2018, 8:18 PM UTC

nl wrote: post #2

If anyone is looking for a good project, alternate Wasserstein distance/EMD solvers would be a good place to start. AFAIK everyone (eg, pyemd, gensim, textacy) uses wrappers around the EMD solver from http://ofirpele.droppages.com/ , which is a zip file from some time in 2008. The limits on performance mean it can't practically be used in things like interactive nearest neighbor calculations (FAISS, nmslib, annoy etc…

FWIW, the code on that page computes EMD for histograms, which is basically the easy case. The EMD over a high dimensional continuous distribution like images is intractable by comparison.