Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boltzmann Encoded Adversarial Machines

arxiv.org

Post reply on HN
Print view View original

21–25 of 25 posts

Re: Boltzmann Encoded Adversarial Machines

#21

Post by drams »

tlarkworthy wrote: post #17
drams wrote: post #9

Earlier quoted context omitted.

I can try. (I am a coauthor of this paper) First off, Unlearn.ai is a startup working to build new tools that make precision medicine a reality. We needed to be able to build generative models which allow us to 1. model multimodal data easily (consider medical datasets with categorical data, binary, and continuous, with various bounds etc. all mixed together) 2. be able to answer counterfactual questions about data (…

RBMs have a native probabilistic output (the output is a distribution you can slice), but vanilla neural networks don't (the output is a vector). Is that right?

It's best to say that an RBM is an undirected NN which models a probability distribution of some variables. You can sample from the distribution (which is a stochastic process). There are other NN models which use feed-forward NNs to do similarly --such as GANs and VAEs and others. The generation process is also stochastic, but the difference is that you sample a noise distribution and then feed that through the NN. In all cases the generated samples are still vectors.

Re: Boltzmann Encoded Adversarial Machines

#22

Post by cs702 »

drams wrote: post #19
cs702 wrote: post #14

Earlier quoted context omitted.

Thanks. Have you made any code available online?

Yes; The following recent review article actually provides code samples: https://arxiv.org/abs/1803.08823 which use an open-source version of our software called 'paysage' ( https://github.com/drckf/paysage ). This has currently not been updated too recently, but we expect to put out a new update quite soon. The update will clean up code, docs, features, but might not yet contain the BEAM training code. The latter is…

Thank you. I'll take a look!

Re: Boltzmann Encoded Adversarial Machines

#23

Post by cycrutchfield »

cs702 wrote: post #2

Very interesting. At a high level (ignoring many details) the main idea is to replace generator networks in GANs with Restricted Boltzman Machines, or RBMs, which are easier to train (more stable). The authors call this kind of architecture "Boltzmann Encoded Adversarial Machines," or BEAM for short. The experiments provide persuasive evidence that BEAMs outperform GANs. Figure 3, in particular, I find very persuasiv…

I am not entirely convinced. In particular, the results shown in Fig 7 remind me of the BEGAN paper which was similarly hyped. But I'll defer further judgment until I read through it more and maybe run some experiments.

Re: Boltzmann Encoded Adversarial Machines

#24

Post by radarsat1 »

w-m wrote: post #6

It's amazing to see deep learning blast through all the benchmarks, for example in computer vision, over the last couple of years. At the same time something starts to feel off about having all these single-use asymmetric feedforward networks solving their own little task. Being trained in one direction, then used in the other, then thrown away. Maybe being chained together for a more complex task, but that seems to…

I found this one pretty interesting in that regard. The basic gist is that learning the function that projects onto the boundary of the dataset is useful for a variety of (linear inverse) problems.

One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models https://arxiv.org/abs/1703.09912

Re: Boltzmann Encoded Adversarial Machines

#25

Post by DanWaterworth »

cycrutchfield wrote: post #23
cs702 wrote: post #2

Very interesting. At a high level (ignoring many details) the main idea is to replace generator networks in GANs with Restricted Boltzman Machines, or RBMs, which are easier to train (more stable). The authors call this kind of architecture "Boltzmann Encoded Adversarial Machines," or BEAM for short. The experiments provide persuasive evidence that BEAMs outperform GANs. Figure 3, in particular, I find very persuasiv…

I am not entirely convinced. In particular, the results shown in Fig 7 remind me of the BEGAN paper which was similarly hyped. But I'll defer further judgment until I read through it more and maybe run some experiments.

There's a good reason that the pictures look similar. Both architectures produce somewhat blurry images.

The problem, in BEGAN's case, is that when your idea of similarity is based of mean squared error, high frequency details are just not important. [1] You can see this by doing PCA on natural image patches. BEGAN uses an autoencoder trained on MSE.

RBMs produce blurry images because the architecture is not good at representing multiplicative interactions. You just get splodges of colour.

[1] http://danielwaterworth.com/posts/what's-wrong-with-autoenco...

Post reply on HN