Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boltzmann Encoded Adversarial Machines

arxiv.org

Post reply on HN
Print view View original

11–20 of 25 posts

Re: Boltzmann Encoded Adversarial Machines

#11

Post by rm_-rf_slash »

w-m wrote: post #6

It's amazing to see deep learning blast through all the benchmarks, for example in computer vision, over the last couple of years. At the same time something starts to feel off about having all these single-use asymmetric feedforward networks solving their own little task. Being trained in one direction, then used in the other, then thrown away. Maybe being chained together for a more complex task, but that seems to…

I have had a similar thought recently. The office I work at has a large e-recycling bin for old computers. I have recovered quite a few desktops, laptops, and monitors, as well as a bunch of tidbits like adapters and RAM.

A lot of the RAM, for instance, is DDR2 and usually a measly 1gb apiece. They take up the exact same amount of space as RAM with 4gb apiece or more. I don’t know entirely why I still have them. Now that I’m doing physical computing/IoT development, Im seeing how pointless it is to have a bunch of desktops/laptops when I can get much more done - conveniently I might add - with a teeny tiny RedBear microcontroller.

I think an inherent feature of technology is having to get used to the idea that things age and die much faster than other products. Whether that’s physical hardware or trained neural networks, there comes a point when we just have to let go.

Re: Boltzmann Encoded Adversarial Machines

#12

Post by drams »

cs702 wrote: post #7
drams wrote: post #5

Earlier quoted context omitted.

To clarify: in the case of a BEAM both the generator and all but the top layer of the discriminator is replaced with an RBM. The adversary in this case operates on features encoded by the RBM, not raw data samples. Secondly the RBM is trained with a combined loss involving log-likelihood and the adversarial term.

Yes. For simplicity's and brevity's sake, I ignored many important details in my summary.

No worries! :)

Re: Boltzmann Encoded Adversarial Machines

#15

Post by MarkMMullin »

I'm wondering if the work on adversarial systems, this one being quite interesting, can help us with our giant bugaboo of "OMG, its overfitted :-(" Right now we model, train, test, fail, and start all over again, and usually fiddle with the hyperparameters to boot - what would happen if we turned training into a two phased approach, with a BEAM/GAN whatnot used on each cycle to measure how 'brittle' the backprop is? The idea being to round down the spikes in the learned model by penalizing the backprop when it is too narrow - training would take longer, but we'd throw away fewer sets, I'd think

Re: Boltzmann Encoded Adversarial Machines

#17

Post by tlarkworthy »

drams wrote: post #9
rememberlenny wrote: post #4

Can someone explain the basic implications of this against current GANs and also provide a practical ML application?

I can try. (I am a coauthor of this paper) First off, Unlearn.ai is a startup working to build new tools that make precision medicine a reality. We needed to be able to build generative models which allow us to 1. model multimodal data easily (consider medical datasets with categorical data, binary, and continuous, with various bounds etc. all mixed together) 2. be able to answer counterfactual questions about data (…

RBMs have a native probabilistic output (the output is a distribution you can slice), but vanilla neural networks don't (the output is a vector). Is that right?

Re: Boltzmann Encoded Adversarial Machines

#19

Post by drams »

cs702 wrote: post #14
drams wrote: post #12

Earlier quoted context omitted.

No worries! :)

Thanks. Have you made any code available online?

Yes; The following recent review article actually provides code samples: https://arxiv.org/abs/1803.08823 which use an open-source version of our software called 'paysage' (https://github.com/drckf/paysage). This has currently not been updated too recently, but we expect to put out a new update quite soon. The update will clean up code, docs, features, but might not yet contain the BEAM training code. The latter is pending some decisions about IP, etc.
Post reply on HN