Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inceptionism: Going Deeper into Neural Networks

googleresearch.blogspot.com

Post reply on HN
Print view View original

141–150 of 166 posts

Re: Inceptionism: Going Deeper into Neural Networks

#141

Post by [deleted] »

rndn wrote: post #128
sangnoir wrote: post #118

Earlier quoted context omitted.

> I think it is highly unethical that there is almost no democratic control over what is being developed at Google, Facebook et al. in secrecy. The most recent XKCD comic is quite relevant: http://xkcd.com/1539/ I consider myself to be very Left of center, but, I can't imagine what form of 'democratic control' you think is necessary over the research that Google and Facebook does. I do not fault Google or Facebook fo…

I'm just questioning whether an autopilot with a profit maximization heuristic is the best tool to guide technological progress. With democratic control I don't necessarily mean our current democratic systems but any kind of decentralization of decision making by voting. Yeah, I know that's vague, but given what appears be at stake it seems unreasonable not to consider alternatives.

[deleted]

Re: Inceptionism: Going Deeper into Neural Networks

#142

Post by PeterisP »

rndn wrote: post #77
davedx wrote: post #38

Worth reading the comments too. One from Vincent Vanhoucke: "This is the most fun we've had in the office in a while. We've even made some of those 'Inceptionistic' art pieces into giant posters. Beyond the eye candy, there is actually something deeply interesting in this line of work: neural networks have a bad reputation for being strange black boxes that that are opaque to inspection. I have never understood those…

Perhaps the argument should be steelmanned in that we should generally avoid using algorithms which are so complex that they aren't glass boxes. I doubt the idea to "simply follow gradients" can prove neural networks to be glass boxes because the output of that is still too complex. And we are clearly onto something here. If we can generate artificially hallucinated pictures today, it is not unreasonable to assume th…

Well, the problem with "avoid using algorithms which are so complex that they aren't glass boxes" is that for quite a few problems, the simple choice is to either use a machine learning solution that will essentially give you a black-box or to use an understandable algorithm that doesn't get nowhere near state-of-art accuracy and is practically useless. Speech recognition, for example.

Re: Inceptionism: Going Deeper into Neural Networks

#143

Post by fortyeight »

joeyspn wrote: post #105

The level of resemblance with a psychotropics' trip is simply fascinating. It's definitely really close to how our brain reacts when is flooded with dopamine + serotonin. I wonder if the engineers at Google can make the same experiment with audio... It'll be funny to listen the results.

I'd be interested to see if the results end up looking something like DIPT which is the only known mainly audio hallucinogenics.

Re: Inceptionism: Going Deeper into Neural Networks

#144

Post by aoeuasdf1 »

visarga wrote: post #75
waffl wrote: post #51

While I think this is beautiful, conceptually, I really am a bit terrified of the potential of this in reverse (the neural network for processing/understanding an image). With Google releasing their 'Photos' app, this network is about to get a direct pipeline for machine learning imagery to accelerate everything – my main fear would be the potential for this technology to be employed by weaponized drones able to scan…

This will happen for sure. Such super-perceptive computers will oversee our every movement. Computers can already understand our emotions in writing, voice and from the expression on our faces, they can also estimate pose and understand your movements. They can label thousands of kinds of objects. And they're just starting. They can also build neural nets 10x smaller by compressing a larger neural net while maintaini…

> They can also build neural nets 10x smaller by compressing a larger neural net while maintaining most of accuracy. That means once a problem such as vision or speech has been solved with a huge net, it can be transferred in a smaller, more efficient net.

This is known as "dark knowledge". Slides from Geoff Hinton: http://www.ttic.edu/dl/dark14.pdf

Re: Inceptionism: Going Deeper into Neural Networks

#145

Post by johnconner »

guelo wrote: post #53

I'm starting to come around to sama's way of thinking on AI. This stuff is going to be scary powerful in 5-10 years. And it will continue to get more powerful at an exponential rate.

You are not alone in your fears. Others have been ringing the alarm for some time. Nick Bostrom's Superintelligence is a good reference.

Re: Inceptionism: Going Deeper into Neural Networks

#146

Post by nl »

tripzilch wrote: post #135
anigbrowl wrote: post #125

Earlier quoted context omitted.

It's a tricky ethical area. The Google post cites several research papers that seemt o provide more than enough information to replicate these results or get similar ones, which is good, because I think everyone should be able to explore these tools - I stick by my view from yesterday that this may be a scientific breakthrough. At the same time, I can see the basis for some anxiety, because it's not hard to imagine p…

> The Google post cites several research papers that seem to provide more than enough information to replicate these results or get similar ones, which is good I agree that it is good, but even though the scientific theories and algorithms seem to be "open", having access to both the computing power and data-sets of Google, is not. So one could replicate these experiments, but not quite on the scale that Google does.…

I'm not at all sure if it's practically possible for a single (really clever) person with a high-end CPU/GPU machine (and possibly some $$$ for Cloud Computing instances), to replicate something similar to the results in this blogpost.

It's very possible.

GoogLeNet[1] is an example in Caffe: BVLC GoogLeNet in models/bvlc_googlenet: GoogLeNet trained on ILSVRC 2012, almost exactly as described in Going Deeper with Convolutions by Szegedy et al. in ILSVRC 2014. (Trained by Sergio Guadarrama @sguada)

[1] http://caffe.berkeleyvision.org/model_zoo.html

Re: Inceptionism: Going Deeper into Neural Networks

#147

Post by tzs »

Understanding what is going on in a neural network (or any other kind of machine learning mechanism) when it makes a decision can be important in real world applications.

For example, suppose you are a bank and you have used built a neural network to decide if credit applications should be approved. The lending laws in the US require that if you reject someone you tell them why.

Your neural network just gives a yes/no. It doesn't give a reason. What do you tell the applicant?

I have an idea how to deal with that, but I have no idea if it would satisfy the law. My approach is to run their application through multiple times, tweaking various items, until you get one that would be approved. You can then tell them it was that item that sunk them. For instance, suppose that if you raise their income by $5k, you get approval. You can tell them they were rejected for having income that is too low.

Re: Inceptionism: Going Deeper into Neural Networks

#148

Post by anigbrowl »

tripzilch wrote: post #136
xtacy wrote: post #133

Earlier quoted context omitted.

You could try Torch libraries. There are a few examples on how to (almost) replicate some of Google's neural network models on Imagenet. Check https://github.com/torch/torch7/wiki/Cheatsheet#demos .

Does this work well enough on a modern desktop PC without having access to Google's computing resources?

No idea, but I'm sure their enormous infrastructure helps a lot. Maybe it could be done with some sort of BOINC-type platform.

Re: Inceptionism: Going Deeper into Neural Networks

#149

Post by fr0styMatt2 »

userbinator wrote: post #74
dnr wrote: post #41

Am I the only one who found those images somewhat disturbing? I wonder if they're triggering something similar to http://www.reddit.com/r/trypophobia

I'm only mildly trypophobic but those images did have a minor effect - and I have a possible hypothesis for why it happens: what these images and trypophobia-triggering ones all have in common is a huge number of edges of various shapes and sizes, and it's this "edge overload" stimulating many more neurons than usual that's causing the disturbance. I find that the repetitive, but not-quite-the-same patterns like (org…

I've always wondered about this! Back when I was a teenager I remember playing around with one of the Kai's Power Tools to generate fractal-like images (I forget exactly which one it was, but it had a bunch of presets that would generate cell-like textures).

Thing was, I found them MASSIVELY anxiety-provoking and have never been able to figure out why. They'd literally make me panicky.

These images are doing the same. Even now, just thinking about them, my stomach is fluttering. It's something about the way they're organic, but I don't know what it is. It's definitely nothing rational.

I've never heard of trypophobia; this does seem like it. I wonder if it's closely related to the feeling of disgust; somehow an evolved response to keep us away from rotten food perhaps? Things like bacteria growing on bread, or beehives. Or any food that's started decomposing.

There's a cool documentary I remember seeing called "How Art Made The World". One of the things it talks about is how we're driven to make more-than-perfect representations of things in our art. Say we find something in the real world aesthetically pleasing. With art, we can take that aesthetically pleasing stimulus and exaggerate it, resulting in the art being more pleasing than anything in the real world.

I wonder if that's what happens here, with the almost-organic images somehow being 'hyper-disgusting' as they coincidentally line up with a hyper-exaggerated version of a stimulus that on the scale of disgusting might be 'slightly unappealing' in the form that we'd encounter it in the real world.

Re: Inceptionism: Going Deeper into Neural Networks

#150

Post by nitrogen »

murbard2 wrote: post #115
rndn wrote: post #114

Earlier quoted context omitted.

They've written though that they have chosen a particular layer in the network, which reads like "independent of the output layer". Features in such a layer correlate with certain classes, but I don't think they have dealt with classes at all. If that's the case, then the question is how they've amplified the detected features.

Yes they play with various layers. Layers closer to the input act more like edge enhancers, while higher layers emphasize whole objects ("animal" enhancers). You get increasingly less syntactical and increasingly more semantic as you go deeper in the network.

So "lower" layers are closer to the input, while "higher" layers are closer to the output?
Post reply on HN