It would be interesting to know what happens if instead of tweaking it to better match a banana, they tweaked it to better match a banana and NOT match everything else.
Inceptionism: Going Deeper into Neural Networks
131–140 of 166 posts
Re: Inceptionism: Going Deeper into Neural Networks
#132Earlier quoted context omitted.
> I think it is highly unethical that there is almost no democratic control over what is being developed at Google, Facebook et al. in secrecy. The most recent XKCD comic is quite relevant: http://xkcd.com/1539/ I consider myself to be very Left of center, but, I can't imagine what form of 'democratic control' you think is necessary over the research that Google and Facebook does. I do not fault Google or Facebook fo…
I'm just questioning whether an autopilot with a profit maximization heuristic is the best tool to guide technological progress. With democratic control I don't necessarily mean our current democratic systems but any kind of decentralization of decision making by voting. Yeah, I know that's vague, but given what appears be at stake it seems unreasonable not to consider alternatives.
I'm reading Rationality: From AI to Zombies, and it goes through exactly this argument. Here's the original post:
Re: Inceptionism: Going Deeper into Neural Networks
#133I understand the theory behind neural networks quite well, but am not so clear on how you feed them with images, eg how do you build a network that can process megapixel images of random aspect ratios or audio files of predictable length? I', trying to get a sense of how much effort would be involved to replicate these results if Google isn't inclined to share its internal tools, to do a neural network version of Fra…
You could try Torch libraries. There are a few examples on how to (almost) replicate some of Google's neural network models on Imagenet.
Check https://github.com/torch/torch7/wiki/Cheatsheet#demos.
Re: Inceptionism: Going Deeper into Neural Networks
#134Earlier quoted context omitted.
Perhaps the argument should be steelmanned in that we should generally avoid using algorithms which are so complex that they aren't glass boxes. I doubt the idea to "simply follow gradients" can prove neural networks to be glass boxes because the output of that is still too complex. And we are clearly onto something here. If we can generate artificially hallucinated pictures today, it is not unreasonable to assume th…
Great more "ZOMG I'm skirred of AI" FUD. Stop being so afraid of the future.
[deleted]
Re: Inceptionism: Going Deeper into Neural Networks
#135Earlier quoted context omitted.
> I think it is highly unethical that there is almost no democratic control over what is being developed at Google, Facebook et al. in secrecy. The most recent XKCD comic is quite relevant: http://xkcd.com/1539/ I consider myself to be very Left of center, but, I can't imagine what form of 'democratic control' you think is necessary over the research that Google and Facebook does. I do not fault Google or Facebook fo…
It's a tricky ethical area. The Google post cites several research papers that seemt o provide more than enough information to replicate these results or get similar ones, which is good, because I think everyone should be able to explore these tools - I stick by my view from yesterday that this may be a scientific breakthrough. At the same time, I can see the basis for some anxiety, because it's not hard to imagine p…
> The Google post cites several research papers that seem to provide more than enough information to replicate these results or get similar ones, which is good
I agree that it is good, but even though the scientific theories and algorithms seem to be "open", having access to both the computing power and data-sets of Google, is not.
So one could replicate these experiments, but not quite on the scale that Google does. I'm not at all sure if it's practically possible for a single (really clever) person with a high-end CPU/GPU machine (and possibly some $$$ for Cloud Computing instances), to replicate something similar to the results in this blogpost.
The recognition nets used in the blogpost seem to be trained on a tremendously high number of training examples, to give the ability to "hallucinate" (or classify) such a great variety of animal species, for instance.
Re: Inceptionism: Going Deeper into Neural Networks
#136I understand the theory behind neural networks quite well, but am not so clear on how you feed them with images, eg how do you build a network that can process megapixel images of random aspect ratios or audio files of predictable length? I', trying to get a sense of how much effort would be involved to replicate these results if Google isn't inclined to share its internal tools, to do a neural network version of Fra…
You could try Torch libraries. There are a few examples on how to (almost) replicate some of Google's neural network models on Imagenet. Check https://github.com/torch/torch7/wiki/Cheatsheet#demos .
Does this work well enough on a modern desktop PC without having access to Google's computing resources?
Re: Inceptionism: Going Deeper into Neural Networks
#137The reason they look so 'fractal-like' (e.g. trippy!) is because they actually are fractals! In the same way a normal fractal is a recursive application of some drawing function, this is a recursive application of different generation or "recognition -> generation" drawing functions built on top of the CNN. So I believe that, given a random noise image, these networks don't generate the crazy trippy fractal patterns…
> The reason they look so 'fractal-like' (e.g. trippy!) is because they actually are fractals!
While I agree with your idea about fractals (though you're a bit vague on the math details to know for sure), I also believe that a large reason the images look so "trippy" is because there is some local contrasting effect at work, generating high-saturation rainbow fringes at the edges of details and features. You get loads of that on psychedelics as well.
I bet there's a pretty straightforward reason to explain these rainbow fringes, if one were to dig into it, though.
Another (unrelated) observation I had was the feeling that the neural net seemed to be reproducing JPEG-artifact type fringes in the images? Though it could be that I was just looking at scaled versions of already JPEG-compressed output images, the article doesn't provide details (if only they had been PNGs ...).
Re: Inceptionism: Going Deeper into Neural Networks
#138Earlier quoted context omitted.
I'm just questioning whether an autopilot with a profit maximization heuristic is the best tool to guide technological progress. With democratic control I don't necessarily mean our current democratic systems but any kind of decentralization of decision making by voting. Yeah, I know that's vague, but given what appears be at stake it seems unreasonable not to consider alternatives.
I'm reading Rationality: From AI to Zombies, and it goes through exactly this argument. Here's the original post: http://lesswrong.com/lw/jb/applause_lights/
[deleted]
Re: Inceptionism: Going Deeper into Neural Networks
#139The level of resemblance with a psychotropics' trip is simply fascinating. It's definitely really close to how our brain reacts when is flooded with dopamine + serotonin. I wonder if the engineers at Google can make the same experiment with audio... It'll be funny to listen the results.
Might be interesting, although I've always liked the visuals* of psychedelica a lot more than the audio effects (which in my experience, mostly tends to make sounds be perceived really "loud" and "close", rather than "trippy"--unless that's what you associate with "trippy" audio, of course). Dunno if my experience is typical, obviously.
* also the particular mind-altering effects, which are hard to describe
Re: Inceptionism: Going Deeper into Neural Networks
#140This is brilliant! I did something similar when I was trying to learn about neural networks a long long time ago. The results were fascinating.
I was writing a neural network trainer - to recognize simple 2D images. This was on a 300MHz desktop PC(!) so the network had to be pretty small. Which implied that the input images were just compositions of simple geometric shapes - a circle within a rectangle, two circles intersecting, etc.
When I tried "recalling" the learnt image after every few X epochs of training, I noticed the neural network was "inventing" more complex curves to better fit the image. Initially, only random dots would show up. Then it would have invented straight lines and would try to compose the target image out of one and more straight lines.
What was absolute fun to watch was, at some point, it would stop trying to compose a circle with multiple lines and just invent the circle. And then proceed to deform the circle as needed.
During different runs, I could even see how it got stuck into various local minima. To compose a rectangle, mostly the net would create four lines - but having the lines terminate was obviously difficult. As an alternative, sometimes the net would instead try a circle, which it would gradually elongate, straighten out the circumference, slowly to look more and more like a rectangle.
I was only an undergrad then, and was mostly doing this for fun - I do believe I should have written it up then. I do not even have the code anymore.
But good to know googlers do the same kinda goofy stuff :-)