Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evolution Is the New Deep Learning

sentient.ai

Post reply on HN
Print view View original

211–220 of 242 posts

Re: Evolution Is the New Deep Learning

#211

Post by PeterisP »

Houshalter wrote: post #146
PeterisP wrote: post #54

Earlier quoted context omitted.

DL models don't often get stuck at local optima. In theory, they could be vulnerable to that, but in practice they are not, it simply doesn't happen in most practical supervised learning applications. I'm not up to date on theoretical research about this topic, but as far as I recall there are some interesting demonstrations on realistic problems showing that all the different "local" optima resulting from different…

This is common wisdom I think is false. You absolutely do get stuck in local optima frequently with reinforcement learning. OpenAI has a good example somewhere of a robot trying to put a peg through a hole. Trained with regular gradient descent it just gets stuck putting the peg pretty close to the hole, but not through it. I'm not even sure that it's not a problem in general. I know I've watched NNs frequently get s…

This is field dependent, it feels more like an attribute of certain types of data rather than certain algorithms.

Reinforcement learning was on my mind when I was writing about "practical supervised learning applications" because yes, RL is different in that regard. And various function calculation examples (starting with XOR) indeed do so.

However, if we're applying neural networks for the (wide and practically important!) class of "pattern recognition" tasks like processing image or language data, then it's different, and those are full fields where you can easily spend a whole career working on just one of these types of data. Perhaps there's a relation with the structure and redundancy inherent in data like this.

Re: Evolution Is the New Deep Learning

#212

Post by ramblerman »

jamesrcole wrote: post #181
resu_nimda wrote: post #64

Earlier quoted context omitted.

1) One of the biggest reasons they fell out of favor for more "mathematical" approaches was that no one could really explain why exactly they worked. Kind of like how nobody can really explain how the brain works, or life in general. My gut feeling is that it is hubris to think that we are going to "figure out" intelligence with increasingly sophisticated mathematical models anytime soon. We are not giving proper cre…

> going off of Arthur C Clarke's definition, life is magic. And we're trying to create something equally magical. I assume you're referring to his "any sufficiently advanced technology is indistinguishable from magic"? If so, you're misrepresenting it, because he's clearly saying it's not magic, it just appears that way to the unadvanced. And there's a big difference between "appears to be" and "is".

But we are the unadvanced on this matter, so it's magic.

Also 'appears to be' != 'indistinguishable', the latter is far closer to 'is' imo.

Re: Evolution Is the New Deep Learning

#214

Post by resource0x »

pas wrote: post #204
resource0x wrote: post #162

Earlier quoted context omitted.

200 years ago there was nothing to understand in electricity: it was just a liquid :) https://en.wikipedia.org/wiki/Fluid_theory_of_electricity

There were experiments that were not explained by the liquid theory. Now we have data and people for some reason want to claim that a theory with magical super complex and not-even-yet-describable and very-very-irreducible element(s) is a better fit than a good old box full of tiny yet specialized parts fine-tuned to work together over millions of years.

Can you think of any experiment that cannot be explained by your theory?

Re: Evolution Is the New Deep Learning

#215

Post by sgt101 »

clicea wrote: post #185
jcora wrote: post #152

Earlier quoted context omitted.

I'd call this pulling a Dennett: trivializing complexity to something that cannot or just doesn't have to be explained. Being unable to conceive consciousness at this moment doesn't mean there's nothing to conceive of: even if we never get to the final satisfactory answer, there is undoubtedly much more room left for useful concepts we don't have yet, around or inside this idea

I'd call bullshit. Dennett's argument not that the brain is complex but that it's not obvious it is not reductible.

That's a lot of nots. So you are saying that Dennett says that the brain might be reducible[1]?

I don't think that's a strong claim or that it even qualifies as a claim at all. Lots of things might decompose into simple components if subjected to the right analysis, very few things definitely won't - for example many clever people have spent a great deal of time attempting to reduce quantum and cosmic scale physics to simple intuitively founded laws... If Dennett's claim is that the human brain is the same order of object as the universe I can accept it only if we agree that all objects share the same order. Where does that get us?

[1] Apologies, I don't know what reductible means, but guessed typo - I'm open to education though and unworried by typos!

Re: Evolution Is the New Deep Learning

#216

Post by [deleted] »

pas wrote: post #95
resu_nimda wrote: post #81

Earlier quoted context omitted.

Your last paragraph seems to contain the kind of overconfidence that I'm talking about. I don't understand how you can say "consciousness is simply X" or "it's easy to do that [if you handwave away the hard parts]." Clearly it's not that simple or easy, or we would have done it. We can't even create life from non-life. How can we begin to understand all the stuff you're talking about that's been layered on top? We do…

I'm assuming a simpler model, no need for magic, because so far I don't see what behavior/data this simple model cannot explain. > Clearly it's not that simple or easy, or we would have done it. We don't have the computational power yet. Not to mention the vast amount of development required. Think of the climate models, that are huge (millions of lines of code), but they're still nowhere near complete enough, and th…

[deleted]

Re: Evolution Is the New Deep Learning

#217

Post by goatlover »

dorgo wrote: post #191
goatlover wrote: post #174

Earlier quoted context omitted.

> Is there a task left which can't be done by computers better and faster than by humans? All the tasks humans still earn money doing. And given that we're nowhere near full automation, I'd say it's quite a few tasks.

Like the collector in the supermarket who scans the products and takes my cash? Surely it must be impossible to automate such a complex task. Industry/Economy lacks behind state-of-the-art technology by decades.

That's a poor example, since self-checkout scanners have existed for a while now. But notice how they aren't used exclusively. The bigger orders still require the manned scanners, and the self-checkout always has someone on duty.

A better example is plumbing. How would you go about automating a human plumber who handles all sorts of piping and crawl spaces in a large variety of settings?

Re: Evolution Is the New Deep Learning

#218

Post by n4r9 »

yters wrote: post #194
_mhr_ wrote: post #188

Earlier quoted context omitted.

That's a very interesting hypothesis, thank you for the link. However, it doesn't seem to support what you're claiming about mutations, from my cursory reading. Can you provide a source for this claim that random mutation is a bad source of variation? http://extendedevolutionarysynthesis.com/about-the-ees/why-i... explicitly rejects the idea of a revolution regarding mutation and other genetic sources of variation: >…

There is John Sanford who claims random mutation is destroying our genome. http://www.geneticentropy.org/

Right, but he also believes the Earth is less than 100,000 years old. Do you believe that, too?

Re: Evolution Is the New Deep Learning

#219

Post by yters »

n4r9 wrote: post #218
yters wrote: post #194

Earlier quoted context omitted.

There is John Sanford who claims random mutation is destroying our genome. http://www.geneticentropy.org/

Right, but he also believes the Earth is less than 100,000 years old. Do you believe that, too?

I don't know :| It seems to be the most plausible explanation if genetic entropy is true. I'd be interested is a good refutation of his thesis, but it makes a lot of sense mathematically.

Re: Evolution Is the New Deep Learning

#220

Post by yters »

randomsearch wrote: post #207
yters wrote: post #166

If evolutionary algorithms are so magical, why do I only encounter them in academia?

They're fairly widely used, but often in places where they are not well publicised - e.g. engineering optimisation or financial trading.

My MSc advisor wrote a big book on multi-objective evolutionary optimization.

http://www.springer.com/gp/book/9780387332543

He liked to have his students compare random search to evolutionary algorithms. There tended to not be a huge difference. I think that's why they are not so widely used when there is any kind of better method around. Probably in most cases you'd want to just understand the domain better.

Hence, I don't believe EAs are the next big thing.

Post reply on HN