Live data from Hacker News

Neurogenesis Deep Learning

arxiv.org

11–20 of 105 posts

Re: Neurogenesis Deep Learning

#11
post #9

Very wishful thinking on my part, but I think we're far closer to a general intelligence than most expect.

Out of curiosity: how far away do you think we are?

I'll say... 5 - 7 years. This is all based on pure speculation, and being a little more than a ML hobbiest. One thing is for sure though - when we do reach that point, everything changes forever.

Re: Neurogenesis Deep Learning

#13
post #9

Earlier quoted context omitted.

Out of curiosity: how far away do you think we are?

I'll say... 5 - 7 years. This is all based on pure speculation, and being a little more than a ML hobbiest. One thing is for sure though - when we do reach that point, everything changes forever.

I would have said 30 years, and I thought I was the optimist :) Anyway I hope you're right and I'm wrong!

Re: Neurogenesis Deep Learning

#14

It never ceases to amaze me that the best steps towards achieving AI is to look at how we perceive that a Neuron works and simulate it. And the thing is, we aren't exactly sure why exactly that is.. it's amazing. Sometimes the best thing we can do is imitate nature

This isn't strictly true though.

Spiking Neural Networks [0] attempt to be more accurate representations of human neurons, but haven't really caught on because they aren't really much better than our perceptron model of neurons, at least for the things we are trying to do with them.

[0]http://www.ane.pl/pdf/7146.pdf

Re: Neurogenesis Deep Learning

#15
post #13

Earlier quoted context omitted.

I'll say... 5 - 7 years. This is all based on pure speculation, and being a little more than a ML hobbiest. One thing is for sure though - when we do reach that point, everything changes forever.

I would have said 30 years, and I thought I was the optimist :) Anyway I hope you're right and I'm wrong!

That's not a thing to hope for. We haven't solved the value alignment problem, so the longer until we have human or better-than-human AI, the better.

Re: Neurogenesis Deep Learning

#16

It never ceases to amaze me that the best steps towards achieving AI is to look at how we perceive that a Neuron works and simulate it. And the thing is, we aren't exactly sure why exactly that is.. it's amazing. Sometimes the best thing we can do is imitate nature

I find that somewhat strange.

Why attribute the idea of introducing new nodes to a graph to biological concepts? It seems like a simple step in exploration, similar to how one might think to vary the weights of the nodes randomly over some range.. unless there is some technique biology uses to pre-configure the nodes upon introduction to the network, that might be rather interesting.

Re: Neurogenesis Deep Learning

#17
post #8

Slightly off topic, but I hate how publications are written. It seems like authors are purposely using big words and sentences that are often 5-6 lines long in order to make it seem more clever. I find myself often having to reread a sentence in order to understand it. These algorithms are often very simple and can be easily explained. Don't over complicate them.

Then here's a challenge: could you write the abstract of the article in "simple English", without changing the meaning?

Re: Neurogenesis Deep Learning

#18
post #13

Earlier quoted context omitted.

I'll say... 5 - 7 years. This is all based on pure speculation, and being a little more than a ML hobbiest. One thing is for sure though - when we do reach that point, everything changes forever.

I would have said 30 years, and I thought I was the optimist :) Anyway I hope you're right and I'm wrong!

Compared to researchers in the field, even your prediction is very optimistic.

Re: Neurogenesis Deep Learning

#19

Very wishful thinking on my part, but I think we're far closer to a general intelligence than most expect.

Well, an interesting property of the brain is that any I/O relation happens within X milliseconds, which puts a limit on the depth of the network (if the speed of a neuron is limited). It would be nice to have some hard numbers on this.

Re: Neurogenesis Deep Learning

#20
post #5
post #3

Neurogensis? How about neural death as a way to prune large neural networks into more compact ones--now that is a research idea!

This idea is at least partially in use with regularisation and dropout. The difference at least with dropout is that the "killed" neurons are then massaged back into the network in order become useful again.

Agreed that this is another way of framing the problem of regularizing a network. Rather than starting with a big network and penalizing complexity, they are starting with a simple network and adding complexity. To that end, I'd've liked to see a comparison to dropout or L1/L2 regularization.
Post reply on HN