Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neurogenesis Deep Learning

arxiv.org

Post reply on HN
Print view View original

61–70 of 105 posts

Re: Neurogenesis Deep Learning

#61

Post by argonaut »

habitue wrote: post #60
argonaut wrote: post #50

Sorry if I'm being snobbish, but I do wonder why this paper is only being submitted to IJCNN, a 2nd tier machine learning conference. I know students who publish undergrad research at workshops with lower acceptance rates than IJCNN. I can't think of any important machine learning papers published in IJCNN in the recent past.

It depends on what conclusions you're trying to draw from that information. What conference a paper was accepted to is a second-order signal of the noteworthiness. It's probably easier for someone versed in the field to just read the paper to determine if it's interesting. If you're using the conference as a quick pass/fail as you skim through the abstracts of hundreds of papers, ok, but you probably wouldn't make ti…

> It's probably easier for someone versed in the field to just read the paper to determine if it's interesting.

> If you're using the conference as a quick pass/fail as you skim through the abstracts of hundreds of papers, ok

You answered your own statement, I think. Most researchers will skip a paper in a second tier conference. In fact, most I know won't read an entire paper - they'll only read some of it and skip stuff.

You're correct that I am not an active researcher (otherwise I would not have time to be commenting). I merely did some research back in college. But honestly that little experience gives me a huge leg up on most HN commenters in understanding research. It's unfortunate that the only reason this paper is #1 on HN is because it has a cool title.

That being said, MNIST is not really a disqualifier. (Unfortunately) MNIST is the most popular dataset referenced in NIPS 2016 papers (https://twitter.com/benhamner/status/805864969065689088). The handwaving is also forgivable; many NIPS papers handwave a lot too.

Re: Neurogenesis Deep Learning

#62

Post by bmh100 »

joantune wrote: post #52
bmh100 wrote: post #46

Earlier quoted context omitted.

In a very hand-wavy sense, yes. The same can be said of paths to food by ant colonies. The way that ANNs have been drawn as circles with arrows between them looks like a cartoon version of neurons and synapses, which is the origin of the "neural network" part. The timing of data from hidden node to hidden node, the activation functions, and the hidden node outputs have very little to do with biological neurons. ANNs…

Wasn't the attempt of modeling a collective of neurons, their synapsis, and the way that some connections are reinforced the genesis of the artificial neural networks? That's how the first person brought that concept to life, no? He didn't even have a theoretical explanation on how/why that would work for something, right? > ANNs have more in common with a CPU than a brain How so? which parts are similar?

Yes, ANNs are inspired by the brain.

Here is a list of properties that ANNs shared with CPUs that are different from brains:

* Synchronized activation vs. asynchronous / partially synchronous activation

* Digital signals vs. analog signals

* Instantaneous transmission of signals vs. delay imposed by axon and dendrite length

* Uniform signal vs. use of various neurotransmitter signals

* Rapid activation speed (GHz) vs. slow activation speed (Hz)

* The use of negative signals vs. strictly positive quantities of neurotransmitters

* Low average connections (10-1000) vs. high average connections (5,000-100,000)

* Low energy efficiency vs. high energy efficiency

For a detailed essay on the topic, see: http://timdettmers.com/2015/07/27/brain-vs-deep-learning-sin...

Re: Neurogenesis Deep Learning

#63

Post by badminton1 »

joantune wrote: post #43
badminton1 wrote: post #32

Earlier quoted context omitted.

Well, neurons have many properties... their information processing capabilities are one aspect, but they also deal with the physical level of communication and staying healthy. Neurons are also a family of cells, and are very diverse in shapes and functions. We tend to oversimplify our representation of neurons. There are simple neurons and then you have neurons like the Purkinje cell that are massive. Neurons also r…

If i'm not mistaken (I had the introductory class on Neural networks quite some years ago) this all started out of trying to simulate neurons in a manner. Much so that one of the pioneers of this got discredited by other scientist that for some reason simply could not accept that these would work, and that same pioneer started to get his funding discredited and started believing in his opposition so much so that he s…

Probably you might be referring to McCulloch and Pitts.

Re: Neurogenesis Deep Learning

#64

Post by bjterry »

bmh100 wrote: post #49
hmate9 wrote: post #44

Earlier quoted context omitted.

For example in the abstract: "adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations" Nobody talks like this. In my head I read this sentence and I have to translate it to "we add extra neurons to existing networks so they can learn new information while remembering everything it already kno…

However, your statement is actually vague and ambiguous. "Extra neurons"? Input layer? Output layer? Just before a final, fully-connected layer? Somewhere in between? "Everything it already knows"? What does it know? Character probabilities, like a charnn? Image categories like in a CNN? Input distributions, like a GAN? From the abstract, I can immediately tell that this paper is about modifying deep auto-encoders in…

Anytime someone uses the phrase "in order to facilitate" they are being more verbose than is necessary in order to signify their greater erudition. There is no meaningful semantic or technical distinction between "in order to facilitate" and "to help."

Re: Neurogenesis Deep Learning

#65

Post by eruditely »

argonaut wrote: post #50

Sorry if I'm being snobbish, but I do wonder why this paper is only being submitted to IJCNN, a 2nd tier machine learning conference. I know students who publish undergrad research at workshops with lower acceptance rates than IJCNN. I can't think of any important machine learning papers published in IJCNN in the recent past.

Does it matter? I assume they just wanted to get it out and publish it.

Re: Neurogenesis Deep Learning

#66

Post by bmh100 »

bjterry wrote: post #64
bmh100 wrote: post #49

Earlier quoted context omitted.

However, your statement is actually vague and ambiguous. "Extra neurons"? Input layer? Output layer? Just before a final, fully-connected layer? Somewhere in between? "Everything it already knows"? What does it know? Character probabilities, like a charnn? Image categories like in a CNN? Input distributions, like a GAN? From the abstract, I can immediately tell that this paper is about modifying deep auto-encoders in…

Anytime someone uses the phrase "in order to facilitate" they are being more verbose than is necessary in order to signify their greater erudition. There is no meaningful semantic or technical distinction between "in order to facilitate" and "to help."

I'll grant you that. It's simple wordiness.

Re: Neurogenesis Deep Learning

#67

Post by argonaut »

eruditely wrote: post #65
argonaut wrote: post #50

Sorry if I'm being snobbish, but I do wonder why this paper is only being submitted to IJCNN, a 2nd tier machine learning conference. I know students who publish undergrad research at workshops with lower acceptance rates than IJCNN. I can't think of any important machine learning papers published in IJCNN in the recent past.

Does it matter? I assume they just wanted to get it out and publish it.

See my sibling comment. It matters because it's a very strong (to academic/industry researchers) sign of quality and whether the paper is worth reading. If you wanted to just put something out there you could just put on arxiv. The authors are academics (?) so they clearly want to publish in the best possible venue.

Re: Neurogenesis Deep Learning

#69

Post by intrasight »

sxg wrote: post #57
throwaway287391 wrote: post #34

Earlier quoted context omitted.

I completely blame my own community, rather than you, for writing this, but as an AI researcher, your comment is terribly painful to read. We have little to no idea how actual neurons (let alone entire brains) really work. The things that are often called "(artificial) neural networks" really shouldn't be called that. I strongly prefer terms like "computational networks" or (where applicable) "recurrent/convolutional…

Actually we really know a lot about how neurons work. We've got the biophysical properties down, and we understand neurotransmission at the cellular/molecular level for a lot of different types of neurons. We understand signal processing where we transduce sound, smell, sight, touch, taste into neurochemical signals. We even know a decent amount about the early phases of the processing of these "raw data" signals int…

We do know a lot. But the knowledge gap is still tremendous as far as the details of neural synaptic plasticity.

Re: Neurogenesis Deep Learning

#70

Post by BeingIncubated »

empath75 wrote: post #36
gallerdude wrote: post #7

Very wishful thinking on my part, but I think we're far closer to a general intelligence than most expect.

I think what we might see is a kind of autonomous corporation that is nominally under the control of shareholders, a CEO or a board, but which makes decisions without very much or any human input, and which gains some amount of legal rights through corporate personhood. It won't be a 'general ai', though. More like a set of loosely connected systems that operate 'in the best interests of the shareholders', however th…

I'm hoping that eventually translates to the state.
Post reply on HN