The only surprise here is that Schmidhuber himself didn't claim to invent it lol
Who Invented Backpropagation?
61–70 of 120 posts
Re: Who Invented Backpropagation?
#62Earlier quoted context omitted.
Because it is terribly low-effort. People are here for interesting and insightful discussions with other humans. If they were interested in unverified LLM output… they would ask an LLM?
Who cares if it is low effort? I got lots of upvotes for my link to Claude about this, and pncnmnp seems happy. The downvoted comment from ChatGPT was maybe a bit spammy?
Yours was a little bit more useful, it you essentially used the LLM as a search engine to find a real article, right?
Directly posting the random text generated by the LLM is more annoying. I mean, they didn’t even vouch or it or verify that it was right.
Re: Who Invented Backpropagation?
#63Who didn't? Depending on exactly how you interpret the notion of "inventing backpropagation" it's been invented, forgotten, re-invented, forgotten again, re-re-invented, etc, about 7 or 8 times. And no, I don't have specific citations in front of me, but I will say that a lot of interesting bits about the history of the development of neural networks (including backpropagation) can be found in the book Talking Nets:…
How do you not have the citations in front of you? They are all in the article? I don't expect that any relevant (re)invention of backprop is missing there. Or, if you really know some reinvention of backprop that is not mentioned here, tell Jürgen Schmidhuber, he is actually very curious to learn about other such instances that he is not aware of yet.
Re: Who Invented Backpropagation?
#64Earlier quoted context omitted.
Honest person would have rejected it and protected the prize's honour
That's a joke, right? Turning down community recognition and a million dollars to make an unclear statement about which category the prize was awarded in?
The argument had to do with honesty while your justification is that money and popularity are worth more than being honest.
Now perhaps Hinton does deserve the award, but certainly it should not be because of the reasons you cite: money and popularity.
Re: Who Invented Backpropagation?
#65Whatever the facts, the OP comes across as sour grapes . The author, Jürgen Schmidhuber, believes Hopfield and Hinton did not deserve their Nobel Prize in Physics, and that Hinton, Bengio, and LeCun did not deserve their Turing Award. Evidently, many other scientists disagree, because both awards were granted in consultation with the scientific community. Schmidhuber's own work was, in fact, cited by the Nobel Prize…
For what it's worth, it's a very mainstream opinion in the physics community that Hinton did not at all deserve a nobel prize in physics for his work. But that's because his work, and wasnt impactful at all to the physics community
Yeah, I agree with that. When I first saw the announcement, my immediate thought was something like "huh, the Nobel guys sure want to make sure they've given an award for something related to AI which they can describe as foundational." However, I do think Hinton, Bengio, and LeCun deserved their Turing Award.
Re: Who Invented Backpropagation?
#66The chain rule was explored by Gottfried Wilhelm Leibniz and Isaac Newton in the 17th century. Either of them would have ”invented” backpropagation in an instant. It’s obvious.
It's not at all obvious, as the article points out it was assumed for 40 years that backpropagation was not an efficient approach for training neural networks.
Re: Who Invented Backpropagation?
#67Who didn't? Depending on exactly how you interpret the notion of "inventing backpropagation" it's been invented, forgotten, re-invented, forgotten again, re-re-invented, etc, about 7 or 8 times. And no, I don't have specific citations in front of me, but I will say that a lot of interesting bits about the history of the development of neural networks (including backpropagation) can be found in the book Talking Nets:…
How do you not have the citations in front of you? They are all in the article? I don't expect that any relevant (re)invention of backprop is missing there. Or, if you really know some reinvention of backprop that is not mentioned here, tell Jürgen Schmidhuber, he is actually very curious to learn about other such instances that he is not aware of yet.
They are all in the article?
Maybe they are. I'm not here to do a deep research project that involves reading every citation in that article. If it makes you feel better, pretend that what I said was instead:
"I don't have all the relevant citations stored in my short-term memory right this second and I am not interested in writing a lengthy thesis to satisfy pedantic navel-gazers on HN."
Or, if you really know some reinvention of backprop that is not mentioned here,
WTF are you on about? I never made any such claim, or anything remotely close to it.
Re: Who Invented Backpropagation?
#68Funny that hinton is not mentioned. Like how childish can the author be?
What do you mean? This popular paper is cited:
[RUM] DE Rumelhart, GE Hinton, RJ Williams (1985). Learning Internal Representations by Error Propagation.
Re: Who Invented Backpropagation?
#69Earlier quoted context omitted.
How do you not have the citations in front of you? They are all in the article? I don't expect that any relevant (re)invention of backprop is missing there. Or, if you really know some reinvention of backprop that is not mentioned here, tell Jürgen Schmidhuber, he is actually very curious to learn about other such instances that he is not aware of yet.
They are all in the article? Maybe they are. I'm not here to do a deep research project that involves reading every citation in that article. If it makes you feel better, pretend that what I said was instead: "I don't have all the relevant citations stored in my short-term memory right this second and I am not interested in writing a lengthy thesis to satisfy pedantic navel-gazers on HN." Or, if you really know some…
I thought that is what you mean when you said "a lot of interesting bits about the history of the development of neural networks (including backpropagation) can be found in the book Talking Nets", that there is some relevant reference to backprop which is missing here in the linked article.
I don't really understand your negativity here, and what you are reading into my comment. I never asked you to do a research project? I just thought you might know some other references which are not in the article. If you don't, fine.
Note that I don't expect that any relevant reference is missing here. Schmidhuber always try to be very careful to be very complete and exhaustive cite everything there is on some topic. That is why I was double curious about the possibility that sth is missing, and what it could be.
Re: Who Invented Backpropagation?
#70The chain rule was explored by Gottfried Wilhelm Leibniz and Isaac Newton in the 17th century. Either of them would have ”invented” backpropagation in an instant. It’s obvious.
It's not at all obvious, as the article points out it was assumed for 40 years that backpropagation was not an efficient approach for training neural networks.
It still is not efficient approach.