Can we back propagate credit?
Who Invented Backpropagation?
41–50 of 120 posts
Re: Who Invented Backpropagation?
#42Whatever the facts, the OP comes across as sour grapes . The author, Jürgen Schmidhuber, believes Hopfield and Hinton did not deserve their Nobel Prize in Physics, and that Hinton, Bengio, and LeCun did not deserve their Turing Award. Evidently, many other scientists disagree, because both awards were granted in consultation with the scientific community. Schmidhuber's own work was, in fact, cited by the Nobel Prize…
I think the unspoken claim here is that the North American scientific establishment takes credit from other sources and elevates certain personas instead of the true innovators who are overlooked. Arguing that the establishment doesn't agree with this idea is kinda pointless.
Re: Who Invented Backpropagation?
#43Earlier quoted context omitted.
Because it is terribly low-effort. People are here for interesting and insightful discussions with other humans. If they were interested in unverified LLM output… they would ask an LLM?
Who cares if it is low effort? I got lots of upvotes for my link to Claude about this, and pncnmnp seems happy. The downvoted comment from ChatGPT was maybe a bit spammy?
> Who cares if it is low effort?
It's a weird thing to wonder after so many people expressed their dislike of the upthread low-effort comment with a down vote (and then another voiced a more explicit opinion). The point is that a reader may want to know that the text they're reading is something a human took the time to write themselves. That fact is what makes it valuable.
> pncnmnp seems happy
They just haven't commented. There is no reason to attribute this specific motive to that fact.
Re: Who Invented Backpropagation?
#44Earlier quoted context omitted.
Reverse move automatic differentiation is not integration. It's still differentiation, but just a different method of calculating the derivative than the one you'd think to do by hand. It basically just applies the chain rule in the opposite order from what is intuitive to people. It has a lot more overhead than regular forwards mode autodiff because you need to cache values from running the function and refer back t…
I don't think most people think to do either direction by hand; it's all just matrix multiplication, you can multiply them in whatever order makes it easier.
Im just talking about the general algorithm to write down the derivative of `f(g(h(x)))` using the chain rule.
For vector valued functions, the naive way you would learn in a vector calculus class corresponds to forward mode AD.
Re: Who Invented Backpropagation?
#45Good ideas are never invented. They are always rediscovered.
Re: Who Invented Backpropagation?
#46[flagged]
Re: Who Invented Backpropagation?
#47Whatever the facts, the OP comes across as sour grapes . The author, Jürgen Schmidhuber, believes Hopfield and Hinton did not deserve their Nobel Prize in Physics, and that Hinton, Bengio, and LeCun did not deserve their Turing Award. Evidently, many other scientists disagree, because both awards were granted in consultation with the scientific community. Schmidhuber's own work was, in fact, cited by the Nobel Prize…
For what it's worth, it's a very mainstream opinion in the physics community that Hinton did not at all deserve a nobel prize in physics for his work. But that's because his work, and wasnt impactful at all to the physics community
Re: Who Invented Backpropagation?
#48Whatever the facts, the OP comes across as sour grapes . The author, Jürgen Schmidhuber, believes Hopfield and Hinton did not deserve their Nobel Prize in Physics, and that Hinton, Bengio, and LeCun did not deserve their Turing Award. Evidently, many other scientists disagree, because both awards were granted in consultation with the scientific community. Schmidhuber's own work was, in fact, cited by the Nobel Prize…
For what it's worth, it's a very mainstream opinion in the physics community that Hinton did not at all deserve a nobel prize in physics for his work. But that's because his work, and wasnt impactful at all to the physics community
I think Hinton himself has made that observation.
In a recent talk he made a quip that he had to change some slides because if you have a Nobel prize in physics you should at least get the units right.
Re: Who Invented Backpropagation?
#49The chain rule was explored by Gottfried Wilhelm Leibniz and Isaac Newton in the 17th century. Either of them would have ”invented” backpropagation in an instant. It’s obvious.
Funny enough. For me it was the other way around. I always knew how to compute the chain rule. But really only understood what the chain rule means when I read up on what back propagation was.
That's essentially it. Learning what the chain rule does, and learning what it can be used for, and how to apply it.
Neither are really inventions, they are discoveries, if anything the chain rule leans slightly more to invention than backdrop.
I understand the need for attribution as a means to track the means and validity of discovery, but I intensely dislike it when people act like it is a deed of ownership of an idea.
Re: Who Invented Backpropagation?
#50Earlier quoted context omitted.
Who cares if it is low effort? I got lots of upvotes for my link to Claude about this, and pncnmnp seems happy. The downvoted comment from ChatGPT was maybe a bit spammy?
> Who cares if it is low effort? It's a weird thing to wonder after so many people expressed their dislike of the upthread low-effort comment with a down vote (and then another voiced a more explicit opinion). The point is that a reader may want to know that the text they're reading is something a human took the time to write themselves. That fact is what makes it valuable. > pncnmnp seems happy They just haven't com…
> The point is that a reader may want to know that the text they're reading is something a human took the time to write themselves.
The reader may also simply want information that helps them.
> They just haven't commented.
Yes, they did.