Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who Invented Backpropagation?

people.idsia.ch

Post reply on HN
Print view View original

81–90 of 120 posts

Re: Who Invented Backpropagation?

#81

Post by dataflow »

epaulson wrote: post #79
dataflow wrote: post #71

Earlier quoted context omitted.

Wow, this is the first time I'm hearing such a thing. For clarity: I pasted the output so a ton of people wouldn't repeat the same question to ChatGPT and burn a ton of CO2 to get the same answer. I didn't paste the query since I didn't find it interesting. And I didn't fact check because I didn't have the time. I was walking and had a few seconds to just do this on my phone. Not sure how this was rude, I certainly d…

The 'it's rude to show your ai(llm) output' is a reference to this: https://distantprovince.by/posts/its-rude-to-show-ai-output-...

What is "this" exactly? Is it a well-known author or website? Or otherwise a reference that one should be familiar with? It looks like a random blog to me... with an opinion declared as fact that's quite easy to refute.

Re: Who Invented Backpropagation?

#82

Post by mellosouls »

pncnmnp wrote: post #8

I have a question that's bothered me for quite a while now. In 2018, Michael Jordan (UC Berkeley) wrote a rather interesting essay - https://medium.com/@mijordan3/artificial-intelligence-the-re... (Artificial Intelligence — The Revolution Hasn’t Happened Yet) In it, he stated the following: > Indeed, the famous “backpropagation” algorithm that was rediscovered by David Rumelhart in the early 1980s, and which is now v…

See Widnall's overview here which discusses some of the ground that crosses over with what has come to be known as backpropagation:

The Minimum-Time Thrust-Vector Control Law in the Apollo Lunar-Module Autopilot (1970)

https://www.sciencedirect.com/science/article/pii/S147466701...

Re: Who Invented Backpropagation?

#83

Post by observationist »

pncnmnp wrote: post #13
psYchotic wrote: post #10

Earlier quoted context omitted.

I found this,maybe it helps: https://gwern.net/doc/ai/nn/1986-rumelhart-2.pdf

Apologies - I should have been clear. I was not referring to Rumelhart et al., but to pieces of work that point to "optimizing the thrusts of the Apollo spaceships" using backprop.

Kelley 1960 (the gradient/adjoint flight‑path paper) https://perceptrondemo.com

AIAA 65‑701 (1965) “optimum thrust programming” for lunar transfers via steepest descent (Apollo‑era) https://arc.aiaa.org/doi/abs/10.2514/6.1965-701

Meditch 1964 (optimal thrust programming for lunar landing) https://openmdao.github.io/dymos/examples/moon_landing/moon_...

Smith 1967 & Colunga 1970 (explicit Apollo‑type trajectory/re‑entry optimization using adjoint gradients) https://ntrs.nasa.gov/citations/19670015714

One thing AI has been great for, recently, has been search for obscure or indirect references like this, that might be one step removed from any specific thing you're searching for, or if you have a tip-of-the-tongue search where you might have forgotten a phrase, or know you're using the wrong wording.

It's cool that you can trace the work of these rocket scientists all the way to the state of the art AI.

Re: Who Invented Backpropagation?

#85

Post by throwaway2562 »

icelancer wrote: post #40
cs702 wrote: post #29

Whatever the facts, the OP comes across as sour grapes . The author, Jürgen Schmidhuber, believes Hopfield and Hinton did not deserve their Nobel Prize in Physics, and that Hinton, Bengio, and LeCun did not deserve their Turing Award. Evidently, many other scientists disagree, because both awards were granted in consultation with the scientific community. Schmidhuber's own work was, in fact, cited by the Nobel Prize…

Didn't click the article, came straight to the comments thinking "I bet it's Schmidhuber being salty." Some things never change.

Do yourself a big favour and read the article before commenting, perhaps?

Hint: Schmidhuber has amassed solid evidence over years of digging.

Re: Who Invented Backpropagation?

#86

Post by andyferris »

bgnn wrote: post #58
pjbk wrote: post #31

Earlier quoted context omitted.

To be fair, any multivariable regulator or filter (estimator) that has a quadratic component (LQR/LQE) will naturally yield a solution similar to backpropagation when an iterative algorithm is used to optimize its cost or error function through a differentiable tangent space.

So yeah, this was what I was thinking for a while. What about a more nonlinear estimator? Intuitively seems similar to me.

I believe the reason it works in nonlinear cases is that the derivative is “naturally linear” (to calculate the derivative, you are considering ever smaller regions where the cost function is approximately linear - exactly “how nonlinear” the cost function is elsewhere doesn’t play a role).

Re: Who Invented Backpropagation?

#89

Post by aaron695 »

pncnmnp wrote: post #8

I have a question that's bothered me for quite a while now. In 2018, Michael Jordan (UC Berkeley) wrote a rather interesting essay - https://medium.com/@mijordan3/artificial-intelligence-the-re... (Artificial Intelligence — The Revolution Hasn’t Happened Yet) In it, he stated the following: > Indeed, the famous “backpropagation” algorithm that was rediscovered by David Rumelhart in the early 1980s, and which is now v…

[dead]

Re: Who Invented Backpropagation?

#90

Post by actionfromafar »

dataflow wrote: post #81
epaulson wrote: post #79

Earlier quoted context omitted.

The 'it's rude to show your ai(llm) output' is a reference to this: https://distantprovince.by/posts/its-rude-to-show-ai-output-...

What is "this" exactly? Is it a well-known author or website? Or otherwise a reference that one should be familiar with? It looks like a random blog to me... with an opinion declared as fact that's quite easy to refute.

”This” is also the novel Blindsight.
Post reply on HN