Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Who invented deep residual learning?

people.idsia.ch

Post reply on HN
Print view View original

31–37 of 37 posts

Re: Who invented deep residual learning?

#31

Post by snthpy »

ansk wrote: post #4

Of all Schmidhuber's credit-attribution grievances, this is the one I am most sympathetic to. I think if he spent less time remarking on how other people didn't actually invent things (e.g. Hinton and backprop, LeCun and CNNs, etc.) or making tenuous arguments about how modern techniques are really just instances of some idea he briefly explored decades ago (GANs, attention), and instead just focused on how this sing…

> That said, I do respect the extent to which he continues his credit-attribution crusade even to his own reputational detriment.

Lol, I still used to notice him before covid when he was railing against Bengio, Hinton, and LeCun. Can't believe he's still going.

Re: Who invented deep residual learning?

#32

Post by ggm »

CamperBob2 wrote: post #19
dchftcs wrote: post #17

Earlier quoted context omitted.

Einstein published his relativity papers originally in German.

German was the lingua franca of physics at the time, so to speak. Starting in the 1930s, though, that tradition began to change... for reasons that I'm sure won't ever apply to American English. Nosirree, Bob, we're special. Great, even.

> German was the lingua franca

A finer expression of the matter cannot be made.

Re: Who invented deep residual learning?

#33

Post by ricklamers »

ekjhgkejhgk wrote: post #5

I spent some time in the academia. The person with whom an idea ends up associated often isn't the first person to have the idea. Most often is the person who explains why the idea is important, or find a killer application for the idea, or otherwise popularizes the idea. That said, you can open what Schmidhuber would say is the paper which invented residual NNs. Try and see if you notice anything about the paper tha…

Perhaps then inventors of promising ideas should make multiple attempts at popularizing their ideas if they care about association, multiple attempts at explaining why the idea is important and demonstrations of killer applications.

Re: Who invented deep residual learning?

#34

Post by sorokod »

CamperBob2 wrote: post #19
dchftcs wrote: post #17

Earlier quoted context omitted.

Einstein published his relativity papers originally in German.

German was the lingua franca of physics at the time, so to speak. Starting in the 1930s, though, that tradition began to change... for reasons that I'm sure won't ever apply to American English. Nosirree, Bob, we're special. Great, even.

A relevant quote from David Hilbert:

"Mathematics in Göttingen?" Hilbert repiled. "There is really none any more."

https://hsm.stackexchange.com/questions/2486/source-for-hilb...

Re: Who invented deep residual learning?

#35

Post by versteegen »

ekjhgkejhgk wrote: post #8
seanmcdirmid wrote: post #7

Earlier quoted context omitted.

Surely they wrote some papers in English even if they wrote their dissertation in German? Most people don’t go straight to dissertations anyway, it’s more of a place to go after you read a much shorter paper.

Correct, that's [2]. In [2] they even say "[we] derive de main result using the approach first proposed in " and cite [1]. So the paper that everyone knows, in English (and with Bengio), explictly say that the original idea is in a paper in German, and still the scientific community chose not to cite the German original. [1] https://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdv... [2] https://sferics.idsia.ch/…

Not to excuse ignoring the thesis, but I want to point out it's bad form to cite a supposed result from a paper you haven't even looked at (or can't read), unless you indicate (admit) you didn't read it. I have even seen it described, in a paper about the citation practices of science, as unethical. It can transform hearsay into established truth. Of course nearly everyone does it anyway when they think the chance of making an erroneous citation is low enough, and that's exactly the problem.

Re: Who invented deep residual learning?

#37

Post by ekjhgkejhgk »

versteegen wrote: post #35
ekjhgkejhgk wrote: post #8

Earlier quoted context omitted.

Correct, that's [2]. In [2] they even say "[we] derive de main result using the approach first proposed in " and cite [1]. So the paper that everyone knows, in English (and with Bengio), explictly say that the original idea is in a paper in German, and still the scientific community chose not to cite the German original. [1] https://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdv... [2] https://sferics.idsia.ch/…

Not to excuse ignoring the thesis, but I want to point out it's bad form to cite a supposed result from a paper you haven't even looked at (or can't read), unless you indicate (admit) you didn't read it. I have even seen it described, in a paper about the citation practices of science, as unethical . It can transform hearsay into established truth. Of course nearly everyone does it anyway when they think the chance o…

That's exactly my point! My whole time in the academia I have not once cited a paper I hadn't read and I think doing so is a little bit dishonest at best.

They wrote the paper in German, so honest people who don't know German can't cite it directly. This is only even worth remarking on because Schmidhuber is annoyed for missing out on those specific citations.

Post reply on HN