Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Turing awardees republished key methods and ideas without credit

people.idsia.ch

Post reply on HN
Print view View original

51–60 of 105 posts

Re: Turing awardees republished key methods and ideas without credit

#51

Post by vbarrielle »

logicchains wrote: post #44
RcouF1uZ4gsC wrote: post #5

Reading the article and some of the links make me feel like the author Jürgen Schmidhuber is the academic version of the patent troll. It sounds like he published some theoretical musings back in 1990s without any real practical implementation that did anything useful and since then has run around accusing AI researchers who actually produced concrete research and techniques to get are actually in use today of plagia…

>It sounds like he published some theoretical musings back in 1990s without any real practical implementation He published working models back then, the problem was compute power was very limited. In the past decade deep learning took off, people took his models, renamed then and ran them on vastly more powerful computers, to great success, then failed to cite him.

It's actually a common problem in lots of computing related research communities. Papers older than 10 years ago, sometimes even 5 years ago, are ignored. Because the results do not compare in terms of computing power, and because reimplementing old papers is often very hard because they are often too vague.

Re: Turing awardees republished key methods and ideas without credit

#52

Post by erostrate »

logicchains wrote: post #37
erostrate wrote: post #11

List of "famous" ML people not to waste time on: - Gary Marcus - Juergen Schmidhuber - Pedro Domingos - Max Tegmark - Eliezer Yudkowsky Some context for people unfamiliar with ML research: the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that: - He doesn't deserve the credit he claims, in most if not all cases. - There's a few cases where his p…

>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases. That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.

Agreed, Ian Goodfellow wasn't the first to come up with the idea of jointly training a generator and discriminator. But he was the first to make it work for image generation with modern neural networks. For that Ian Goodfellow deserved to get most of the credit, and he did.

Re: Turing awardees republished key methods and ideas without credit

#53

Post by abdullahkhalids »

erostrate wrote: post #36
YeGoblynQueenne wrote: post #22

Earlier quoted context omitted.

>> Credit belongs to whoever actually makes it work. That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.

> credit belongs to the people who deserve credit for the work they've done that was useful to others Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets. For example, just taking a random paper that Schmidhuber claims…

In many fields, this is how citations would work

"Introductory theoretical work in GAN was done by Schmidhuber [1], but it was not until large experimental efforts [2,3,4] on image generations that the power of GANs was revealed."

Re: Turing awardees republished key methods and ideas without credit

#54

Post by erostrate »

GlenTheMachine wrote: post #19
erostrate wrote: post #11

List of "famous" ML people not to waste time on: - Gary Marcus - Juergen Schmidhuber - Pedro Domingos - Max Tegmark - Eliezer Yudkowsky Some context for people unfamiliar with ML research: the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that: - He doesn't deserve the credit he claims, in most if not all cases. - There's a few cases where his p…

Why Tegmark?

He talks a lot about the singularity but the biggest singularity I can see is in his ratio of "talks about AI" divided by "actual AI contributions".

Re: Turing awardees republished key methods and ideas without credit

#55

Post by jltsiren »

KRAKRISMOTT wrote: post #43
logicchains wrote: post #38

Earlier quoted context omitted.

>atrophying his skills too much to do anything now that the industry is moving so quikcly. His recent papers are still cutting edge. He's already solved self-improving AI: https://arxiv.org/abs/2202.05780 , it just needs to be scaled up.

His students you mean, otherwise he would be first author. Academia is just like capitalism, with credits substituting for capital. The capital owner (laboratory head/professor) always gets a cut of everything published.

Authorship conventions vary a lot within the academia. In general, the closer the name is to either end of the author list, the more significant their contributions likely were.

Things also vary from paper to paper. Sometimes the first author just did the actual work for somebody else, and sometimes they also made significant intellectual contributions. (If the first author is listed as the sole corresponding author, it usually indicates the latter.) Sometimes the last/senior author just brought the money in, sometimes they were primarily mentoring the first author, and sometimes they were the driving force behind the project.

Re: Turing awardees republished key methods and ideas without credit

#56

Post by dr_dshiv »

YeGoblynQueenne wrote: post #24

This whole comment section is full of absolutely unacceptable ad-hominem attacks on Schmidhuber, from people who most likely haven't even read any of the works in question and are certainly not showing any of the "intellectual curiosity" this site is supposed to be about. Anyone who cares about academic integrity should at least not attack someone complaining of plagiarism. That sort of attack is the academic equival…

I came to Schmidhuber's work learning about his approach on creativity and curiosity and I loved it so much. It’s still so relevant. And beautiful, actually.

https://arxiv.org/pdf/0709.0674.pdf

Re: Turing awardees republished key methods and ideas without credit

#57

Post by caddemon »

erostrate wrote: post #36
YeGoblynQueenne wrote: post #22

Earlier quoted context omitted.

>> Credit belongs to whoever actually makes it work. That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.

> credit belongs to the people who deserve credit for the work they've done that was useful to others Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets. For example, just taking a random paper that Schmidhuber claims…

I agree a significant amount of work (and often insight too) is needed to translate an architecture idea into something that works in practice, and there are certainly plenty of ideas that are obvious in the abstract. But I also think it's important to avoid dismissing work only on the basis that it doesn't involve "real life datasets".

Deep learning is a relatively unexplored field and there are many open mathematical and scientific questions to ask that involve only model equations or contrived datasets. Novel theoretical results are not just about some architecture idea but about proving facts that can be useful for understanding how the model class would perform in different scenarios. Which in turn can help shape the search space for applied work.

Additionally, I don't think credit assignment should be so discrete. 100% agree that vomiting out vague ideas shouldn't grant claims to credit, but academic science much too often gives only a single author the "real" credit.

Incidentally, in other fields the person who actually makes it work very well may not be the person that receives this credit. Like biology can involve a lot of hard manual work (that isn't really intellectual) in order to realize a project plan. It varies how much of the credit those people receive, and I'm not even sure how much they should receive. This topic is extremely nuanced.

Re: Turing awardees republished key methods and ideas without credit

#58

Post by anonymousDan »

erostrate wrote: post #36
YeGoblynQueenne wrote: post #22

Earlier quoted context omitted.

>> Credit belongs to whoever actually makes it work. That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.

> credit belongs to the people who deserve credit for the work they've done that was useful to others Certainly agree. The point is that coming up with the idea, writing it as an equation, or an architecture diagram in a paper, is a small fraction of the effort that goes into making the idea work in a model showing good performance on real life datasets. For example, just taking a random paper that Schmidhuber claims…

I don't buy that this is standard practice in the ML community, and even if it is it's BS. If the basic idea/principle has been published previously but in a different context you should cite it and say why the solution is not directly applicable or has not been evaluated in the current context. Anything else is unprofessional.

Re: Turing awardees republished key methods and ideas without credit

#59

Post by P-NP »

I can see some angry comments here, but so far I have not seen any facts that refute his claims. Once I spent a long time reviewing a related paper on Hacker News, and I think he is right about disputes B1, B2, B5, H2, H4, H5. I'd have to study the others more closely:

B: Priority disputes with Dr. Bengio (original date v Bengio's date): B1: Generative adversarial networks or GANs (1990 v 2014) B2: Vanishing gradient problem (1991 v 1994) B3: Metalearning (1987 v 1991) B4: Learning soft attention (1991-93 v 2014) for Transformers etc. B5: Gated recurrent units (2000 v 2014) B6: Auto-regressive neural nets for density estimation (1995 v 1999) B7: Time scale hierarchy in neural nets (1991 v 1995)

H: Priority disputes with Dr. Hinton (original date v Hinton's date): H1: Unsupervised/self-supervised pre-training for deep learning (1991 v 2006) H2: Distilling one neural net into another neural net (1991 v 2015) H3: Learning sequential attention with neural nets (1990 v 2010) H4: NNs program NNs: fast weight programmers (1991 v 2016) and linear Transformers H5: Speech recognition through deep learning (2007 v 2012) H6: Biologically plausible forward-only deep learning (1989, 1990, 2021 v 2022)

L: Priority disputes with Dr. LeCun (original date v LeCun's date): L1: Differentiable architectures / intrinsic motivation (1990 v 2022) L2: Multiple levels of abstraction and time scales (1990-91 v 2022) L3: Informative yet predictable representations (1997 v 2022) L4: Learning to act largely by observation (2015 v 2022)

Re: Turing awardees republished key methods and ideas without credit

#60

Post by P-NP »

refulgentis wrote: post #50
logicchains wrote: post #37

Earlier quoted context omitted.

>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases. That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.

Frankly, I do, and it comes across as quibbling about categories and trying to define unnecessarily general taxonomic categories, as a reaction to positive reactions to other people. ex. in the article: "Goodfellow eventually admitted that my PM is adversarial...but emphasized that it's not generative. However, [it] is both adversarial and generative (its generator contains probabilistic units)...It is actually a gen…

This sounds a bit like a justification of plagiarism. In science, you must cite the original work.
Post reply on HN