Turing awardees republished key methods and ideas without credit • Hacker News Board

Re: Turing awardees republished key methods and ideas without credit

#101

Post by martopix » Sat, Dec 16, 2023, 11:00 AM UTC

logicchains wrote: post #40

martopix wrote: post #18

Earlier quoted context omitted.

If a guy becomes famous almost exclusively for having a beef with someone else (and this someone else really doesn't need to beat any drum in order to be famous), then something is wrong. Basically these days this is what Schmidhuber is associated to. Which is kind of sad, really.

>If a guy becomes famous almost exclusively for having a beef with someone else He was famous well before that for having one of the best labs in Europe, and many key papers in early deep learning. He's only famous for the beef among people who aren't familiar with his earlier work and the huge contributions it made to the field.

I should not have said "becomes famous for...". I meant that today, his name is mostly associated with that, when people talk about him.

Re: Turing awardees republished key methods and ideas without credit

#102

Post by oldesthacker » Sun, Dec 17, 2023, 9:09 AM UTC

The machine learning field as a whole has a huge credit assignment problem. This post seems to encourage other ML researchers to come out with their own priority disputes. Tomas Mikolov just aired his grievances:

> I wanted to popularize neural language models by improving Google Translate. I did start collaboration with Franz Och and his team, during which time I proposed a couple of models that could either complement the phrase-based machine translation, or even replace it. I came up (actually even before joining Google) with a really simple idea to do end-to-end translation by training a neural language model on pairs of sentences (say French - English), and then use the generation mode to produce translation after seeing the first sentence. It worked great on short sentences, but not so much on the longer ones. I discussed this project many times with others in Google Brain - mainly Quoc and Ilya - who took over this project after I moved to Facebook AI. I was quite negatively surprised when they ended up publishing my idea under now famous name "sequence to sequence" where not only I was not mentioned as a co-author, but in fact my former friends forgot to mention me also in the long Acknowledgement section, where they thanked personally pretty much every single person in Google Brain except me. This was the time when money started flowing massively into AI and every idea was worth gold. It was sad to see the deep learning community quickly turn into some sort of Game of Thrones. Money and power certainly corrupts people...

Reddit post: "Tomas Mikolov is the true father of sequence-to-sequence" https://www.reddit.com/r/MachineLearning/comments/18jzxpf/d_...

Re: Turing awardees republished key methods and ideas without credit

#103

Post by oldesthacker » Sun, Dec 17, 2023, 5:38 PM UTC

jahsome wrote: post #77

lacker wrote: post #30

Earlier quoted context omitted.

Whenever you cite papers just because you're "politically supposed to", even though you don't think it's really intellectually useful for anyone to be aware of the cited paper, you're making the citations less useful for everybody. Schmidhuber is only the worst of many abusers of the citation system. Everyone knows that anonymous reviewers are more likely to approve of people who cite their work. Someone is political…

> Whenever you cite papers just because you're "politically supposed to", even though you don't think it's really intellectually useful for anyone to be aware of the cited paper, you're making the citations less useful for everybody. The audacity of deciding on behalf of readers whether information is valuable to them when publishing an academic paper is quite an interesting concept to me. I'm not published in journa…

> Isn't that exercising political capital?

These are very good points.

> ... the worst of many abusers of the citation system.

Is this ad hominem argument by user "lacker" meant to distract from the omissions of the awardees Bengio, Hinton, and LeCun? Especially the first two got tons of citations for work that should have credited Schmidhuber's lab: the analysis of vanishing gradients in neural networks, the principle of generative adversarial networks, attention in neural networks, distilling neural networks, speech recognition with LSTM neural networks, self-supervised pre-training, and more.

The disputes with LeCun are more recent and of lesser magnitude IMO.

Re: Turing awardees republished key methods and ideas without credit

#104

Post by oldesthacker » Sun, Dec 17, 2023, 6:27 PM UTC

calf wrote: post #29

TimPC wrote: post #13

Earlier quoted context omitted.

I think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make…

But if other people identify and further develop one or two out of his models, he should still be cited. If they did not use his work at all, coming ith one or two models independently, then that's a different situation. It's a bit of an honor code thing as well, it's hard to prove if somebody has a read a paper or not. But then there's a more stringent standard where one cites as part of surveying preexisting work,…

Let's take an example: his "unnormalised linear Transformer," a neural network with "self-attention" published in 1992 under another name. It wasn't just an idea, it was implemented and tested in experiments. However, few people cared, because the computational hardware was so slow, and such networks were not yet practical. Decades later, the basic concepts were "reinvented" and renamed, and today they are really useful on much faster computers. Unquestionably, however, their origins must be cited by those who are applying them.

Why are some people here even debating the generally recognised rules of scientific publishing mentioned in the paper:

> The deontology of science requires: If one "reinvents" something that was already known, and only becomes aware of it later, one must at least clarify it later, and correctly give credit in all follow-up papers and presentations.

Re: Turing awardees republished key methods and ideas without credit

#105

Post by ahazred8ta » Sun, Dec 24, 2023, 4:58 AM UTC

calf wrote: post #89

ahazred8ta wrote: post #81

Earlier quoted context omitted.

"The literature on this problem is extensive, widely scattered, and not always aware of itself."

Is that quote from something? It's funny. I couldn't find any Google result for it.

an article on Malfatti circles in geometry https://en.wikipedia.org/wiki/Malfatti_circles#History