41–50 of 365 posts
Scientific community tainted itself by producing fake medical cannabis "research" as they could only get funding if their "studies" could prove negative effects.
Out of curiosity could you refer me to some sources to read more about this.
This article scratches the surface.
One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your best results? But the best results are what matters. If you can create an amazing song from scratch by telling a computer what to do, you only need to do it once. The song is still good. And it's easy to automate. If you have a way of detecting a good song, or a…
> . If you have a way of detecting a good song, or at least filtering out rubbish, then you can generate thousands of attempts to get the good result. So often the filtering technique is a human in the loop, manually going through the output. Humans become the hidden, undifferentiatable and undisclosed loss function.
It took a long time to accept that this is both normal and effective. It's not some hidden secret to be ashamed of. The headline images in https://openai.com/blog/dall-e/ are probably the best and most interesting images they could find; they were picked out by humans. It's a positive statement to say that dall-e can do that some of the time, rather than "it fails X% of the time."
I ran into these issues in an adjacent field a generation ago. Many candidly spoke to me about having to be a ‘team player’. This is all due to the criteria for survival and graduation. Let’s be honest, not every idea will work and this does not reflect badly on the researcher who came up with it. But the only way to graduate and put food on the table is to keep getting funding through novel papers that have great re…
> this does not reflect badly on the researcher who came up with it. > But the only way to graduate and put food on the table is to keep getting funding through novel papers that have great results. So in the way in which it matters, it does reflect badly on the researcher who came up with it.
Earlier quoted context omitted.
How is it possible that professors with impeccable academic credentials get fired for jokes nearly instantly [1], yet this student's professor was allowed to carry on until he resigned two years later [2]? [1] https://en.wikipedia.org/wiki/Online_shaming#Tim_Hunt_contro... [2] https://eu.gainesville.com/story/news/education/campus/2021/...
Well... non-PC jokes are a liability risk for the institution at large. Dead students are not. We have an identical phenomenon in hospitals: there are huge campaigns against sexism and the like, while there are multiple young professionals committing suicide from burn-out every year and noone does as much as bat an eyelash. Contrary to victims of discrimination, dead people can't easily organize into coordinated lega…
Wow, the amount of depravity in some academic circles is astonishing. The article links [1] the case of a PhD student at the University of Florida who was forced to participate in such a publication collusion ring and was pressured to commit scientific fraud by fabricating results and submitting them to a conference [2], being threatened with physical harm should he decide to go public. This student saw no way out an…
Imagine if this happened in private enterprise, like at Google. The press would be all over it 24/7.
Wow, the amount of depravity in some academic circles is astonishing. The article links [1] the case of a PhD student at the University of Florida who was forced to participate in such a publication collusion ring and was pressured to commit scientific fraud by fabricating results and submitting them to a conference [2], being threatened with physical harm should he decide to go public. This student saw no way out an…
How is it possible that professors with impeccable academic credentials get fired for jokes nearly instantly [1], yet this student's professor was allowed to carry on until he resigned two years later [2]? [1] https://en.wikipedia.org/wiki/Online_shaming#Tim_Hunt_contro... [2] https://eu.gainesville.com/story/news/education/campus/2021/...
"It's strange that such a chauvinist monster like me has been asked to speak to women scientists. Let me tell you about my trouble with girls. Three things happen when they are in the lab: you fall in love with them, they fall in love with you, and when you criticise them they cry. Perhaps we should make separate labs for boys and girls? Now, seriously, I'm impressed by the economic development of Korea. And women scientists played, without a doubt, an important role in it. Science needs women, and you should do science, despite all the obstacles, and despite monsters like me."
That is not a joke.
One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your best results? But the best results are what matters. If you can create an amazing song from scratch by telling a computer what to do, you only need to do it once. The song is still good. And it's easy to automate. If you have a way of detecting a good song, or a…
> But the best results are what matters. Not the kind of cherry-picking condemned here, which is more like your drug killing 90% of the animals it was tested on, and making a small difference in 2% of them, and you present it as only that 2% ever happened (and thus represent 100% of the cases).
Trying that shiny new algorithm out on a couple dozen seeds, and then only reporting the best few. Running a big hyperparameter sweep on your proposed approach but using the defaults for the baseline. Cherry-picking examples where your model looks good, or cherry-picking whole datasets to test on, where you’ve confirmed your model’s advantage. Making up new problem settings, new datasets, new objectives in order to claim victory on an empty playing field. Proclaiming that your work is a “promising first step” in your introduction, despite being fully aware that nobody will ever build on it.
Yeah, I misread; sorry. Testing your model on a small dataset and presenting it as a general solution is all too common.
That's why generative models are cooler than classifiers, to me at least. You can show the outputs visually, whether it's text or image or sound. But with classifiers, you're chasing an accuracy rating. I forgot that people often test on CIFAR-10 (smol data) without verifying on Imagenet (big-ish data) and then present their paper as very general.
But, I do have something to say about that kind of dataset cherrypicking: Researchers often find it hard to test on large datasets because of limited compute resources. One of the pioneers of DDPM wasn't able to train on imagenet due to lack of GPUs. CIFAR-10 was the best they could do. So it's sometimes hard to tell whether there's intentional deceit, or just a shoestring budget. (I was surprised how much research a lot of people get done in spite of limitations.)
Thanks for the correction; cheers.
Yes, as someone in the system, it's all true and an open secret, everyone knows it and we complain about it over some beers all the time. But isn't something like this true everywhere? And blaming the individual is futile. I like Scott Alexander's Moloch [0] concept more useful. If you apply a high standard against your own research regarding thoroughness, not overclaiming etc. never looking at the test set etc., you…
> everyone knows it and we complain about it over some beers all the time. Then why not speak out publicly against it? Obviously because you know that there are still a significant number of individuals that want the system to work this way. Your fear of those individuals is why you don't speak out against it. So who are they? Who are you scared of?
It's not "who are they", its everyone. The incentives are set up wrong. Blaming individuals is insufficient. It's like dreaming if only politicians stopped lying. Academia is a hierarchy and a power system. People have interests, reputation, prestige and it often directly translates to money.
Again, I'm not talking about blatant outright lies or fudging numbers.
Thing is, after some rejections or sabotaging yourself a few times by being too critical with your own research and not having publications you will imitate what you see around you or you drop out due to a lack of publications. It's rewarded to be less critical with yourself. And ultimately you want to graduate after all that sunk cost. Even switching to industry isn't so easy without publications in prestigious venues to show for those years.
Earlier quoted context omitted.
With laws, there are full time police officers, investigators, and prosecutors working to catch people. And even then, most crimes go unsolved and unpunished. Even without the complicity, stopping fraud and bad behavior is a difficult problem. Part time reviewers and reputation seem woefully insufficient counters.
Yes, I'm very skeptical of Buckman's claim that more fraud will help. Seems similar to a claim around 2015-2016 that voting for Donald Trump will inspire America to clean up its act politically. Institutions aren't biological systems that operate according to mysterious hormetic processes. Institutions are created by humans and thrive or decay based on whether they effectively channel human effort. Be the change you…
1) If people realize it's more like a Facebook post, that will be better. People will be able to push back.
2) Alternatively, if we clean up this mess, that will be even better. Academia ought to be trustworthy.
I don't see a path to #2 without a lot of dishonest people with tenure being laid off. At elite institutions, most people hired in the past decade or two cheated at least a little bit. I don't see a path to get there without a high-profile scandal.
Earlier quoted context omitted.
Imagine if this happened in private enterprise, like at Google. The press would be all over it 24/7.
You mean the number of suicides at foxconn? The forced labour that is likely contributing to many of the large mobile phone company bottomlines. The blatant benchmark fraud that happens all the time by all the large GPU and CPU manufacturers, I could go on. But I don't see the press all over this 24/7 t all, maybe some small niche outlets sometimes. In comparison the scrutiny that the academic world is under (if we r…
That we have heard about it, implies a degree of scrutiny.
When Google fires an AI researcher in a sensitive position, it's an international event.
If Exxon executives committed suicide in the face of some kind of forced fraud/corruption, it would be a national story.