21–30 of 365 posts
Earlier quoted context omitted.
Yes, I agree. It seems to me that academia feels external to the power structure, so it rarely gets the same kind of attention as industry because it doesn't stir up resentment.
They are part of the power structure, they're just 'protected'. Most publications don't want to be seen as promoting a narrative that goes 'against science' even though of course it's not. There's no room for nuance in populism.
Thank goodness I am retired and don't have to publish stuff in the current environment. Thirty years ago when machine learning was a sleepy backwater, life was a lot easier for us.
The neat thing is, the next AI winter, we'll still have massive hardware rigs. That's very different from 30 years ago. It's a lot easier to discover new things when you can just test every possibility overnight, rather than carefully planning. So if "winter is coming," it will be a lot less harsh this time.
But isn't something like this true everywhere?
And blaming the individual is futile. I like Scott Alexander's Moloch [0] concept more useful. If you apply a high standard against your own research regarding thoroughness, not overclaiming etc. never looking at the test set etc., you won't beat the benchmark, which contains entries you know are too good to be true. But if you don't beat them, you'll get rejected. The pace is extreme, lots of groups work on very similar things and it's a race to publish very similar stuff first. As a PhD student you need publications. You are in your late twenties, early thirties, you can't afford to lose several years.
The big lie is that there is so much discovery happening as suggested by the thousand upon thousands of papers at big conferences, most of which is never really engaged later on. We have to pretend that all grad students in every group can contributing some actual valuable, bulletproof novel thing to science every few months and pretend that you can have multiple papers at conferences while remaining painfully critical towards your own work. I mean how could it be that people happen to always find something that works out in the end? How can people sit into a project at the start of the PhD and immediately crank out a valuable novel contribution? It isn't realistic, but we have to pretend because this is the background assumption behind a PhD. But the current state of affairs is inflating away the value and prestige of a PhD. It means less and less because everyone knows that it's not just a measure of scientific research skill and knowledge.
However, what people are suggesting, namely making it less based on peer review and more on some social media like system is also misguided. It's like the instagrammization of the field: the sexiness, the dopamine hit, the celebrities.
But this isn't specific to AI or CS or academia. This is how humans are. It's how status hierarchies are always gamed. Think office politics, actual politics, dating.
Atvthe same time actual AI methods are really getting better, it's not a lie and not just hype. It's actually quite paradoxical how well the system works despite the flaws. Vision algorithms work really damn well, its very obvious to anyone who remembers the pre-2010 times. Perhaps there is somewhat of a stagnation in the last few years but also a lot is happening in smaller niches, as the big deep leaning ideas are adapted and applied specifically to particular tasks. The methods work, but the literature is distorted. You need to be an insider to know what to actually believe.
[0] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
The model of academia works well when some part of society is willing to fund researchers spending time on open-ended basic research, even if it produces no results. Saying "Don't engage in unethical behavior" is well and good, but if the incentives are towards unethical behavior, you shouldn't expect very much.
Academic funding and employment is very closely linked to the number of published papers. The article says that a grad student who publishes three papers a year is a professor's dream - but on reflection, it should be clear that this is not because three papers a year is necessarily good science, it's just a good metric. The professor's lab looks good for publishing so much, the professor can ask for more funding from grant agencies, and the student is likely to get a good faculty job which reflects well on the professor.
So it's extraordinarily difficult to stay in the field, have a well-funded lab, hire enough grad students, and get tenure if you're optimizing for the quality of your science over the number of publications. The "publish or perish" culture (literally, in at least one case, it seems) isn't driven by the practitioners; it's driven by the requirements of getting grants and getting academic jobs in an increasingly competitive market.
Now in some fields you do need the facilities from a university to do your research, but for CS in general - and especially for AI - you'll be just fine in industry. All you need is a pile of cloud VMs, and industry can get you that. Perhaps you also need interesting data and real-world problems; industry can get you that too.
Society today underfunds academia and overfunds industry. You, the individual researcher, are not going to be able to fix this. Go where the incentives are better aligned for you.
(And if you're a Ph.D. student who sees no future in your life after being complicit in fraud, please, please reach out to industry. Your prior publications are nowhere near as high-stakes as in an academic job; you can contact the journal, get it retracted, drop out of the program, and have a great life ahead of you. What saddens me is that the student who allegedly took his own life was probably so deeply surrounded by academia that he didn't know there's a world who won't judge him for having one fewer paper on his CV and will even look on him positively for reporting fraud and getting his advisor into career trouble.)
---
That said, my argument is undermined a bit because the papers OP denounces have coauthors in industry. So I don't have an answer for why those coauthors went along with it.
Wow, the amount of depravity in some academic circles is astonishing. The article links [1] the case of a PhD student at the University of Florida who was forced to participate in such a publication collusion ring and was pressured to commit scientific fraud by fabricating results and submitting them to a conference [2], being threatened with physical harm should he decide to go public. This student saw no way out an…
Imagine if this happened in private enterprise, like at Google. The press would be all over it 24/7.
Wow, the amount of depravity in some academic circles is astonishing. The article links [1] the case of a PhD student at the University of Florida who was forced to participate in such a publication collusion ring and was pressured to commit scientific fraud by fabricating results and submitting them to a conference [2], being threatened with physical harm should he decide to go public. This student saw no way out an…
[1] https://en.wikipedia.org/wiki/Online_shaming#Tim_Hunt_contro... [2] https://eu.gainesville.com/story/news/education/campus/2021/...
One surprising discovery about AI: There's nothing inherently wrong with cherrypicking. I used to think it was this dirty thing. Oh, you're only showing your best results? But the best results are what matters. If you can create an amazing song from scratch by telling a computer what to do, you only need to do it once. The song is still good. And it's easy to automate. If you have a way of detecting a good song, or a…
Not the kind of cherry-picking condemned here, which is more like your drug killing 90% of the animals it was tested on, and making a small difference in 2% of them, and you present it as only that 2% ever happened (and thus represent 100% of the cases).
This is all due to the criteria for survival and graduation. Let’s be honest, not every idea will work and this does not reflect badly on the researcher who came up with it. But the only way to graduate and put food on the table is to keep getting funding through novel papers that have great results.