Appreciate the honesty. AI isn’t magic, and it’s refreshing to see someone actually say it out loud.
AI in my plasma physics research didn’t go the way I expected
11–20 of 307 posts
Re: AI in my plasma physics research didn’t go the way I expected
#12I think this is mostly just a repeat of the problems of academia - no longer truth-seeking, instead focused on citations and careerism. AI is just a.n.other topic where that is happening.
I don't want to generalize because I do not know how widespread this pattern is, but my job has me hopping between a few HPC centers around Germany, and a pattern I notice is that, a lot of these places are chuck full of reject physicists, and a lot of the AI funding that gets distributed gets gobbled up by these people and the consequence of which is a lot of these ML4Science projects.
I personally think it is a bit of a shame, because HPC centers are not there to only serve physicists, and especially with AI funding we in Germany should be doing more AI-core research.
Re: AI in my plasma physics research didn’t go the way I expected
#13I am not a AI booster at all, but the fact that negative results are not published and that everyone is overselling their stuff in research papers is unfortunately not limited to AI. This is just a consequence of the way scientists are evaluated and of the scientific publishing industry, which basically suffers from the same shit than traditional media does (craving for audience). Anyway, winter is coming, innit?
Sure, it's not. But often on AI papers one sees remarks that actually mean: "...and if you throw in one zillion GPUs and make them run until the end of time you get {magic_benchmark}". Or "if you evaluate this very smart algo in our super-secret, real-life dataset that we claim is available on request, but we'd ghost you if you dare to ask, then you will see this chart that shows how smart we are".
Sure, it is often flag-planting, but when these papers come from big corps, you cannot "just ignore them and keep on" even when there are obvious flaws/issues.
It's a race over resources, as a (former) researcher on a low-budget university, we just cannot compete. We are coerced to believe whatever figure is passed on in the literature as "benchmark", without possibility of replication.
Re: AI in my plasma physics research didn’t go the way I expected
#14Re: AI in my plasma physics research didn’t go the way I expected
#15The article initially appears to suggest that all AI in science (or at least the author’s field) is hype. But their gripe seems to be specific to an architecture named PINN that seems to be overhyped, as they mention in the end how they end up using other DL models to successfully compute PDEs faster than traditional numerical methods.
Re: AI in my plasma physics research didn’t go the way I expected
#16The author is a Princeton PhD grad working in physics. Funding for this type of work usually comes from the NSF. NSF is under attack by DOGE, and Trump has proposed slashing the NSF budget by 55%. A reason used to justify these massive cuts is that AI will soon replace traditional research. This post demonstrates this assumption is likely false.
Kinda off-topic, but about the cutting itself:
I used to work in academia and was involved in NSF-funded programs, so I have mixed feelings about this. Waste and inefficiency are rampant. BTW I'm not talking about failed projects or research that were dimmed "not important", but things like buying million-dollar equipment just to use up the budget, which then sits idle.
That said, slashing NSF funding by 50% won’t fix any of that. It’ll just shrink both the waste and the genuinely valuable research proportionally. So it's indeed a serious blow to real scientific progress.
I don’t really have a point, just want to put it here. Also to be fair this kind of inefficiency isn't unique to academia; it’s common anywhere public money is involved.
Re: AI in my plasma physics research didn’t go the way I expected
#17Are complex math problems just solvable by LLMs, as a stream of language tokens?
I mean, there ought to be an element of abstract thought, abstract reasoning, abstract inter-linking of concepts, etc, to enable mathematicians to solve complex math theorems and problems.
What am I missing?
Re: AI in my plasma physics research didn’t go the way I expected
#18I understand lots of people would (rightfully) say "no shit", but I think it's good to actually describe how it is in details. So kudos to the author.
Re: AI in my plasma physics research didn’t go the way I expected
#19I am not a AI booster at all, but the fact that negative results are not published and that everyone is overselling their stuff in research papers is unfortunately not limited to AI. This is just a consequence of the way scientists are evaluated and of the scientific publishing industry, which basically suffers from the same shit than traditional media does (craving for audience). Anyway, winter is coming, innit?
Sure, it's not. But often on AI papers one sees remarks that actually mean: "...and if you throw in one zillion GPUs and make them run until the end of time you get {magic_benchmark}". Or "if you evaluate this very smart algo in our super-secret, real-life dataset that we claim is available on request, but we'd ghost you if you dare to ask, then you will see this chart that shows how smart we are". Sure, it is often…
I agree with that. Classically used "AI benchmarks" need to be questioned. In my field, these guys have dropped a bomb, and no one seem to care: https://hal.science/hal-04715638/document
Re: AI in my plasma physics research didn’t go the way I expected
#20Earlier quoted context omitted.
Sure, it's not. But often on AI papers one sees remarks that actually mean: "...and if you throw in one zillion GPUs and make them run until the end of time you get {magic_benchmark}". Or "if you evaluate this very smart algo in our super-secret, real-life dataset that we claim is available on request, but we'd ghost you if you dare to ask, then you will see this chart that shows how smart we are". Sure, it is often…
I agree with that. Classically used "AI benchmarks" need to be questioned. In my field, these guys have dropped a bomb, and no one seem to care: https://hal.science/hal-04715638/document
Can you give brief summary why this paper is a breakthrough for an outsider of the field?