AI boosts research careers but narrow the span of ideas explored: study

That doesn't disagree with this article. Proving a theorem that a human already proposed in an existing discipline of math - math, the most formalized and easiest discipline to involve computers in even before LLMs - is very different from expanding the boundaries of science.

> AI is largely automating the most tractable parts of science rather than expanding its frontiers

By definition, creativity cannot be automated, and AI is a fantastic automation machine. It can explore thinking paths at a rate humans cannot match. But creativity is bringing the unthinkable into the thinkable, and that requires sensory experience [1]. Specifically, new definitions and symbols which never existed before. Imagine the concept vector space, and expanding that with new independent dimensions. Is that even possible ? When you look at history the answer is yes !. And each time there was an independent dimension added, it was an act of genius. It is an instructive exercise to name these moments in history where an independent dimension was added to human thought. Some examples in math would be the invention of a number, and in politics could be the idea of democracy. By contrast, LLMs are trapped in the vector space they are trained on, and they lack the feedback loop with sensory experience to be able to create and validate theories.

[1] https://philsci-archive.pitt.edu/28024/1/Scientific_Inventio...

I agree with some parts, but not all.

I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become bad. This is common in programming too—things that are correct in the beginning but become problematic as they grow larger.

From an individual researcher's perspective, it's rational. You get more citations, your career accelerates. Everyone knows this. Paper counts aren't everything. Citation counts aren't everything. Journal impact factors aren't everything. You shouldn't only play it safe. But everything is tied to those metrics anyway.

Most researchers who give me work are fully aware of these facts. But are they going to change anything? Funding is still distributed based on those metrics.

Max Planck said, 'Science advances one funeral at a time.' Science doesn't progress purely through reasoned argument. The authority of the older generation, research funding networks, journals, and school-specific evaluation criteria all move together.

And honestly, I think discoveries will keep happening—probably quite rapidly. Because AI doesn't have the factional conflicts or interpersonal issues that humans do. It's very good at connecting papers across schools of thought without bias. In other words, the current human system is flawed at consolidating research, but I think AI is actually strong in this area. I expect AI-driven discoveries will continue for some time. The people who ride this wave will clearly be the winners.

Everyone knows things are broken, but no one is trying to fix them. I always think human society is inefficient. I read this post, but I'm more curious about who will actually lead the improvement effort.

"Science advances one funeral at a time"

Well, these AI are never going to die in any real sense, so expect them to make orthodoxy more sticky, not less.

I don’t think we have spent enough time on the creativity axis.

When we solve problems we usually follow a heuristically guided energy efficient path. We just prune a lot of possibilities based on our existing knowledge and experience.

Creativity happens when we consciously (or not) go off the beaten path and explore. Most of those explorations are dead ends. But some will yield unexpected connections, patterns etc that we call “creativity” .

An AI system could also go on those kinds of explorations. Today they aren’t it because we are not asking them to.