Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosts research careers but narrow the span of ideas explored: study

spectrum.ieee.org

Post reply on HN
Print view View original

21–30 of 112 posts

Re: AI boosts research careers but narrow the span of ideas explored: study

#21

Post by skybrian »

bwfan123 wrote: post #13

> AI is largely automating the most tractable parts of science rather than expanding its frontiers By definition, creativity cannot be automated, and AI is a fantastic automation machine. It can explore thinking paths at a rate humans cannot match. But creativity is bringing the unthinkable into the thinkable, and that requires sensory experience [1]. Specifically, new definitions and symbols which never existed befo…

That paper argues that an LLM “lacks the mechanism for Abduction,” which is not the same thing as a claim that “creativity cannot be automated.” They propose a different kind of AI:

> The emergence of physically consistent World Models offers a pathway to a synthetic laboratory. By enabling agents to run counterfactual simulations—to experience the physical consequences of a thought experiment—we may finally mechanize the feedback loop between intuition and logic.

Re: AI boosts research careers but narrow the span of ideas explored: study

#22

Post by nathan_compton »

BurningFrog wrote: post #16
xmcp123 wrote: post #3

“Technology that is based on everything humanity has already done, fails to do things that humanity has not yet done”

Wasn't Einstein's discoveries based on things humanity had already done? AIs do things no human has done before millions of times a day.

Einstein's discoveries were based (to a large degree) on negating very specific parts of scientific orthodoxy and then taking the steps forward to carefully derive results with those rejections in place.

LLMs are aggressively trained to reproduce facts and consequently struggle to reject orthodoxy. There isn't any reason they can't, in principal, make big new discoveries just by getting lucky, which is sort of also how humans do it, but its ok to acknowledge that current AIs aren't so good at certain things.

Re: AI boosts research careers but narrow the span of ideas explored: study

#23

Post by Marha01 »

nathan_compton wrote: post #17
jdw64 wrote: post #15

I agree with some parts, but not all. I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become…

"Science advances one funeral at a time" Well, these AI are never going to die in any real sense, so expect them to make orthodoxy more sticky, not less.

AIs get replaced with newer models.

Re: AI boosts research careers but narrow the span of ideas explored: study

#24

Post by CuriouslyC »

Arainach wrote: post #10
Nevermark wrote: post #5

Any flattening of discovery due to AI, but will be temporary. We tend to think that obvious potential is the same as realized potential, for new technology. For any specific context, there are generally innumerable smaller adaptations and capability thresholds that have to be crossed. And the price for that journey is often temporary loss off overt productivity.

No, this is significantly more permanent. LLMs are autocomplete generators based off current context, and training generations of people to always ask the planet burners instead of learning to think for themselves - and never having the experience of having to slowly think over the same thing for an extended period - may well mean a permanent cap to human knowledge and a dramatic slowdown or end to new knowledge.

You act like humanity doesn't exist in a competitive environment. If you think AI codegen is a mistake? Just relax, keep writing code by hand and wait for the pendulum to prove you right while showering you in wealth. There are plenty of people making this bet, and I wish the best of luck to you because I'm 99% certain you're on the losing end of it.

Re: AI boosts research careers but narrow the span of ideas explored: study

#25

Post by jdw64 »

nathan_compton wrote: post #17
jdw64 wrote: post #15

I agree with some parts, but not all. I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become…

"Science advances one funeral at a time" Well, these AI are never going to die in any real sense, so expect them to make orthodoxy more sticky, not less.

I agree. AI will likely reinforce mainstream schools of thought through literature. I think I used the wrong example in this case—I should have framed it as the system itself rather than specific schools of thought. Thanks for the correction

Re: AI boosts research careers but narrow the span of ideas explored: study

#26

Post by spongebobstoes »

Arainach wrote: post #10
Nevermark wrote: post #5

Any flattening of discovery due to AI, but will be temporary. We tend to think that obvious potential is the same as realized potential, for new technology. For any specific context, there are generally innumerable smaller adaptations and capability thresholds that have to be crossed. And the price for that journey is often temporary loss off overt productivity.

No, this is significantly more permanent. LLMs are autocomplete generators based off current context, and training generations of people to always ask the planet burners instead of learning to think for themselves - and never having the experience of having to slowly think over the same thing for an extended period - may well mean a permanent cap to human knowledge and a dramatic slowdown or end to new knowledge.

when a parent answers their child's question, does it decrease the curiosity of the child?

many children have an unlimited capacity to ask "why?". many adults are the same

if the abilities of AI are finite, then we will continue to have burning curiosity, questions to ask, and discoveries to make

Re: AI boosts research careers but narrow the span of ideas explored: study

#27

Post by Diogenesian »

Labo333 wrote: post #6

> “It’s not about the architecture per se,” Evans says. “It’s about the incentives.” It would have been useful to check whether less original work was already getting more citations before AI adoption. That could reflect broader trends and network effects: heavily cited research areas attract more authors optimizing for citations, so high-productivity researchers end up clustering on the same topics.

They did. The article explains tbat this is a trend which has been getting worse for years, specifically pointing to search engines as a major turning point. Your comment is completely off the mark.

Re: AI boosts research careers but narrow the span of ideas explored: study

#28

Post by jltsiren »

jdw64 wrote: post #15

I agree with some parts, but not all. I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become…

> Because AI doesn't have the factional conflicts or interpersonal issues that humans do.

All the factional conflicts are in there, and there are also plenty of reports of people getting weird / toxic / passive aggressive responses from AI.

Because the model is trained with everything, you can in principle get anything out of it. You want to get an answer based on all the right things, while keeping all the wrong things suppressed. But it's easy to get something less than ideal, due to the specifics of training, harnesses, context, prompts etc.

Re: AI boosts research careers but narrow the span of ideas explored: study

#29

Post by Arainach »

esafak wrote: post #12
Arainach wrote: post #11

Earlier quoted context omitted.

That doesn't disagree with this article. Proving a theorem that a human already proposed in an existing discipline of math - math, the most formalized and easiest discipline to involve computers in even before LLMs - is very different from expanding the boundaries of science.

How is it different? Before there was no proof, and now there is. What counts as expanding the boundary to you?

Identifying what questions to ask is often much harder than answering them. Proposing new theorems - and new areas of investigation - is what expands boundaries. Proving them is confirmation.

Once the Pythagorean theorem was proposed, many different proofs have been identified. In art, once a new style is created it's often straightforward for others to replicate. In physics, the idea of Relativity was what enabled the design of experiments to demonstrate its correctness. Proposing the idea is what's essential.

Re: AI boosts research careers but narrow the span of ideas explored: study

#30

Post by jdw64 »

jltsiren wrote: post #28
jdw64 wrote: post #15

I agree with some parts, but not all. I see it as an overfitting problem. Fundamentally, the topic here seems to be that citation indices and similar metrics are actually flawed indicators, and obsessing over them is just Goodhart's law in action. Ultimately, the argument is that the entire design of those metrics is wrong. To be precise, it was a good metric at first, but now that the scale has changed, it's become…

> Because AI doesn't have the factional conflicts or interpersonal issues that humans do. All the factional conflicts are in there, and there are also plenty of reports of people getting weird / toxic / passive aggressive responses from AI. Because the model is trained with everything, you can in principle get anything out of it. You want to get an answer based on all the right things, while keeping all the wrong thi…

I was too hasty in drawing my conclusion.I didn't think it through thoroughly enough.you're right
Post reply on HN