Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosts research careers but narrow the span of ideas explored: study

spectrum.ieee.org

Post reply on HN
Print view View original

81–90 of 112 posts

Re: AI boosts research careers but narrow the span of ideas explored: study

#81

Post by washadjeffmad »

aborsy wrote: post #73

A new breed of academics has appeared whose jobs is to put their names in every paper possible. Literally, their job is to work on frameworks to buy co-authorship. They do this in various ways, like establishing paper pipelines, collecting rents on labs and committees, focusing on money layer, using their profiles and citation count to help with acceptance of papers of other people , etc. You talk to them and they ca…

It's not new, and entire disciplines exist because of these dependent structures. It doesn't do to unseat and discredit the connected and well regarded, but you might enjoy some mild comfort in security in numbers through a little citational flattery.

It's a game of cultural tribalism. The only thing worse for one than not engaging is to upset the status quo unblessed.

Re: AI boosts research careers but narrow the span of ideas explored: study

#82

Post by claytongulick »

Nevermark wrote: post #76
claytongulick wrote: post #58

Earlier quoted context omitted.

Richard Sutton apparently disagrees [1]. He argues that it's impossible for anything novel to come from a LLM. [1] https://youtu.be/kEbVTcncuX0?is=gEMe5zD9sXWD4ONy

Actually: > its output can be novel or good, but rarely both at the same time. > rarely That is not a viewpoint they can't do something useful and new. With that criteria, he could be talking about anyone. I find it rare that people critiquing AI today, actually hold people to the same standards. Or are as enthusiastic about referencing ways machines keep surpassing us, as for ways they have not yet, when speaking ab…

LLMs are the tech in question, not ML (AI?) in general.

LLMs are fundamentally limited by their architecture to only return a token predicted by a statistical inference, essentially lossy decompression.

It's like arguing that taking an image, compressing it with JPEG and low quality, then decompressing it into something blurry with some random color values thrown in is creating new art.

No one is arguing that everything a human has created is good. No one is arguing that LLMs can't be useful.

Sutton is arguing that it can't be novel. Cherry picking a couple of words doesn't change his argument, which is very clear.

Re: AI boosts research careers but narrow the span of ideas explored: study

#83

Post by onraglanroad »

nathan_compton wrote: post #22
BurningFrog wrote: post #16

Earlier quoted context omitted.

Wasn't Einstein's discoveries based on things humanity had already done? AIs do things no human has done before millions of times a day.

Einstein's discoveries were based (to a large degree) on negating very specific parts of scientific orthodoxy and then taking the steps forward to carefully derive results with those rejections in place. LLMs are aggressively trained to reproduce facts and consequently struggle to reject orthodoxy. There isn't any reason they can't, in principal, make big new discoveries just by getting lucky, which is sort of also h…

I was under the impression that it was more accepting the othodoxy of Galilean/Newtonian relativety and joining it up with Maxwell's discovery about electromagnetism.

So if the speed of propogation of EM waves is the same no matter your frame of reference (along with all the rest of physics) then the speed of light can't be relative (a conclusion that was aided by the Michaelson-Morley experiment) and what are the logical consequences of that.

If I'm incorrect in my understanding I'd appreciate any correction.

Re: AI boosts research careers but narrow the span of ideas explored: study

#84

Post by rbartelme »

As a bioinformatics person that's spent time in and out of industry/academia, I agree with some of the article's thesis. While I don't think LLMs or AI are going away, I do think it will allow people in academia to pump out a bunch of inane papers and continue to prop up predatory scientific journal publishing via tenure and promotion. In fact outside of how utterly useless Fable 5 is via their aggressive guard rails for my work, I quite like using statically typed and/or functional languages with other LLMs since there are some baked in guardrails via compiler + type system.

I think the flattening of progress is the most interesting dimension to the article. For an example a useful biological product discovery with a nonlinear path to get to there, look at the Taq polymerase (https://en.wikipedia.org/wiki/Taq_polymerase). Without some NSF funded exploratory ecological research by Tom Brock in Yellowstone Hot Springs to test the theoretical limit of life at high temperatures (https://en.wikipedia.org/wiki/Thermus_aquaticus) we never get to the Taq polymerase, we never get reliable/robust PCR (https://en.wikipedia.org/wiki/Polymerase_chain_reaction), which is now a gold standard method in both clinical and environmental testing! It is rather improbable to think that large language models would associate those domain connections across the topic (molecular biotechnology + ecology + microbial physiology). I also did some exploratory work with text embedding models people might use for RAG and challenged them with an open source scientific MCA question dataset, generalist embedders performed worse vs. domain specific embedders trained on scientific corpora (doesn't surprise me at all). However, if everything regresses to the median of the universe of possible knowledge, it seems like scientific leaning frontier models would get locked into this asymptotic flattening before turning cashflow positive for model vendors OR they become so locked down that only big pharma, state actors, or big ag can afford the API rates and vetting process.

Re: AI boosts research careers but narrow the span of ideas explored: study

#85

Post by matthewdgreen »

skeledrew wrote: post #9

As with other fields touched, AI is merely amplifying what was already there. The aim of many scientists isn't discovery in and of itself. Discovery is a side effect of their primary drive to publish and - hopefully - become well known. And establishments only make things worse, because it's the things that are most likely to produce tangible results (the papers, or economically valuable products) that get the most f…

It's a little more complicated than that. If all you're interested in disovery, you can wander off into rabbit holes or disappear into areas that are only interesting to you. Publishability is useful because it gives you useful external feedback into whether a community thinks what you're doing is "worth discovering". Like any metric, you can find yourself gaming it (intentionally or unintentionally) and it has a slew of other failure modes. Still, you can prize discovery and also care about what other people think.

Re: AI boosts research careers but narrow the span of ideas explored: study

#86

Post by fnordpiglet »

radarsat1 wrote: post #59

"boost research careers".. seems like a pretty drastic conclusion to draw based on a technology that has existed for like 3 years and only lately is any good..

Yeah that was my instinct too. What sort of career defining trends are visible with this much historical data? Feels like someone wrote clickbait research to get published.

Re: AI boosts research careers but narrow the span of ideas explored: study

#87

Post by cliglot »

skeledrew wrote: post #67
analog31 wrote: post #47

Earlier quoted context omitted.

Do you know any scientists? Disclosure: Physicist.

I do, and through conversations have learned that they enjoy what they do and publish patents (they're PhD in industry), but ultimately what they seek is "fame and glory" (literal quote). I was also in academics myself up to the Master's level (research track), and personally had to deal with the politics of getting support for what I wanted to work on; that experience helped to discourage me from going on to a PhD,…

> but ultimately what they seek is "fame and glory" (literal quote).

lol how old are these people? You have better chance at fame and glory if you started a stupid YouTube channel.

Re: AI boosts research careers but narrow the span of ideas explored: study

#88

Post by Ekaros »

skeledrew wrote: post #9

As with other fields touched, AI is merely amplifying what was already there. The aim of many scientists isn't discovery in and of itself. Discovery is a side effect of their primary drive to publish and - hopefully - become well known. And establishments only make things worse, because it's the things that are most likely to produce tangible results (the papers, or economically valuable products) that get the most f…

I wouldn't even be certain about being well known. I would guess there is lot of pressure to stay employed or get the next funding. So optimising for this is the new goal and lot of publications and citations are metrics that help with that.

Re: AI boosts research careers but narrow the span of ideas explored: study

#89

Post by matthewdgreen »

aborsy wrote: post #73

A new breed of academics has appeared whose jobs is to put their names in every paper possible. Literally, their job is to work on frameworks to buy co-authorship. They do this in various ways, like establishing paper pipelines, collecting rents on labs and committees, focusing on money layer, using their profiles and citation count to help with acceptance of papers of other people , etc. You talk to them and they ca…

A lot of this is downstream of compensation schemes that explicitly reward dumb metrics, like raw paper-count without subjective evaluation of contribution or quality. I don't want to generalize, but this seems to be more common in countries that are not the US or Europe.

Re: AI boosts research careers but narrow the span of ideas explored: study

#90

Post by esafak »

pton_xd wrote: post #45
esafak wrote: post #8

Earlier quoted context omitted.

Are you following the news? https://news.ycombinator.com/item?id=48863490 LLMs don't just 'average' their data.

They interpolate data in an XYZ dimensional space. The implications of that is beyond our comprehension. I have a hard time believing that all novel concepts yet to be discovered are contained within that space, though.

You might as well say AI can only think of things humans can, so even if they invent new maths or science they can't go beyond the space of human thought.
Post reply on HN