Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can get "brain rot"

llm-brain-rot.github.io

Post reply on HN
Print view View original

21–30 of 310 posts

Re: LLMs can get "brain rot"

#21

Post by pluc »

bpt3 wrote: post #18
Barrin92 wrote: post #13

Earlier quoted context omitted.

Yes, I am concerned about the Computer Science profession >"“Brain Rot” for LLMs isn’t just a catchy metaphor—it reframes data curation as cognitive hygiene for AI" A metaphor is exactly what it is because not only do LLMs not possess human cognition, there's certainly no established science of thinking they're literally valid subjects for clinical psychological assessment. How does this stuff get published, this is…

It is a blog post, it was published as a Github page and on arXiv. I think it's intended as a catchy warning to people who are dumping every piece of the internet (and synthetic data based on it!) that there are repercussions.

I think it's an interesting line of thought. So we all adopt LLMs and use it everywhere we can. What happens to the next generation of humans, born with AI and with diminished cognitive capacity to even wonder about anything? What about the next generation? What happens to the next generation of AI models that can't train on original human-created datasets free of AI?

Re: LLMs can get "brain rot"

#22

Post by BoredPositron »

nradov wrote: post #20
BoredPositron wrote: post #16

Earlier quoted context omitted.

I doubt they are highly created you would need experts in every field to do so. Which gives me more performance anxiety for LLMs because one of the most curated fields should be code...

OpenAI has been literally hiring human experts in certain targeted subject areas to write custom proprietary training content.

I bet the dataset is mostly comprised of certain areas™.

Re: LLMs can get "brain rot"

#24

Post by echelon »

philipallstar wrote: post #8
pixelmelt wrote: post #5

Isn't this just garbage in garbage out with an attention grabbing title?

Attention is all you need.

In today's hyper saturated world, attention is everything:

- consumer marketing

- politics

- venture fundraising

When any system has a few power law winners, it makes sense to grab attention.

Look at Trump and Musk and now Altman. They figured it out.

MrBeast...

Attention, even if negative, wedges you into the system and everyone's awareness. Your mousey quiet competitors aren't even seen or acknowledged. The attention grabbers suck all the oxygen out of the room and win.

If you go back and look at any victory, was it really better solutions, or was it the fact that better solutions led to more attention?

"Look here" -> build consensus and ignore naysayers -> keep building -> feedback loop -> win

It might not just be a societal algorithm. It might be one of the universe's fundamental greedy optimization algorithms. It might underpin lots of systems, including how we ourselves as individuals think and learn.

Our pain receptors. Our own intellectual interests and hobbies. Children learning on the playground. Ant colonies. Bee swarms. The world is full of signals, and there are mechanisms which focus us on the right stimuli.

Re: LLMs can get "brain rot"

#25

Post by iwontberude »

pluc wrote: post #21
bpt3 wrote: post #18

Earlier quoted context omitted.

It is a blog post, it was published as a Github page and on arXiv. I think it's intended as a catchy warning to people who are dumping every piece of the internet (and synthetic data based on it!) that there are repercussions.

I think it's an interesting line of thought. So we all adopt LLMs and use it everywhere we can. What happens to the next generation of humans, born with AI and with diminished cognitive capacity to even wonder about anything? What about the next generation? What happens to the next generation of AI models that can't train on original human-created datasets free of AI?

They will accept that their orders come from a terminal and they will follow them.

Re: LLMs can get "brain rot"

#28

Post by icyfox »

pixelmelt wrote: post #5

Isn't this just garbage in garbage out with an attention grabbing title?

Yes - garbage in / garbage out still holds true for most things when it comes to LLM training.

The two bits about this paper that I think are worth calling out specifically:

- A reasonable amount of post-training can't save you when your pretraining comes from a bad pipeline; ie. even if the syntactics of the input pretrained data are legitimate it has learned some bad implicit behavior (thought skipping)

- Trying to classify "bad data" is itself a nontrivial problem. Here the heuristic approach of engagement actually proved more reliable than an LLM classification of the content

Post reply on HN