Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

Post reply on HN
Print view View original

111–120 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#111

Post by sdoering »

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now.

Where I'm skeptical of this study:

- 54 participants, only 18 in the critical 4th session

- 4 months is barely enough time to adapt to a fundamentally new tool

- "Reduced brain connectivity" is framed as bad - but couldn't efficient resource allocation also be a feature, not a bug?

- Essay writing is one specific task; extrapolating to "cognition in general" seems like a stretch

Where the study might have a point:

Previous tools outsourced partial processes - calculators do arithmetic, Google stores facts. LLMs can potentially take over the entire cognitive process from thinking to formulating. That's qualitatively different.

So am I ideologically inclined to dismiss this? Maybe. But I also think the honest answer is: we don't know yet. The historical pattern suggests cognitive abilities shift rather than disappear. Whether this shift is net positive or negative - ask me again in 20 years.

[Edit]: Formatting

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#112

Post by Archer6621 »

mcv wrote: post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

That's a nice anecdote, and I agree with the sentiment - skill development comes from practice. It's tempting to see using AI as free lunch, but it comes with a cost in the form of skill atrophy. I reckon this is even the case when using it as an interactive encyclopedia, where you may lose some skill in searching and aggregating information, but for many people the overall trade off in terms of time and energy savings is worth it; giving them room to do more or other things.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#113

Post by cimi_ »

sdoering wrote: post #111

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now. Where I'm skeptical of this study: - 54 participants, only 18 in the critical 4th session - 4 months is barely enough time…

> The historical pattern suggests cognitive abilities shift rather than disappear.

Shift to what? This? https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d...

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#114

Post by sothatsit »

isolli wrote: post #102
mcv wrote: post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

This mirrors my experience exactly. We have to learn how to tame the beast.

I think we all just need to avoid the trap of using AI to circumvent understanding. I think that’s where most problems with AI lie.

If I understand a problem and AI is just helping me write or refactor code, that’s all good. If I don’t understand a problem and I’m using AI to help me investigate the codebase or help me debug, that’s okay too. But if I ever just let the AI do its thing without understanding what it’s doing and then I just accept the results, that’s where things go wrong.

But if we’re serious about avoiding the trap of AI letting us write working code we don’t understand, then AI can be very useful. Unfortunately the trap is very alluring.

A lot of vibe coding falls into the trap. You can get away with it for small stuff, but not for serious work.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#115

Post by newswasboring »

"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise."

- Socrates on Writing.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#117

Post by curl-up »

Prompt they use in `Figure 28.` is a complete mess, all the way from starting it with "Your are an expert" to the highly overlapping categories to the poorly specified JSON without clear direction on how to fill in those fields.

Similar mess with can be found in `Figure 34.`, with an added bonus of "DO NOT MAKE MISTAKES!" and "If you make a mistake you'll be fined $100".

Also, why are all of these research papers always using such weak LLMs to do anything? All of this makes their results very questionable, even if they mostly agree with "common intuition".

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#118

Post by nly »

netsharc wrote: post #18
captain_coffee wrote: post #6

Curious what the long-term effects from the current LLM-based "AI" systems embedded in virtually everything and pushed aggressively will be in let's say 10 years, any strong opinions or predictions on this topic?

Hopefully the brainrot will mean older developers, who know how to code the old-fashioned way, don't get replaced so quickly..

Or they'll be fired for not working fast enough, which already happens

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#119

Post by Madmallard »

mannanj wrote: post #82
greggoB wrote: post #36

Earlier quoted context omitted.

> Similar to the mass psychosis we were hearing about during COVID Can you be more specific and/or provide some references? The "demonstrating curiosity about controversial topics" part is sounding like vaccine skepticism, though I don't recall ever hearing that being referred to as any kind of "psychosis".

Noting that it is straw man to connect my argument with vaccine skepticism. The mass psychosis was that early on in the COVID response, we were hearing so much early advice from people that were ahead of CDC/FDA, things like: - Masks work (CDC/FDA discouraged, then flip-flopped and took credit for these things) despite it originating from Scott Alexander and skeptic communities like his, I also heard it from Tim Ferr…

Thought I recall Vinay Prasad saying (back in 2020 or 2021) that masks don't really work well enough for us to force all kids to wear them. Like chance passing encounters they have some effectiveness, but an imperfectly used non-n95 mask is basically worthless. But the latter scenario is what nearly everyone was doing.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#120

Post by probably_wrong »

blackqueeriroh wrote: post #79

I encourage folks to listen to brilliant psychologist for software teams Cat Hicks [1] and her wife, teaching neuroscientist Ashley Juavinett [2] on their excellent podcast, Change, Technically discussing the myriad problems with this study: https://www.buzzsprout.com/2396236/episodes/17378968 1: https://www.catharsisinsight.com 2: https://ashleyjuavinett.com

I'm not a fan of "TL;DR" but I think 52 minutes would qualify. I jumped to a random point of the transcript and found just platitudes, which didn't quite hook me into listening to all of it.

How about some more info on what their main conclusions are?

Post reply on HN