Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

Post reply on HN
Print view View original

361–370 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#361

Post by alfalfasprout »

vidarh wrote: post #284
taco_emoji wrote: post #281

Earlier quoted context omitted.

I don't care what my "actual" job is. I like writing code. I like exercising my brain in that way. I like building things. I do not want to be a supervisor of AI agents. I do not want to engineer prompts, I want to engineer software.

I sympathise, in as much as I love writing code too, but I increasingly restrict that to my personal projects. It is simply not cost effective any more to write code manually vs. proper use of agents, and developers who resist that will find it increasingly hard to stay employed.

> It is simply not cost effective any more to write code manually vs. proper use of agents, and developers who resist that will find it increasingly hard to stay employed.

In practice, this isn't bearing out at all though both among my peers and with peers in other tech companies. Just making a blanket statement like this adds nothing to the conversation.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#362

Post by Capricorn2481 »

iambateman wrote: post #333
wisty wrote: post #160

Earlier quoted context omitted.

Soapbox time. They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft. We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them. And it's not even that…

You’re currently using the internet.

That doesn't contradict anything they wrote.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#363

Post by asdff »

ASalazarMX wrote: post #357
vidarh wrote: post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

Then your job has turned into designing solutions, and asking a (sometimes unreliable) LLM to make them for you. If you keep at it, soon you'll accumulate enough cognitive debt to become a fossil, knowing what has to be done, but not quite how it is done.

And really where is your moat? Why pay for a senior when a junior can prompt an LLM all the same? People are acting like its juniors who are going to be out of work like companies are going to just keep paying seniors for their now obsolete skills.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#364

Post by Capricorn2481 »

discreteevent wrote: post #154
jack_pp wrote: post #135

Earlier quoted context omitted.

Instead of being the architect, engineer, plumber, electrician, carpenter you can (most of the time) just be the architect/planner. You for sure need to know how everything works in case LLMs mess the low level stuff up but it sure is nice not needing to lay bricks and dig ditches anymore and just build houses.

It won't turn most people into architects. It will turn them into PMs. The function of PMs is important but without engineers you are not going to build a sustainable system. And an LLM is not an engineer.

> It won't turn most people into architects. It will turn them into PMs

That sounds awful. Every PM I've ever met, I did their job for them. They did nothing. And I've met some heavy hitter PMs with a lot of stripes and recommendations.

The job of being a PM is over-exaggerated. It boils down to writing things down and bringing them up later. Something I ended up doing for them, because they didn't know enough to know what to write down. Their skills are interviewing well and drinking beers with important people.

So what you said is a dreadful future, if true.

And side note, my last PM didn't even take notes, he had AI do it for him. They were always wrong. I had to correct them constantly.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#365

Post by alfalfasprout »

sothatsit wrote: post #114
isolli wrote: post #102

Earlier quoted context omitted.

This mirrors my experience exactly. We have to learn how to tame the beast.

I think we all just need to avoid the trap of using AI to circumvent understanding. I think that’s where most problems with AI lie. If I understand a problem and AI is just helping me write or refactor code, that’s all good. If I don’t understand a problem and I’m using AI to help me investigate the codebase or help me debug, that’s okay too. But if I ever just let the AI do its thing without understanding what it’s…

I'm writing a blog post on this very thing actually.

Outsourcing learning and thinking is a double edged sword that only comes back to bite you later. It's tempting: you might already know a codebase well and you set agents loose on it. You know enough to evaluate the output well. This is the experience that has impressed a few vocal OSS authors like antirez for example.

Similarly, you see success stories with folks making something greenfield. Since you've delegated decision making to the LLM and gotten a decent looking result it seems like you never needed to know the details at all.

The trap is that your knowledge of why you've built what you've built the way it is atrophies very quickly. Then suddenly you become fully dependent on AI to make any further headway. And you're piling slop on top of slop.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#366

Post by alfalfasprout »

WarmWash wrote: post #261
albumen wrote: post #180

Earlier quoted context omitted.

They view the framing of the MIT paper not just as bad science, but as a dangerous social tool that uses brain data to "consign people" to being less worthy or "stupid" for using cognitive aids. It flags the paper's alarmist findings as "pseudoscience" designed to provoke fear rather than provide rigorous insight. They highlight several "red flags" in the study's design: lack of a coherent scientific framework, metho…

Similar to the media, I've picked up on vibes from academia that have a baseline AI negative tilt. In my own (classic) engineering work, AI has become so phenomenally powerful that I can only imagine that if I was still in college, I'd be mostly checked out during those boring lectures/bad teacher classes, and then learning on my own with the textbook and LLMs by night. Which begs the question, what do we need the pr…

> Similar to the media, I've picked up on vibes from academia that have a baseline AI negative tilt.

The media is extremely pro-AI (and a quick look at their ownership structure gives you a hint as to why). You seem to be projecting your own biases here, no?

And how would those LLMs learn? How would you learn to ask the right questions that further scientific research?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#367

Post by asdff »

jampekka wrote: post #221
culi wrote: post #108

My friend works with people in their 20s. She recently brought up her struggles to do the math in her head for when to clock in/out for their lunches (30 minutes after an arbitrary time). The young coworker's response was "Oh I just put it into ChatGPT" The kids are using ChatGPT for simple maths...

I've been using a Python prompt or the browser URL bar for simple maths for over a decade. I don't see much added value in doing arithmetic manually, humans really suck at it.

You can’t add 30 minutes in your head?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#368

Post by BurningFrog »

wisty wrote: post #160
sdoering wrote: post #111

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now. Where I'm skeptical of this study: - 54 participants, only 18 in the critical 4th session - 4 months is barely enough time…

Soapbox time. They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft. We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them. And it's not even that…

> They were arguably right

I think they were right that something was lost in each transition.

But something much bigger was also gained, and I think each of those inventions were easily worth the cost.

But I'm also aware that one cost of the printing press was a century of very bloody wars across Europe.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#369

Post by alexjplant »

mcv wrote: post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

When I used Copilot autocomplete more I noticed myself slipping a bit when it comes to framework and syntax particulars so I instituted a moratorium on it on Fridays to prevent this.

Claude Code seems to be a much better paradigm. For novel implementations I write code manually while asking it questions. For things that I'm prototyping I babysit it closely and constantly catch it doing things that I don't want it to do. I ask it questions about why it built things certain ways and 80% of the time it doesn't have a good answer and redoes it the way that I want. This takes a great deal of cognitive engagement.

Rule nombre [sic] uno: Never anthropomorphize the LLM. It's a giant pattern-matching machine. A useful one, but still just a machine. Do not let it think for you because it can't.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#370

Post by Capricorn2481 »

mikemarsh wrote: post #299
DocTomoe wrote: post #48

Earlier quoted context omitted.

Pathologising those who disagree with a current viewpoint follows a long and proud tradition. "Possessed by demons" of yesteryear, today it's "AI psychosis".

It's not about "Pathologising those who disagree", it's about how demonic influence (which anyone can fall victim to) _actually_ works. You've highlighted a very real equivalency in spite of yourself.

> demonic influence

Is this an unironic usage of this word? If you're trying to make a different point, it doesn't come across.

> You've highlighted a very real equivalency in spite of yourself

The equivalence doesn't help you, because "possessed by demons" has been used to describe people who are sick, playing D&D, reading comics, listening to music, being women, and it is frivolous and embarrassing to take seriously.

Post reply on HN