Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

401–410 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#401
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

I just went through an eerily similar situation where the coding agent was able to muster some pretty advanced math (information geometry) to solve my problem at hand.

But while I was able to understand it enough to steer the conversation, I was utterly unable to make any meaningful change to the code or grasp what it was doing. Unfortunately, unlike in the case you described, chatting with the LLM didn’t cut it as the domain is challenging enough. I’m on a rabbit hunt now for days, picking up the math foundations and writing the code at a slower pace albeit one I can keep up with.

And to be honest it’s incredibly fun. Applied math with a smart, dedicated tutor and the ability to immediately see results and build your intuition is miles ahead of my memories back in formative years.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#402
post #296

Earlier quoted context omitted.

> then perhaps AI is the electric scooter for the mind I have a whole half-written blog post about how LLMs are the cars of the mind. Massive externalities, has to be forced on people, leads to cognitive/health issues instead of improving cognition and health.

Cars didn't have to be forced on people. They were adopted enthusiastically.

were they? [0] I think we have a different idea of "enthusiasm" [1]

[0] https://thereader.mitpress.mit.edu/when-cities-treated-cars-... [1] https://usa.streetsblog.org/2020/03/19/study-most-car-owners...

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#403
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

> My "actual job" isn't to write code, but to solve problems Solve enough problems relying on AI writing the code as a black box, and over time your grasp of coding will worsen, and you wont be undestanding what the AI should be doing or what it is doing wrong - not even at the architectural level, except in broad strokes. One ends like the clueless manager type who hasn't touched a computer in 30 years. At which poi…

I have very little knowledge of how transistors shuffle ones and zeros out of registers. That doesn't prevent me from using them to solve a problem.

Computing is always abstractions. We moved from plugging to assembly, then to c, then we had languages that managed memory for you -- how on earth can you understand what the compiler should be doing or what it is doing if you don't deal with explicit pointers on a day by day basis.

We bring in libraries when we need code. We don't run our own database, we use something else, and we just do "apt-get install mysql", but then we moved onto "docker run" or perhaps we invoke it with "aws cli". Who knows what teraform actually does when we declare we want a resource.

I was thinking the other day how abstractions like AWS or Docker are similar to LLM. With AWS you just click a couple of buttons and you have a data store, you don't know how to build a database from scratch, you don't need one. Of course "to build a database from scratch you must first create the universe".

Some people still hand-craft assembly code to great benefit, but that vast majority don't need to to solve problems, and they can't.

This musing was in the context of what do we do if/when aws data centres are not available. Our staff are generally incapable of working in a non-aws environment. Something that we have deliberately cultivated for years. AWS outputs are one option, or perhaps we should run a non-aws stack that we fully own and control.

Is relying on LLMs fundamentally any different than relying on AWS, or apt, or java. Is is different from outsourcing? You concentrate on your core competency, which is understanding the problem and delivering a solution, not managing memory or running databases. This comes with risk -- all outsourcing does, and if outsourcing to a single supplier you don't and can't understand is acceptable risk, then is relying on LLMs not?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#404
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

> My "actual job" isn't to write code, but to solve problems Solve enough problems relying on AI writing the code as a black box, and over time your grasp of coding will worsen, and you wont be undestanding what the AI should be doing or what it is doing wrong - not even at the architectural level, except in broad strokes. One ends like the clueless manager type who hasn't touched a computer in 30 years. At which poi…

It is not that clear to me.

Let us use an analogy. Many (most?) people can tell a well-written book or story from a mediocre or a terrible one, even though the vast majority of the readers hasn't written any in their lives.

To distinguish good from bad doesn't necessarily require the ability to create.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#405
post #160

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now. Where I'm skeptical of this study: - 54 participants, only 18 in the critical 4th session - 4 months is barely enough time…

Soapbox time. They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft. We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them. And it's not even that…

> They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft.

> We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them.

You're right about the first part, wrong about the second part.

Pre-Gutenberg people could memorize huge texts because they didn't have that many texts to begin with. Obtaining a single copy cost as much as supporting a single well-educated human for weeks or months while they copied the text by hand. That doesn't include the cost of all the vellum and paper which also translated to man-weeks of labor. Rereading the same thing over and over again or listening to the same bard tell the same old story was still more interesting than watching wheat grow or spinning fabric, so that's what they did.

We're offloading our brains onto technology because it has always allowed us to function better than before, despite an increasing amount of knowledge and information.

> Yes, it's too early to be sure, but the internet, Google and Wikipedia arguably haven't made the world any better (overall).

I find that to be a crazy opinion. Relative to thirty years ago, quality of life has risen significantly thanks to all three of those technologies (although I'd have a harder time arguing for Wikipedia versus the internet and Google) in quantifiable ways from the lowliest subsistence farmers now receiving real time weather and market updates to all the developed world people with their noses perpetually stuck in their phones.

You'd need some weapons grade rose tinted glasses and nostalgia to not see that.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#406
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

Then your job has turned into designing solutions, and asking a (sometimes unreliable) LLM to make them for you. If you keep at it, soon you'll accumulate enough cognitive debt to become a fossil, knowing what has to be done, but not quite how it is done.

So no different than management up to the CEO who simply “delegate” to the underlyings?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#407
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

Exactly this. The shift from "writing code" to "reviewing code and focusing on architecture" is the natural evolution. Every abstraction layer in computing history freed us to think at higher levels - assembler to C, C to Python, and now Python to "describe what you want." The people framing this as "cognitive debt" are measuring the wrong thing. You're not losing the ability to think - you're shifting what you think…

The problem is that how do you review code if you don't know what it is supposed to look like? Creativity is not only in the problem solving step but also when implementing it, and letting an LLM do most of it is incredibly dangerous for the future, more so on juniors are gaining experience this way. The software quality will be much worse, and the churn even higher, and I will be in a farm with my chickens

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#408
As a student who has used these tools extensively, I can confirm that AI-assistance in learning does more harm than benefit. The struggle to learn, backtracking from an incorrect assumption and reflection after completing the objective are all short-circuited with agentic tool use. I don't have to say that these tools aren't useful, but I wish they wouldn't sell such an utopian dream of productivity. It's good for some, bad for most.

Earlier, I had to only keep my phone away and not open Instagram while studying. Now, even thinking can be partially offloaded to an automated system.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#409
post #330

Earlier quoted context omitted.

> The notion that you need to keep spending more than a tiny fraction of your time writing code in order to understand enough to be able to solve business problem will seem increasingly quaint. Completely disagree. Reading books doesn't make you an author. Reading books AND writing books makes you an author.

The entire point is we increasingly don't need to be authors. Most of us aren't paid to be authors in your analogy. (Which is good, because outside of your analogy, most authors are paid peanuts, and most of those of us who do write do so because we enjoy it, not as a job) But even if our jobs were to be authors, while I learned some things about writing books from writing the novels I have written and published, I l…

I recommend you to read https://lucumr.pocoo.org/2026/1/18/agent-psychosis/

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#410
post #360

idk, if anything I’m thinking more. The idea that I might be able to build everything I’ve ever planned out. At least the way I’m using them, it’s like the perfect assistive device for my flavor of ADHD — I get an interactive notebook I can talk through crazy stuff with. No panacea for sure, but I’m so much higher functioning it’s surreal. I’m not even using em in the volume many folks claim, more like pair programmi…

Yeah, I'd say I'm thinking and doing way more. One of my favorite things is that I no longer feel like I need to keep up with "framework of the year" I came up over a decade ago, places I worked were heavy on Java and Spring. Frontends were Jquery back then. Since then I've moved around positions quite a bit, many different frameworks, but typically service side rendered MVC types and these days I work as an SRE. The…

I've had the same type of experience where I feel like the knowledge barrier for a lot of projects has been made much smaller than it used to be :D

btw, I have a couple of questions just out of curiosity: What tools do you use besides Claude? Do you have a local or preferred setup? and do you know of any communities where discussion about LLM/general AI tool use is the focus, amongst programmers/ML engineers? Been trying to be more informed as to what tools are out there and more up to date on this field that is progressing very quickly.

Post reply on HN