Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

311–320 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#311
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

wait, did you see the part where the person you are replying to said that writing the code themself was essential to correctly solving the problem? Because they didn't understand the architecture or the domain models otherwise. Perhaps in your case you do have strong hands-on experience with the domain models, which may indeed have shifted you job requirements to supervising those implementing the actual models. I do…

If your product has code on it that can only be understood and worked on by the person that wrote it, then your code is too complex and underdocumented and/or doesn't have enough test coverage.

Your time would be better spent, in a permanent code base, trying to get that LLM to understand something than it would be trying to understand the thing yourself. It might be the case that you need to understand the thing more thoroughly yourself so you can explain it to the LLM, and it might be the case that you need to write some code so that you can understand it and explain it, but eventually the LLM needs to get it based on the code comments and examples and tests.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#312
post #287

Earlier quoted context omitted.

This feels like it conflates problem solving with the production of artifacts. It seems highly possible to me that the explosion of ai generated code is ultimately creating more problems than it is solving and that the friction of manual coding may ultimately prove to be a great virtue.

I measure what I do by output. Just about a week ago I launched a 100% AI generated project that shortcircuits a bunch of manual tasks. What before took 3+ weeks of manual work to produce, now takes us 1-2 days to verify instead. It generates revenue. It solved the problem of taking a workflow that was barely profitable and cutting costs by more than 90%. Half the remaining time is ongoing process optimization - we h…

>This was a problem that wasn't even tractable without AI, and there's no "explosion of AI generated code".

People often say this when giving examples, but what specifically made the problem intractable?

Sometimes before beginning work on a problem, I dramatically overestimate how hard it will be (or underestimate how capable I am of solving it.)

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#313

Earlier quoted context omitted.

Smartphones I think did the most damage. Used to be you had to memorize people's phone numbers. I'm sure other things like memorizing how to get from your house to someone else is also less cognitive when the GPS just tells you every time, instead of you busting out a map, and thinking about your route. I've often found that if I preview a route I'm supposed to take, and use Google Street Maps to physically view key…

About the phone numbers thing: I am now 35yo. Do I still remember the phone number of one of my best friends from primary school back then? Hell yeah, I do! These days though, I am struggling a bit with phone numbers, mostly because I don't even try. If the number is important, I will save it somewhere. Memorizing it? Nahhh... But sometimes my number brain still does that and it seems some weird pattern in the number…

Same. Somehow there tends to be some "pattern" that stands out, but I guess it's just a mix of the likelihood of "something interesting" and our minds being tuned to pick out "anything interesting". I've memorized a few SSNs and license plate numbers this way, and some digits of pi. I like it; it feels like normal memorization with a twist, without having to resort to "hardcore" techniques.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#314
post #160

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now. Where I'm skeptical of this study: - 54 participants, only 18 in the critical 4th session - 4 months is barely enough time…

Soapbox time. They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft. We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them. And it's not even that…

> People will risk their life in a horribly janky self driving car if it means they can swipe on social media instead of watching the road - acceptance doesn't mean it's good.

People will risk their and others' lives in a horribly janky car if it means they can swipe on social media instead of watching the road - acceptance doesn't mean it's good.

FTFY

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#315
post #270
post #183

Earlier quoted context omitted.

> My "actual job" isn't to write code, but to solve problems. Yes, and there's often a benefit to having a human have an understanding of the concrete details of the system when you're trying to solve problems. > That has increasingly shifted to "just" reviewing code It takes longer to read code than to write code if you're trying to get the same level of understanding. You're gaining time by building up an understan…

> It takes longer to read code than to write code if you're trying to get the same level of understanding. You're gaining time by building up an understanding deficit. That works for a while, but at some point you have to go burn the time to understand it. This is true whether an AI wrote the code or a co-worker, except the AI is always on hand to answer detailed questions about the code, do detailed analysis, and ru…

>very rarely productive any more to dig into low level code manually

What data are you basing this assumption on?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#316

Earlier quoted context omitted.

No, it's useful evidence in the same way that contemporaneous fiction is often useful evidence. The first season aired from 1989-1990. The living conditions from the show were plausible. I know because I was alive during that time. My best friend was the son of a vacuum cleaner salesman with a high school education, and they owned a three bedroom house in a nice area, two purebred dogs, and always had new cars. His m…

I read this as "anything can be used as evidence if it confirms my preconceived notions". Your anecdotes about a friend or two are just that - anecdotes. This claim of "a single man could feed a whole family on one factory job" is misleading and untrue. It's usually the 1950s that people claim this was true and they wish we could go back to the 1950s. It's easy to show that that the 1950s were no picnic ( https://arc…

Evidence doesn't mean overwhelming proof. My post was confined to 1989-1990. I didn't make the claim in your post. Homer Simpson also doesn't work in a factory; he's a nuclear engineer at a power plant. I dunno what you're trying to get at here. There are at least as many problems with trying to use statistics as evidence as there are with using anecdotes and fictional references.

I would also trust 100 fictional cartoon characters before I would trust anything said in a pirated article written by Noah Smith about anything. If Noah Smith said that grass was green I would assume that it's blue.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#317
post #167

Earlier quoted context omitted.

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

> My "actual job" isn't to write code, but to solve problems. You're like 836453th person to say this. It's not untrue, but many of us will take writing over reviewing any day. Reviewing is like the worst part of the job.

I strongly prefer writing the code myself and letting AI review it, over the other way around.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#318
Your brain on calculators.

We find that people having to perform mental arithmetics as opposed to people using calculators exhibited more neural activities. They were also able to recall the specific numbers in the equations more.

... So what?

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#319
Excellent scientific quantification that Search Engines and Large Language Models reduce the burden of writing — i.e., they make writing easier.

The consequence of making anything easier is of course that the person and the brain is less engaged in the work, and remembers less.

This debate about using technology for thinking has been ongoing for literally millennia. It is at least as old as Socrates, who criticized writing as harming the ability to think and remember.

>>And now, since you are the father of writing, your affection for it has made you describe its effects as the opposite of what they really are. In fact, it will introduce forgetfulness into the soul of those who learn it: they will not practice using their memory because they will put their trust in writing, which is external and depends on signs that belong to others, instead of trying to remember from the inside, completely on their own. You have not discovered a potion for remembering, but for reminding; you provide your students with the appearance of wisdom, not with its reality. Your invention will enable them to hear many things without being properly taught, and they will imagine that they have come to know much while for the most part they will know nothing. And they will be difficult to get along with, since they will merely appear to be wise instead of really being so.”[0]

To emphasize: 'instead of trying to remember from the inside, completely on their own ... not a potion for remembering, but for reminding ... the appearance of wisdom, not its reality.'

There is no question this is a true dichotomy and trade-off.

The question is where on the spectrum we should put ourselves.

That answer is likely different for each task or goal.

For learning, we should obviously be working at a lower level, but should we go all the way to banning reading and writing and using only oral inquiry and recitation?

OTOH, a peer software engineer manager with many Indians in his group said he was constantly trying to get them to write down more of their plans and documentation, because they all wanted to emulate the great mathematician Ramanujan who did much of his work all in his head, and it was slowing down the SE's work.

When I have an issue with curing a particular polymer for a project, should I just get the answer from the manufacturer or search engine, or take the sufficient chemistry courses and obtain the proprietary formulas necessary to derive all the relevant reactions in my head? If it is just to deliver one project, obviously just get the answer and move on, but if I'm in the business of designing and manufacturing competing polymers, I should definitely go the long route.

As always, it depends.

[0] https://newlearningonline.com/literacies/chapter-1/socrates-...

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#320
post #271
post #269

Earlier quoted context omitted.

I saw that part and I disagreed with the very notion, hence why I wrote what I did. > Because they didn't understand the architecture or the domain models otherwise. My point is that requiring or expecting an in-depth understanding of all the algorithms you rely on is not a productive use of developer time, because outside narrow niches it is not what we're being paid for. It is also not something the vast majority o…

Most of the tests are BS too. And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field. Especially if it’s doing it inline somewhere. The underlying issue with AI slop, is that it’s harder to recognize unless you look closely, and then you realize the whole thing is bullshit.

> Most of the tests are BS too.

Why are you creating BS tests?

> And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field.

Verifying time and space complexity is part of what your tests should cover.

But this is also a funny example - I'm willing to bet the average AI model today can write a far better sort than the vast majority of software developers, and is far more capable of analyzing time and space complexity than the average developer.

In fact, I just did a quick test with Claude, and asked for a simple sort that took into account time and space complexity, and "of course" it knows that it's well established that pure quicksort is suboptimal for a general-purpose sort, and gave me a simple hybrid sort based on insertion sort for small arrays, heapsort fallback to stop pathological recursion, and a decently optimized quicksort - this won't beat e.g. timsort on typical data, but it's a good tradeoff between "simple" (quicksort can be written in 2-20 lines of code or so depending on language and how much performance you're willing to sacrifice for simplicity) and addressing the time/space complexity constraints. It's also close to a variant that incidentally was covered in an article in DDJ ca. 30 years ago because most developers didn't know how to, and were still writing stupidly bad sorts manually instead of relying on an optimized library. Fewer developers knows how to write good sorts today. And that's not bad - it's a result of not needing to think at that level of abstraction most of the time any more.

And this is also a great illustration of the problem: Even great developers often have big blind spots, where AI will draw onresults they aren't even aware of. Truly great developers will be aware of their blind spots and know when to research, but most developers are not great.

Post reply on HN