Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

441–450 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#441

Earlier quoted context omitted.

I have very little knowledge of how transistors shuffle ones and zeros out of registers. That doesn't prevent me from using them to solve a problem. Computing is always abstractions. We moved from plugging to assembly, then to c, then we had languages that managed memory for you -- how on earth can you understand what the compiler should be doing or what it is doing if you don't deal with explicit pointers on a day b…

> Is relying on LLMs fundamentally any different than relying on AWS, or apt, or java yes, because when I call "javac" it won't decide randomly to delete my home directory > Is is different from outsourcing? no, not really, and has about the same level of quality

I think it's a lot like outsourcing. And, expected quality of outsourcing aside, more importantly, I don't see outsourcing as the next step up on the ladder of programming abstraction. It's having someone else do the programming for you (at the same abstraction level).

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#443
post #167
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

[dead]

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#444

Earlier quoted context omitted.

> My "actual job" isn't to write code, but to solve problems Solve enough problems relying on AI writing the code as a black box, and over time your grasp of coding will worsen, and you wont be undestanding what the AI should be doing or what it is doing wrong - not even at the architectural level, except in broad strokes. One ends like the clueless manager type who hasn't touched a computer in 30 years. At which poi…

Yet most organizations in existence pay the people “who hasn’t touched a computer in 30 years” quite a large amount of money to continue to solve problems, for some inscrutable reason… =)

There is a track now for people to stay in engineering and get paid more then their manager

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#445
post #371

Earlier quoted context omitted.

> I don’t have to do boilerplate, generally. I meant this more in the sense of there is nothing new under the sun, and that LLMs have been trained on essentially everything that's available online "under the sun". Sure, there are new SaaS ideas every so often, but the software to produce the idea is rarely that novel (in that you can squint and figure out roughly how it works without thinking too hard), and is in tha…

hahaha, oh boy. that is roughly as useful or accurate as saying that all machines are just combinations of other machines, and hence there is nothing unique about any machine.

Vertical CNC mills and CNC lathes are, obviously, different machines with different use cases. But if you compare within the categories, the designs are almost all conceptually the same.

So, what about outside of some set of categories? Well, generally, no such thing exists: new ideas are extremely rare.

Anyone who truly enjoys entering code character for character, refusing to use refactoring tools (e.g. rename symbol), and/or not using AI assistance should feel free to do so.

I, on the other hand, want to concern myself with the end product, which is a matter of knowing what to build and how to build it. There’s nothing about AI assistance that entails that one isn’t in the driver’s seat wrt algorithm design/choices, database schema design, using SIMD where possible, understanding and implementing protocols (whether HTTP or CMSIS-DAP for debugging microcontrollers over USB JTAG probe), etc, etc.

AI helps me write exactly what I would write without it, but in a fraction of the time. Of course, when the rare novel thing comes up, I either need to coach the LLM, or step in and write that part myself.

But, as a Staff Engineer, this is no different than what I already do with my human peers: I describe what needs doing and how it should be done, delegate that work to N other less senior people, provide coaching when something doesn’t meet my expectations, and I personally solve the problems that no one else has a chance of beginning to solve if they spent the next year or two solely focused on it.

Could I solve any one of those individual, delegated tasks faster if I did it myself? Absolutely. But could I achieve the same progress, in aggregate, as a legion of less experienced developers working in parallel? No.

LLM usage is like having an army of Juniors. If the result is crap, that’s on the user for their poor management and/or lack of good judgement in assessing the results, much like how it is my failing if a project I lead as a Staff Engineer is a flop.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#446
I've recently become interested in using LLMs for things that are actually beyond human comprehension by using kind of insane prompts and then consistently having the model create i.e. "a coherent mathematical model" of the conceptual space we're in at the moment.

I'm very curious to see if we start to see things like this as a new skill, requiring a different cognitive style that's not measured in studies like this.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#447

Earlier quoted context omitted.

I read this as "anything can be used as evidence if it confirms my preconceived notions". Your anecdotes about a friend or two are just that - anecdotes. This claim of "a single man could feed a whole family on one factory job" is misleading and untrue. It's usually the 1950s that people claim this was true and they wish we could go back to the 1950s. It's easy to show that that the 1950s were no picnic ( https://arc…

Evidence doesn't mean overwhelming proof. My post was confined to 1989-1990. I didn't make the claim in your post. Homer Simpson also doesn't work in a factory; he's a nuclear engineer at a power plant. I dunno what you're trying to get at here. There are at least as many problems with trying to use statistics as evidence as there are with using anecdotes and fictional references. I would also trust 100 fictional car…

"There are at least as many problems with trying to use statistics as evidence as there are with using anecdotes and fictional references."

This is a dubious claim.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#448
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

I see the opposite effect with AI: I quickly find some error that it has made, because it always makes errors in my field, and that keeps me from disengaging with the problem, because it helps define what can be wrong. I mainly use AI like I used to use my blog, for writing out my ideas in prose that I think is comprehensible and organized. Neither AI nor my old blog ever solved a problem for me, but they help me fig…

I think that's still in line with what I mean. Letting the AI solve the problem doesn't work, but I've had several times that simply trying to explain the problem to the AI helped me solve it. Sometimes it's not an interactive encyclopedia, but an interactive rubber duck. That works too.

Don't outsource the thinking to the AI, is what I mean. Don't trust it, but use it to talk to, to shape your thoughts, and to provide information and even ideas. But not the solution, because that has never worked for me for any non-trivial problem.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#449
post #167
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

> I get to spend more time on my actual job.

If you spend all your time on that, you might actually lose the ability to actually do it. I find a lot of "non core" tasks are pretty important for skill building and maintenance.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#450

Earlier quoted context omitted.

I'm not actually sure that's true. Theres plenty of controversy now that books that are popular and beloved now are actually not very well written. I mean I've been hearing this complaint since Twilight was popular.

Sure, but a professional developer is (or should be) equivalent of a more demanding reader and critic, not just any customer in a bookshop.

My sentiments exactly
Post reply on HN