Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

Critical thinking, forming ideas, writing, etc, those are too stuff that can atrophy if not used.

For example, a lot of people can't locate themselves without a GPS today.

To be frank I see it really similar to our muscles: don't want to lose it? Use it. Whether that is learning a language, playing an instrument or the task llms perform.

Further elaborating from my experience.

1. I think we're in the early stages, where agents are useful because we still know enough to coach well - knowledge inertia.

2. I routinely make the mistake of allowing too much autonomy, and will have to spend time cleaning up poor design choices that were either inserted by the agent, or were forced upon it because I had lost lock on the implementation details (usually both in a causal loop!)

I just have a policy of moving slowly and carefully now through the critical code, vs letting the agent steer. They have overindexed on passing tests and "clean code", producing things that cause subtle errors time and time again in a large codebase.

> burn the time to understand it.

It seems to me to be self-evident that writing produces better understanding than reading. In fact, when I would try to understand a difficult codebase, it often meant that probing+rewriting produced a better understanding than reading, even if those changes were never kept.

Because they didn't understand the architecture or the domain models otherwise.

Perhaps in your case you do have strong hands-on experience with the domain models, which may indeed have shifted you job requirements to supervising those implementing the actual models.

I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job?

If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to?

Funny story-- I asked a LLM to review a call transcript to see if the caller was an existing customer. The LLM said True. It was only when I looked closer that I saw that the LLM mean "True-- the caller is an existing customer of one of our competitors". Not at all what I meant.

> To summarize, the delta-band differences suggest that unassisted writing engages more widespread, slow integrative brain processes, whereas assisted writing involves a more narrow or externally anchored engagement, requiring less delta-mediated integration.

There is no intellectual judgement regarding this difference, though the authors do supply citations from related work that they claim may be of interest to those wanting "to know if the offloading of cognitive tasks changes my own brain and my own cognition". If your brain changes, it might change for the worse at least as far as you experience it. Is this ableism, to examine your own cognitive well-being and make your own assessment? If you don't like how you're thinking about something, are you casting aspersion on yourself and shaming your own judgement? Ableist discourse is, unsurprisingly, a stupid language game for cognitively impaired dummies. It's a pathetic attempt to redefine basic notions of capability and impairment, of functioning and dysfunction as inherently evil concepts, and then to work backward from that premise to find fault with the research results. Every single person experiences moments or lifetime's of psychological and mental difficulty. Admitting this and adapting to it or remediating harmful effects has nothing to do with calling stupid people stupid or ableism. It's just a means of providing tools and frameworks for "cognitive wellness", but even just the implication of "wellness" being distinct from "illness" makes the disturbed and confused unwell.

If you outsource a skill consistently, you will be engaging less with that skill. Depending on the skill, this may be acceptable, or a desirable tradeoff.

For example, using a very fast LLM to interactively make small edits to a program (a few lines at a time), outsources the work of typing, remembering stdlib names and parameter order, etc.

This way of working is more akin to power armor, where you are still continuously directing it, just with each of your intentions manifesting more rapidly (and perhaps with less precision, though it seems perfectly manageable if you keep the edit size small enough).

Whereas "just go build me this thing" and then you make a coffee is qualitatively very different, at that point you're more like a manager than a programmer.