Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

31–40 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#31
post #8
post #5

This is called cognitive offloading. Anyone who’s spent enough time working with coding assistants will recognize it.

Or working as an engineering manager. It's the inevitable consequence of working at a different level of abstraction. It's not the end of the world. My assembly is rusty too...

I don't think not using assembly is going to affect my brain / my life quality in any significant way, but not speaking / chatting with someone is.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#32
post #4

I can’t believe riding a horse and carriage wouldn’t make you better at riding a horse. Sure a horserider wouldn’t want to practice the wrong way, but anyone else just wants to get somewhere

> I can’t believe riding a horse and carriage wouldn’t make you better at riding a horse. Surely you mean "would"? Because riding a horse and carriage doesn't imply any ability at riding a horse, but the reverse relation would actually make sense, as you already have historical, experiential, intimate knowledge of a horse despite no contemporaneous, immediate physical contact. Similarly, already knowing what you want…

The first sentence in the comment you are responding to is sarcasm. Just replace "I can't believe" with "Of course".

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#34

AI is the anti-Zettelkasten. Rather than getting ever deeper insight into a subject matter by actively working on it, you iterate fast but shallow over a corpus of AI generated content. Example: I wanted to understand the situation in the Middle East better so I wrote an 10 page essay on the genesis if Hammas and Hizbulah using OpenAI as a cowriter. I remember nothing, worse of the things I remember I don’t know if i…

I'm on the optimistic side with how useful LLMs are, but I have to agree. You cultivate the instinct for how to steer the models and reduce hallucinations, but you're not building articulable knowledge or engaging in challenging thinking. It's more learning muscle-memory reactions to certain forms of LLM output that lean you towards trusting the output more, trying another prompting strategy, clearing context or not, and so on.

To the extent we can call it skill, it's probably going to be made redundant in a few years as the models get better. It gives me a kind of listlessness that assembly line workers would feel.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#35

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need.

Why do I still know how to optimize free conventional memory in DOS by configuring config.sys and autoexec.bat?

I haven’t done this in 2 decades and I’m reasonably sure I never again will

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#36
post #30

The results are not surprising to me personally. When I have used AI to help with my own writing and translation tasks, I do not feel as mentally engaged with the writing or translation process as I would be if I were doing it all on my own. But I have found that using AI in other ways to be incredibly mentally engaging in its own way. For the past two weeks, I’ve been experimenting with Claude Code to see how well i…

I use AI tools for amusement and asking random questions, but for actual work, I basically don't use them at all. I wonder if I'll be part of the increasingly rare group who is actually able to do anything while the rest become progressively more incompetent.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#37
post #19
post #3

Now, let's do same exercise but with programming and over longer period of time. Would really like to present it to management that pushes ai assistance for coding

> Would really like to present it to management that pushes ai assistance for coding Your management presumably cares more about results, than your long term cognitive decline?

i guess one of the questions is how quick cognitive decline sets it and how it influences system stability (we have big system with very high sla due to nature of system and it takes some serious cognitive abilities to reason about it operation).

if todays productivity is traded for longer term stability, i am not sure that it's a risk they would like to take

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#38
post #35

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need. Why do I still know how to optimize free conventional memory in DOS by configuring config.sys and autoexec.bat? I haven’t done this in 2 decades and I’m reasonably sure I never again will

Because these are core memories that provide stepping stones to later knowledge. It is a part of the story of you. It is very hard to integrate all knowledge in this way.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#39
post #24
post #14

Earlier quoted context omitted.

> The brain does not retain information that it does not need. Sounds very plausible, though how does that square with the common experience that certain skills, famously 'riding a bike', never go away once learned?

Such a good question - I hope someone answers with more than an anecdote (which is all I can provide) - I've found the skills that don't leave you like riding a bike, swimming, cooking are all physical skills. Tangible. The skills that leave: arguments, analysis, language, creativity often seem abstract and primarily if not exclusively sourced in our minds

Google "procedural memory". Procedural memory is more resistant to forgetting than other types of memory.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#40
post #35

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need. Why do I still know how to optimize free conventional memory in DOS by configuring config.sys and autoexec.bat? I haven’t done this in 2 decades and I’m reasonably sure I never again will

Probably because you learned it during that brief period in your development in which humans are most impressionable.

Now think about the effect on those humans currently using LLMs at that stage of their development.

Post reply on HN