Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

21–30 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#21

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

[deleted]

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#22
post #6
post #3

Now, let's do same exercise but with programming and over longer period of time. Would really like to present it to management that pushes ai assistance for coding

Why not try it for social media? There’s got to be the world’s largest class action lawsuit if we can get some science behind what that industry has done.

> There’s got to be the world’s largest class action lawsuit

You'd have to articulate harm, so this is basically dead in the water (in the US). Good luck.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#23
AI is the anti-Zettelkasten.

Rather than getting ever deeper insight into a subject matter by actively working on it, you iterate fast but shallow over a corpus of AI generated content.

Example: I wanted to understand the situation in the Middle East better so I wrote an 10 page essay on the genesis if Hammas and Hizbulah using OpenAI as a cowriter.

I remember nothing, worse of the things I remember I don’t know if it was hallucinations I fixed or actual facts.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#24
post #14

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need. Sounds very plausible, though how does that square with the common experience that certain skills, famously 'riding a bike', never go away once learned?

Such a good question - I hope someone answers with more than an anecdote (which is all I can provide) - I've found the skills that don't leave you like riding a bike, swimming, cooking are all physical skills. Tangible.

The skills that leave: arguments, analysis, language, creativity often seem abstract and primarily if not exclusively sourced in our minds

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#25

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> You can't just skim a math textbook and know all the math. You have to stop and think.

And most importantly you have to write. A lot. Writing allows our brain to structure our thinking. Enables us to have a structured dialogue with ourselves. Explore different paths. Thinking & pondering can only do so much and will reach the limits soon. Writing, on the other hand enables one to explore thoughts nearly endlessly.

Given that thinking is so intimately associated with writing (could be prose, drawing, equations, graphs/charts, whatever) and that LLMs are doing more and more of writing it'll be interesting to see the effect of LLMs on our cognitive skills.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#26
The results are obviously predictable, but it's nice that the authors took the time to prove a thing everyone already knows to be true with the rigors of science.

I wonder how the participants felt writing an essay while being hooked up to an EEG.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#27
post #14

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need. Sounds very plausible, though how does that square with the common experience that certain skills, famously 'riding a bike', never go away once learned?

Closer to the truth is that the brain never completely forgets something, in the sense that there are always vestiges left over, even after the ability to recall or instantly draw upon it is long gone. Studies show, for example, that after one has "forgotten" a language, they're quicker to pick up it again later on compared to someone without that prior experience; how quickly being time dependent, but more quickly nonetheless.

OTOH, IME the quickest way to truly forget something is to overwrite it. Photographs being a notorious example, where looking at photographs can overwrite your own personal episodic memory of an event. I don't know how much research exists exploring this phenomenon, though, but AFAIU there are studies at least showing that the mere act of recalling can reshape memories. So, ironically, perhaps the best way not to forget is to not remember.

Left unstated in the above is that we can categorize different types of memory--episodic, semantic, implicit, etc--based on how they seem to operate. Generalizations (like the above ;) can be misleading.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#28
post #3

Now, let's do same exercise but with programming and over longer period of time. Would really like to present it to management that pushes ai assistance for coding

This opinion is the exact thinking that has lead to the massive layoffs in the design industry. Their jobs are being destroyed because they think lawsuits and current state of the art will show they are right. These models actually can't produce unique input and if you use them for ideation they do only help you get to already solved problems.

But engineers aren't being fired completely in droves because we have adapted. The human can still break down the problem, tell the LLM to come up with multiple different ways of solving the problem, throw away all of them and asking for more. My most effective use is usually looking and seeing what I would do normally, breaking it down, and then asking for it in chunks that make sense that would touch multiple places, then coding details. It's just a shift in thinking like knowing when to copy and paste when being DRY.

Designers are screwing themselves right now waiting for case law instead of using their talents to make one unique thing not in the training set to boost their productivity and shaming tools that let them do that.

It will be a competitive advantage in the future to short sighted companies that took humans out the loop completely, but any company not using the tech will be horse shoe makers not worried because of all the mechanical issues with horseless carriages

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#29

AI is the anti-Zettelkasten. Rather than getting ever deeper insight into a subject matter by actively working on it, you iterate fast but shallow over a corpus of AI generated content. Example: I wanted to understand the situation in the Middle East better so I wrote an 10 page essay on the genesis if Hammas and Hizbulah using OpenAI as a cowriter. I remember nothing, worse of the things I remember I don’t know if i…

Most intelligent people are aware of the fact that writing is about thinking as much as it is about getting the written text.

LLMs can be great sparring partners for this, if you don't use it as a tool that writes for you, but as a tool that finds mistakes, points out gaps and errors (which you may or may not ignore) and helps in researching general questions aboit the world around you (always woth caution and sources).

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#30
The results are not surprising to me personally. When I have used AI to help with my own writing and translation tasks, I do not feel as mentally engaged with the writing or translation process as I would be if I were doing it all on my own.

But I have found that using AI in other ways to be incredibly mentally engaging in its own way. For the past two weeks, I’ve been experimenting with Claude Code to see how well it can fully automate the brainstorming, researching, and writing of essays and research papers. I have been as deeply engaged with the process as I have ever been with writing or translating by myself. But the engagement is of a different form.

The results of my experiments, by the way, are pretty good so far. That is, the output essays and papers are often interesting for me to read even though I know an AI agent wrote them. And, no, I do not plan to publish them or share them.

Post reply on HN