Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

131–140 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#131
post #100
post #54

Earlier quoted context omitted.

Companies don't own employees: workers can leave at any time. Thus protecting employees productivity in the long run doesn't necessarily help the company. (Unless you explicitly set up contracts that help there, or there are strong social norms in your place around this.)

i am not talking about productivity. i am talking about quality and knowledge

Your workers can still quit, and take their knowledge with them.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#132
post #107
post #60

Earlier quoted context omitted.

To quote myself: > Companies don't own employees: workers can leave at any time. > Thus protecting employees productivity in the long run doesn't necessarily help the company. (Unless you explicitly set up contracts that help there, or there are strong social norms in your place around this.)

You are talking about productivity, I'm talking about knowledge. You may come-up with a product, then fire all engineers having built it. Then, what? It's not sustainable for a business to start from scratch every other year. Your LLM won't be a substitute for owning your product.

Your workers can still quit, and take their knowledge with them.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#133
post #79
post #18

Earlier quoted context omitted.

> What is ranked as “top” is ultimately influenced by the priorities of the LLM's shareholders [123, 125]. As if that's anything new. There's the adage that's older than electronics, that freedom of the press is freedom for those who can afford to own a printing press. > However, this convenience came at a cognitive cost, diminishing users' inclination to critically evaluate the LLM's output or ”opinions” (probabilis…

> Reminds me of Plato's concern about reading and writing dulling your mind. (I think he had his sock puppet Socrates express the concern. But I could be wrong.) Nope. Read the dialogue (Phaedrus). It's about rhetoric and writing down political discourses. Writing had existed for millennia. And the bit about writing being detrimental is from a mythical Egyptian king talking to a god, just a throwaway story used in th…

> In fact the conclusion of that bit of the dialogue is that merely having access to text may give an illusion of understanding. Quite relevant and on point I'd say.

Well, so that's exactly my point: Plato was an old man who yelled at clouds before it was cool.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#134
post #14

Earlier quoted context omitted.

> The brain does not retain information that it does not need. Sounds very plausible, though how does that square with the common experience that certain skills, famously 'riding a bike', never go away once learned?

> Sounds very plausible, though how does that square with the common experience that certain skills, famously 'riding a bike', never go away once learned? I worked with some researchers who specifically examined this when developing training content for soldiers. They found that 'muscle memory' skills such as riding a bike could persist for a very long time. At the other end of the spectrum were tasks that involved p…

Very interesting! Thanks for bringing this up.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#135
I am just finishing a book that took about two years to write. I thought I would be done a year ago. It’s been a slog.

So now I am in the final editing stage, and I am going back over old writing that I don’t remember doing. The material has come together over many many drafts, and parts of it are still not quite consistent with other parts.

But when I am done, it will be mine. And any mistakes will be honest ones that represent the real me. That’s a feeling no one who uses AI assistance will ever have.

I have never and will never use AI to write anything for me.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#136
post #35

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

> The brain does not retain information that it does not need. Why do I still know how to optimize free conventional memory in DOS by configuring config.sys and autoexec.bat? I haven’t done this in 2 decades and I’m reasonably sure I never again will

I also think the claim that "the brain does not retain information it does not need" is an insufficient explanation, and short-sighted. As an example, reading books informs and shapes our thinking, and while people may not immediately recall a book that they read some time ago, I've had conversations where I remembered that I had read a particular passage (sentence, phrase, idea) and referred to it in the conversation.

People do stuff like that all the time, bringing up past memories in spontaneity. The brain absolutely does remember things it "doesn't need".

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#137
post #129

I don't quite see their point. Obviously if you're delegating the task to someone/something then you're not getting as good at it as if you were to do it yourself. If I were to write machine code by hand, rather than having the compiler do it for me, I would definitely be better at it and have more neural circuitry devoted to it. As I see it, it's much more interesting to ask not wherther we are still good at doing t…

Your question is answered by the study abstract. > Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning.

But it's not that they "underperformed" at life in general - they underperformed when assessed on various aspects of the task that they weren't practicing. To me it's as if they ran a trial where one group played basketball, while another were acting as referees - of course that when tested on ball control, those who were dribbling and throwing would do better, but it tells us nothing about how those acting as referees performed at their thing.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#138
post #63

Earlier quoted context omitted.

Plato's sock puppet Socrates? I think that you and I have read different history books, or at least different books regarding the history of philosophy. That said, I would love to hear your perspective on this.

I presume they refer to the fact that Socrates is basically used as a rhetorical device in Plato’s writings, and it’s not entirely clear how much of the dialogues were Socrates’s thoughts and how much was Plato’s own.

Yes, exactly.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#139

When I write with AI, it feels smooth in the moment, but I’m not really thinking through the ideas. The writing sounds fine, but when I look back later, I often can’t remember why I phrased things that way. Now I try to write my own draft first, then use AI to help polish it. It takes a bit more effort upfront, but I feel like I learn more and remember things better.

The rule of thumb "LLMs are good at reducing text, not expanding it" is a good one here.

> "LLMs are good at reducing text, not expanding it"

You put it in quote marks, but the only search results are from you writing it here on HN. Obviously LLMs are extremely good at expanding text, which is essentially what they do whenever they continue a prompt. Or did you mean that in a prescriptive way - that it would be better for us to use it more for summarizing rather than expanding?

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#140
post #3

Now, let's do same exercise but with programming and over longer period of time. Would really like to present it to management that pushes ai assistance for coding

If by "cognitive debt", you mean "you don't really understand the code of the application that we're trying to extend/maintain", then yes, it's almost certainly going to apply to programming.

If I write the application, I have an internal map that corresponds (more or less) to what's going on in the code. I built that map as I was writing it, and I use that map as I debug, maintain, and extend the application.

But if I use AI, I have much less clear of a map. I become dependent on AI to help me understand the code well enough to debug it. Given AI's current limitations of actually understanding, that should give you pause...

Post reply on HN