Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

Post reply on HN
Print view View original

151–160 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#151

Post by Frummy »

OhNotAPaper wrote: post #10
Frummy wrote: post #4

I can’t believe riding a horse and carriage wouldn’t make you better at riding a horse. Sure a horserider wouldn’t want to practice the wrong way, but anyone else just wants to get somewhere

> I can’t believe riding a horse and carriage wouldn’t make you better at riding a horse. Surely you mean "would"? Because riding a horse and carriage doesn't imply any ability at riding a horse, but the reverse relation would actually make sense, as you already have historical, experiential, intimate knowledge of a horse despite no contemporaneous, immediate physical contact. Similarly, already knowing what you want…

From the thread: yes, it's sarcasm. Here's some clarification as well: https://news.ycombinator.com/item?id=44291314

Yes, I'm acknowledging a lack of skill transfer, but that there are new ways of working and so I sarcastically imply the article can't see the forest for the trees, missing the big picture. A horse and carriage is very useful for lots of things. A horse is more specialised. I'm getting at the analogy of a technological generalisation and expansion, while logistics is not part of my argument. If you want to write a very good essay and if you're good at that then do it manually. If you want to create scalable workflows and have 5 layers of agents interacting with each other collaboratively and adversarially scouring the internet and newssites and forums to then send investment suggestions to your mail every lunch then that's a scale that's not possible with a pen and paper and so prompting has an expanded cause and effect cone

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#152

Post by tguvot »

eru wrote: post #131
tguvot wrote: post #100

Earlier quoted context omitted.

i am not talking about productivity. i am talking about quality and knowledge

Your workers can still quit, and take their knowledge with them.

not if because of ai they have no knowledge

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#153

Post by tguvot »

AnimalMuppet wrote: post #140
tguvot wrote: post #3

Now, let's do same exercise but with programming and over longer period of time. Would really like to present it to management that pushes ai assistance for coding

If by "cognitive debt", you mean "you don't really understand the code of the application that we're trying to extend/maintain", then yes, it's almost certainly going to apply to programming. If I write the application, I have an internal map that corresponds (more or less) to what's going on in the code. I built that map as I was writing it, and I use that map as I debug, maintain, and extend the application. But if…

i think that more far reaching consequences it's that "accumulation of cognitive debt" essentially leads to diminished cognitive capabilities, as you loose ability to understand things, analyze and reason.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#154

Post by niemandhier »

bsenftner wrote: post #148
atoav wrote: post #29

Earlier quoted context omitted.

Most intelligent people are aware of the fact that writing is about thinking as much as it is about getting the written text. LLMs can be great sparring partners for this, if you don't use it as a tool that writes for you, but as a tool that finds mistakes, points out gaps and errors (which you may or may not ignore) and helps in researching general questions aboit the world around you (always woth caution and source…

Exactly! Never ever ever have AI write for you. Ask it to critique what you wrote, ask it to pick your arguments apart. Then use your mind to fix what it pointed out. If you cannot figure out how, ask the AI to explain how. Then take a break, 20 minutes is fine, and then return and fix the issue yourself using your own mind to write without assistance. This is how one uses AI to learn.

The problem with this strategy is that unless you commit logical fallacies you cannot trust the AI critic. Why? It might cite non existing diverging opinions, misuse sources or introduce subtle changes in a citation.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#155

Post by tguvot »

alex77456 wrote: post #144
NetRunnerSu wrote: post #126

The discussion here about "cognitive debt" is spot on, but I fear it might be too conservative. We're not just talking about forgetting a skill like a language or losing spatial memory from using GPS. We're talking about the systematic, irreversible atrophy of the neural pathways responsible for integrated reasoning. The core danger isn't the "debt" itself, which implies it can be repaid through practice. The real da…

It's up to everyone to decide what to use LLMs for. For high friction / low throughput (eg, online research using inferieor search tools) tasks, i find text models to be great. To ask about what you don't know, to skip the 'tedious part' (I don't feel like looking for answers, especially troubleshooting arcane technical issues among pages of forums or social media, makes me smarter in any way whatsoever, especially t…

>To ask about what you don't know, to skip the 'tedious part' (I don't feel like looking for answers, especially troubleshooting arcane technical issues among pages of forums or social media, makes me smarter in any way whatsoever, especially that the information usually needs to be verified and taken with a grain of salt).

quoting the article:

Perhaps one of the more concerning findings is that participants in the LLM-to-Brain group repeatedly focused on a narrower set of ideas, as evidenced by n-gram analysis (see topics COURAGE, FORETHOUGHT, and PERFECT in Figures 82, 83, and 85, respectively) and supported by interview responses. This repetition suggests that many participants may not have engaged deeply with the topics or critically examined the material provided by the LLM.

When individuals fail to critically engage with a subject, their writing might become biased and superficial. This pattern reflects the accumulation of cognitive debt, a condition in which repeated reliance on external systems like LLMs replaces the effortful cognitive processes required for independent thinking.

Cognitive debt defers mental effort in the short term but results in long-term costs, such as diminished critical inquiry, increased vulnerability to manipulation, decreased creativity. When participants reproduce suggestions without evaluating their accuracy or relevance, they not only forfeit ownership of the ideas but also risk internalizing shallow or biased perspectives.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#156

Post by 542354234235 »

falcor84 wrote: post #139
energy123 wrote: post #102

Earlier quoted context omitted.

The rule of thumb "LLMs are good at reducing text, not expanding it" is a good one here.

> "LLMs are good at reducing text, not expanding it" You put it in quote marks, but the only search results are from you writing it here on HN. Obviously LLMs are extremely good at expanding text, which is essentially what they do whenever they continue a prompt. Or did you mean that in a prescriptive way - that it would be better for us to use it more for summarizing rather than expanding?

>You put it in quote marks, but the only search results are from you writing it here on HN.

They said it was a rule of thumb, which is a general rule based on experience. In context with the comment they were replying to, it seems that they are saying that if you want to learn and understand something, you should put the effort in yourself first to synthesize your ideas and write out a full essay, then use an LLM to refine, tighten up, and polish it. In contrast to using an LLM as you go to take your core ideas and expand them. Both might end up very good essays, but your understanding will be much deeper if you follow the "LLMs are good at reducing text, not expanding it" rule.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#157

Post by 542354234235 »

eesmith wrote: post #97
gnabgib wrote: post #82

Earlier quoted context omitted.

Can you point at some references? Horse riding started around 3500 BC[0], while horse carriages started around 100BC [1], oxen/buffalo drawn devices around 3000 BC[1]. [0]: https://en.wikipedia.org/wiki/Equestrianism [1]: https://en.wikipedia.org/wiki/Carriage

I think you are reading "carriage" too specifically, when I suspect it's meant as a wider term for any horse-drawn wheeled vehicle. Your [0] says "Chariot burials about 2500 BC present the most direct hard evidence of horses used as working animals. In ancient times chariot warfare was followed by the use of war horses as light and heavy cavalry.", just after "the most unequivocal early archaeological evidence of equ…

Now I am more interested in prehistoric horse domestication than the AI essay writing.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#158

Post by 542354234235 »

bsenftner wrote: post #116

Well duh. Writing is thinking ordered, and thinking in your mind is not ordered unless one has specific training that organizes and orders their thinking - and even then it requires effort to maintain an organized perception. That is why we write: writing is our thoughts organized and frozen in an order that will be remain in order when related, without writing as the communications foundation the ideas/concepts woul…

Validating what is written is just confirming facts, and figures and making sure it is logical. It is not the same as synthesizing the original data, in terms of your level of understanding. If you need something to submit, an AI essay will do. But if you want to understand something, you really need to write it yourself.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#159

Post by Scratchthat »

An interesting thinking point on this is to, more broadly, consider the impact that advances in machinery have made to humanity's industrial sector. There are vast stories and accounts of people fearful of job loss/redundancy when we have inevitably developed an automation to take over more repetitive/mind numbing tasks. What ends up happening, generally, is you see humanity gain the ability to discover and innovate as they now have the time and energy to put into it.

What's interesting is I have to wonder if this is something that would extend to our own way of thinking, as discussed here with the short term affects we're already describing with increased dependence on LLMs, GPS systems, etc. There have been studies which have shown that those of who grew up using search engines exclusively did not lose or gain anything with respect to brain power, instead they developed a different means of retaining the information (i.e. they are less likely to remember the exact fact but they will remember how to find it). It makes me wonder if this is the next step in that same process and those of us in the transition period will lament what we think we'll lose, or if LLM dependency presents a point of diminishing return where we do lose a skill without replacing it.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#160

Post by acc_297 »

This has been on my mind for awhile and is why I only briefly used copilot on a daily basis.

I'm at the beginning of my career and learning every day - I could do my job faster with an LLM assistant but I would lose out on an opportunity to acquire skills. I don't buy the argument that low-level critical thinking skills are obsolete and high level conceptual planning is all that anyone will need 10 years from now.

On a more sentimental level I personally feel that there is meaning in knowing things and knowing how to do things and I'm proud of what I know and what I know how to do.

Using LLM's doesn't look particularly hard and if I need to use one in the future I'll just pick whichever one is supposedly the newest and best but for now I'm content to toil away on my own.

Post reply on HN