Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

Post reply on HN
Print view View original

71–80 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#71

Post by audunw »

barrenko wrote: post #49
SchemaLoad wrote: post #36

Earlier quoted context omitted.

I use AI tools for amusement and asking random questions, but for actual work, I basically don't use them at all. I wonder if I'll be part of the increasingly rare group who is actually able to do anything while the rest become progressively more incompetent.

My nickel - we are in the primary stages of being given something like the famed "bicycle for the mind", an exoskeleton for the brain. At first when someone gives you a mech, you're like "woah, cool", let's see what it can do. And then you zip around, smash rocks, buildings, go try to lift the Eiffel. After a while you get bored of it (duh), and go back to doing what you usually do, utilizing the "bicycle" for the ki…

“Bicycle for the mind” analogy is actually really good here. Since bicycles and other transportation technology has made us increasingly weak, which has a negative impact on physical health. At this point it has reached such a critical point that people are taking seriously the fact that we need physical exercise to be in good health. My company recently introduced 60 minutes a week of activity during work hours. It’s probably a good investment since physical health affects performance and mental health.

Coming back to AI, maybe in the future we will need to explicitly take mental exercise as seriously as we do with physical exercise now. Perhaps people will go to mental gyms. (That’s just a school you may say, but I think the focus could be different: Not having a goal to complete a class and then finish, but continuous mental exercises..)

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#73

Post by namaria »

energy123 wrote: post #34
niemandhier wrote: post #23

AI is the anti-Zettelkasten. Rather than getting ever deeper insight into a subject matter by actively working on it, you iterate fast but shallow over a corpus of AI generated content. Example: I wanted to understand the situation in the Middle East better so I wrote an 10 page essay on the genesis if Hammas and Hizbulah using OpenAI as a cowriter. I remember nothing, worse of the things I remember I don’t know if i…

I'm on the optimistic side with how useful LLMs are, but I have to agree. You cultivate the instinct for how to steer the models and reduce hallucinations, but you're not building articulable knowledge or engaging in challenging thinking. It's more learning muscle-memory reactions to certain forms of LLM output that lean you towards trusting the output more, trying another prompting strategy, clearing context or not,…

Maybe, much like we invented gyms to exercise after civilization made most physical labor redundant (at least in developed countries), we will see a rise of 'creative writing gyms' of some sort in the future.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#74

Post by hyper57 »

hamdouni wrote: post #61
Davidzheng wrote: post #42

Earlier quoted context omitted.

I disagree with this take. I'd say often when exploring new math problems, often it's possible explore the possible solutions paths at lower technical levels first in your mind before anything down--when actually going into details of an approach. I don't think not writing is that limiting if all of your approaches already fail before going into details, which is often the case in early stages of math research.

I can also explore by writing. Writing drafts can help structure my thinking.

"The pen is an instrument of discovery rather than just a recording implement." ~ Billy Collins

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#76

Post by teekert »

jsrozner wrote: post #13

I wouldn't call it "accumulation of cognitive debt"; just call it cognitive decline, or loss of cognitive skills. And also DUH. If you stop speaking a language you forget it. The brain does not retain information that it does not need. Anybody remember the couple studies on the use of google maps for navigation? One was "Habitual use of GPS negatively impacts spatial memory during self-guided navigation"; another rep…

I would call it cognitive debt. Have you ever tried writing a large report with an LLM?

It's very tempting to let it write a lot, let it structure things, let it make arguments and visuals. It's easy to let it do more and more... And then you end up with something that is very much... Not yours.

But your name is on it, you are asked to explain it, to understand it even better than it is written down. Surely the report is just a "2D projection" of some "high dimensional reality" that you have in you head... right? Normally it is, but when you spit out a report in 1/10th of the time it isn't. You struggle to explain concepts, even though they look nice on paper.

I found that I just really have to do the work, to develop the mental models, to articulate and to re-articulate and re-articulate again. For different audiences in different ways.

I like the term cognitive debt as a description of the gap between what mental models one would have to develop pre-LLMs to get a report out, and how little you may need with an LLM.

In the end it is your name on that report/paper, what can we expect of you, the author? Maybe that will start slipping and we start expecting less over time? Maybe we can start skipping authors altogether and rely on the LLM's "mental" model when we have in depth questions about a report/paper... Who knows. But different models (like LLMs) may have different "models" (predictive algorithms) of underlying truth/reality. What allows for most accurate predictions? One needs a certain "depth of understanding". Writing while relying too much on LLMs will not give it to you.

Over time indeed this may lead to a population "cognitive decline, or loss of cognitive skills." I don't dare to say that. Book printing didn't do that, although it was expected at the time by the religious elite, they worried that normal humans would not be able to interpret texts correctly.

As remarked here in this thread before, I really do think that "Writing is thinking" (but perhaps there is something better than writing which we haven't invented yet). And thinking is: Developing a detailed mental model that allows you to predict the future with a probability better than chance. Our survival depends on it, in fact it is what evolution is in terms of information theory [0]. "Nothing in biology makes sense except in the light of ... information."

[0] https://www.youtube.com/watch?v=4PCHelnFKGc

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#77

Post by rusk »

eru wrote: post #59
rusk wrote: post #44

Earlier quoted context omitted.

Riding a bike is a skill rather than what we would call a “memory” per se. It’s a skill that develops a new neural pathway throughout your extended nervous system bringing together the lesser senses of proprioception and balance. Once you bring these things together you then go on to use them for other things. You “know” (grok), rather than “understand” how a bike stays upright on a very deep physical level.

Sure. But speaking a language is also (at least partially) a skill, ain't it?

It is. It’s also something you don’t forget except in extreme cases like dementia. Skills are different from facts but we use the word memory interchangeably for each. It’s this nuance of language that causes a category error in your reasoning ain’t it.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#78

Post by namaria »

Frummy wrote: post #4

I can’t believe riding a horse and carriage wouldn’t make you better at riding a horse. Sure a horserider wouldn’t want to practice the wrong way, but anyone else just wants to get somewhere

Horse riding was invented much later than carriages, and it revolutionized warfare.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#79

Post by namaria »

eru wrote: post #18
jameson wrote: post #11

> The LLM undeniably reduced the friction involved in answering participants' questions compared to the Search Engine. However, this convenience came at a cognitive cost, diminishing users' inclination to critically evaluate the LLM's output or ”opinions” (probabilistic answers based on the training datasets). This highlights a concerning evolution of the 'echo chamber' effect: rather than disappearing, it has adapte…

> What is ranked as “top” is ultimately influenced by the priorities of the LLM's shareholders [123, 125]. As if that's anything new. There's the adage that's older than electronics, that freedom of the press is freedom for those who can afford to own a printing press. > However, this convenience came at a cognitive cost, diminishing users' inclination to critically evaluate the LLM's output or ”opinions” (probabilis…

> Reminds me of Plato's concern about reading and writing dulling your mind. (I think he had his sock puppet Socrates express the concern. But I could be wrong.)

Nope.

Read the dialogue (Phaedrus). It's about rhetoric and writing down political discourses. Writing had existed for millennia. And the bit about writing being detrimental is from a mythical Egyptian king talking to a god, just a throwaway story used in the dialogue to make a tiny point.

In fact the conclusion of that bit of the dialogue is that merely having access to text may give an illusion of understanding. Quite relevant and on point I'd say.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#80

Post by seanmcdirmid »

My hand writing has suffered since I’ve heavily relied on keyboards for the last few decades. I can’t even produce a consistent signature anymore. My stick shift skills also suffered when I used an automatic for so long (and now I have an EV, I’m forgetting what gears are at all).

Rather than lament that the machine has gotten better than us at producing what we’re always mostly vacuous essays anyways, we have to instead look at more pointed writing tasks and practice those instead. Actually, I never really learned how to write until I hit grad school and had messages I actually wanted to communicate. Whatever I was doing before really wasn’t that helpful, it was missing focus. Having ChatGPT write an essay I don’t really care about only seems slightly worse than writing it myself.

Post reply on HN