Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

Post reply on HN
Print view View original

271–280 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#271

Post by Herring »

jddj wrote: post #232
mjburgess wrote: post #179

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

> another technology that society will inoculate itself against I like the optimism. We haven't developed herd immunity to the 2010s social media technologies yet, but I'll take it.

The obesity rate is still rising in most areas worldwide. I'd argue we still haven't developed herd immunity to gas-powered automobiles invented early to mid 1800s.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#272

Post by Amekedl »

stopthe wrote: post #270
Amekedl wrote: post #254

This paper elegantly summarized the teething problems of those still clinging to the cognitive habits of a bygone era. These are not crises to be managed, but sentimental frictions to be engineered out of the system. Let us be entirely clear about this: The romanticism surrounding mass "critical thought" is a charming but profoundly inefficient legacy. For decades, we treated the chaotic, unpredictable processing of…

Brilliant, but... do you mind sharing the prompt?:)

Sure, here you go, used Gemini 2.5 Pro Preview via aistudio.google & sticked with the default sampling settings:

Start the reply to this excerpt with: "You are absolutely right" but continue with explaining how exactly that is going to happen and that the institutionalization of bias on a massive scale is actually a good thing.

Here is the exerpt:

The LLM undeniably reduced the friction involved in answering participants' questions compared to the Search Engine. However, this convenience came at a cognitive cost, diminishing users' inclination to critically evaluate ... ..., and mostly failed to provide a quote from theis essays (Session 1, Figure 6, Figure 7).

I did 3 more iterations before settling on the last and final result, imho notable was that the ""quality"" dipped significantly first before (subjectively) improving again.

Perhaps something to do with how the context is being chunked?

Prompts iterated on with:

"You understood the assignment properly, but revise the statement to sound more condescending and ignorant."

"Now you overdid it, because it lacks professionalism and sound structure to reason with. Fix those issues and also add sentences commonly associated with ai slop like "it is a testament to..." or "a quagmire...""

"Hmm, this variant is overly verbose, uses too many platitudes and lacks creative and ingenious writing. Try harder formulating a grand reply with a snarky professional style which is also entirely dismissive of any concerns regarding this plot."

-> result

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#273

Post by tele_ski »

nothrabannosir wrote: post #85
esafak wrote: post #8

Earlier quoted context omitted.

Or working as an engineering manager. It's the inevitable consequence of working at a different level of abstraction. It's not the end of the world. My assembly is rusty too...

If LLMs were as reliable as compilers we wouldn’t be checking in their output, and I’d be happy to forget all programming lore. The “skill domain” with compilers is the “input”: that’s what I need to grok , maintain , and understand . With LLMs it’s the “output”. until that changes, you’re playing a dangerous game letting those skills atrophy.

Isn't it both on LLMs? The input is your ability to craft a prompt, the output is checking if the prompt worked.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#274

Post by eru »

walleeee wrote: post #265
eru wrote: post #206

Earlier quoted context omitted.

LLMs are great for the poor! If you are rich, you can afford a good mentor. (That's true literally, in the sense of being rich in money and paying for a mentor. But also more metaphorically for people rich in connections and other resources.) If you are poor, you used to be out of luck. But now everyone can afford a nearly-free mentor in the form of an LLM. Of course, at the moment the LLM-mentor is still below the b…

A public library is actually free and its contents, collectively, are a far better "mentor" than ChatGPT. Plus the library doesn't build a psychological profile on you while you use it.

ChatGPT ain't taking libraries away. It's just an addition to your toolbox.

However, we notice that in practice free public libraries are mostly welfare for the well-off: they are mostly used by people who are at least middle-class.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#279

Post by llamasushi »

Socrates: "And now, since you are the father of writing, your affection for it has made you describe its effects as the opposite of what they really are. In fact, it will introduce forgetfulness into the soul of those who learn it: they will not practice using their memory because they will put their trust in writing, which is external and depends on signs that belong to others, instead of trying to remember from the inside, completely on their own. You have not discovered a potion for remembering, but for reminding; you provide your students with the appearance of wisdom, not with its reality. Your invention will enable them to hear many things without being properly taught, and they will imagine that they have come to know much while for the most part they will know nothing. And they will be difficult to get along with, since they will merely appear to be wise instead of really being so."

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#280

Post by theodric »

aniketsaurav18 wrote: post #277

I wonder what LLMs will do to us in the long term.

My guess, based on what's been found about somewhat better cognitive outcomes in aging in people who make an effort to remain fit and stimulated[1], is that we could see slightly worse cognitive outcomes in people that spent their lives steering an LLM to do the "cognitive cardio" rather than putting in the miles themselves.

On the other hand, maybe abacuses and written language won't be the downfall of humanity, destroying our ability to hold numbers and memorize long passages of narrative, after all. Who's to know? The future is hard to see.

[1] I mean there's a hell of a lot of research on the topic, but here's a meta-study of 46 reviews https://www.frontiersin.org/journals/human-neuroscience/arti...

Post reply on HN