Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

Post reply on HN
Print view View original

181–190 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#181

Post by stevage »

tuatoru wrote: post #177
fhd2 wrote: post #171

Earlier quoted context omitted.

Appears to align with good old Ironies of Automation [1]. If humans just review and rubber stamp results, they do a pretty terrible job at it. I've been thinking for a while now that in order to truly make augmented workflows work, the mode of engagement is central. Reviewing LLM code? Bah. Having an LLM watch over my changes and give feedback? Different story. It's probably gonna be difficult and not particularly po…

Didn't realise the pedigree of the idea went back to 1983. I read about this in a book "Our Robots, Ourselves". That talked about airline pilots' experience with auto-land systems introduced in the late 1990s/ early 2000s. As you'd expect after having read Ironies of Automation, after a few near misses and not misses, auto-land is not used any more. Instead, pilot augmentation with head-up displays is used. What is t…

>What is the programming equivalent of a head-up display?

Syntax highlighting, Intellisense, and the millions of other little features built into modern editors.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#182

Post by jonplackett »

I think we need to shift our idea of what LLMs do and stop thinking they are ‘thinking’ in any human way.

The best mental description I have come up with is they are “Concept Processors”. Which is still awesome. Computers couldn’t understand concepts before. And now they can, and they can process and transform them in really interesting and amazing ways.

You can transform the concept of ‘a website that does X’ into code that expresses a website X.

But it’s not thinking. We still gotta do the thinking. And actually that’s good.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#183

Post by sesm »

mjburgess wrote: post #179

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

One of the effects on software development is: the fact that you submitted a PR with any LoC count doesn't mean that you did any work. You need to explain your solution and answer questions to prove that.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#184

Post by dzonga »

I worry about the adverse effects of LLM on already disfranchised populations - you know the poor etc - that usually would have to pull themselves up using hard work etc studying n reading hard.

now if you don't have a mentor to tell you in the age of LLM you still have to do things the hard / old school way to develop critical thinking - you might end up taking shortcuts and have the LLMs "think" for you. hence again leaving huge swaths of the population behind in critical thinking which is already in shortage.

LLMs are bad that they might show you the sources but also hallucinate about the sources. & most people won't bother going to check source material and question it.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#185

Post by metalman »

ToucanLoucan wrote: post #120
larodi wrote: post #56

Earlier quoted context omitted.

The impact of writing is immensely undervalued. Even writing with a keyboard or screen is a lot more than non writing. Exercising writing on any topic is still beneficial, and you can find many psychologists recommend having a daily blog of some sort to help people observe themselves from a side. The same goes for speaking, public speech if u want, and therapeutic daily acting-playing which is also overlooked. I’d lo…

The only writing I've ever used ChatGPT for is writing I openly don't give a shit about, and even then I constantly find myself prompting it to write less because holy shit do LLMs love to go on and on and on. Like not only do I cosign all said above, but I will also add to this: brevity is the soul of wit and none of these fucking things are brief. No matter what you ask for you end up getting just paragraphs of shi…

not all humans are brief, and not all situations are amenable to brevity, but I get the point, as brevity can be be exceptionaly informationaly dense, but like in humor(sports), it only works if someone else plays the strait guy or set up artist. Also true masters will switch up, happy to join ingeneral blather, and then drop a subtle, brief comment that is the bridge piece for an otherwise huge informational set.Another thing many performers and writers describe is the finding of the voice or stage/writing persona....perhaps quite different from the one that they inhabit at home. The topic at hand leaves out the trap of standing behind a persona, that the person cant then inhabit, and then can be caught out in a real world situation as an imposter, ha!

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#186

Post by HPsquared »

bayindirh wrote: post #176
Magmalgebra wrote: post #175

Well... yes? Essays are tools to force students to structure and communicate thinking - production of the essay forces the thinking. If you want an equivalent result from LLMs you're going to need a much more iterative process of critique and iteration to get the same kind of mental effort out of students. We haven't designed that process yet.

I mean, they found brain atrophy. If this doesn't get someone worried, I don't know what would. I joked that "I don't do drugs" when someone asked me whether I played MMORPGs, but this joke becomes just too real when we apply it to generative AI of any soırt.

LLMs are the tip of the iceberg when it comes to this stuff.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#187

Post by Magmalgebra »

bayindirh wrote: post #176
Magmalgebra wrote: post #175

Well... yes? Essays are tools to force students to structure and communicate thinking - production of the essay forces the thinking. If you want an equivalent result from LLMs you're going to need a much more iterative process of critique and iteration to get the same kind of mental effort out of students. We haven't designed that process yet.

I mean, they found brain atrophy. If this doesn't get someone worried, I don't know what would. I joked that "I don't do drugs" when someone asked me whether I played MMORPGs, but this joke becomes just too real when we apply it to generative AI of any soırt.

As someone who used to teach, this does not worry me (also, they mention skill atrophy - inherently less concerning).

Putting ChatGPT in front of a child and asking them to do existing tasks is an obviously disasterous pedagogical choice for the reasons the article outlines. But it's not that hard to create a more constrained environment for the LLM to assist in a way that doesn't allow the student to escape thinking.

For writing - it's clear that finding the balance on how much time you ordering your thoughts and getting the LLM to write things is its own skillset, this will be its own skill we want to teach independent of "can you structure your thoughts in an essay"

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#188

Post by darkwater »

Love this study because it reinforces my own biases but also love that a study was done to actually check it.

With that said, would be a study that finds out that people using motorcycles or cars to move around exclusively gets their leg and body atrophied in comparison to people who walk all the day to do their things. Totally. It's just plain obvious. The gist is in the trade-offs: can I do more things or things I wasn't able to do before commuting by car? Sure. Am I going to be exposed to health issues if I never walk day in, day out? Most probably.

The exact same thing will happen with LLM, we are in the hype phase and any criticism is downplayed with "you are being left behind if you don't drink rocket fuel like we do" but in 10-15 years we will be complaining as a society that LLMs dumbed down our kids.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#189

Post by darkwater »

sesm wrote: post #183
mjburgess wrote: post #179

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

One of the effects on software development is: the fact that you submitted a PR with any LoC count doesn't mean that you did any work. You need to explain your solution and answer questions to prove that.

Explanation that the smartpants and some management are already totally willing to outsource to an LLM as well...

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#190

Post by antithesizer »

mjburgess wrote: post #179

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

>Authoring is two-thirds of the point of most communication.

Not when there's money to be made.

Post reply on HN