Earlier quoted context omitted.
> I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people Now one like me might go and ask how much of communication is actually worthwhile? Sometimes I consider that there is lot of communication that might not actually be. It is still done, but if no one actually reads it, why not automate generation. Not to say there is not significant amount of stuff you…
It's not about getting it right, its about having thought about it. Authoring means thinking-thru, owning, etc. There's a tremendous hollowing-out of our mental capacities caused by the computer science framing of activities in terms of input->output, as-if the point is to obtain the output "by any means". It would not matter if the LLM gave exactly the same output as you had written, and always did. Because you stil…
Accumulation of cognitive debt when using an AI assistant for essay writing task
211–220 of 442 posts
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#212It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…
> In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people
I believe that one of the most underappreciated skills in business is the ability to string a coherent narrative together. I attend many meetings with extremely-talented engineers who are incapable of presenting their arguments in a manner that others (both technical and non-technical) can follow them. There is an artistry to writing and speaking that I am only now in my late forties beginning to truly appreciate. Language is a powerful tool, the choice of a single word can sometimes make or break an argument.
I don't see how LLMs can do anything but significantly worsen this situation overall.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#213Honestly, my general feeling with LLMs and large language models is that they cure very man-made issues.
They're brilliant in what I always feel is entangled communication, beurocratic maintenence. Like someone mentioned further down, they work great at Concept Processing.
But it feels like a solution to the over saturation of stupid SEO, terrible google search, and overall rise in massive documents that write for the sake of writing.
I've actually found myself beginning to use LLMS more to find me the core sources of information that are useful rather than terrible SEO optimization, rather than as a personal assistant.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#214Earlier quoted context omitted.
> I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people Now one like me might go and ask how much of communication is actually worthwhile? Sometimes I consider that there is lot of communication that might not actually be. It is still done, but if no one actually reads it, why not automate generation. Not to say there is not significant amount of stuff you…
It's not about getting it right, its about having thought about it. Authoring means thinking-thru, owning, etc. There's a tremendous hollowing-out of our mental capacities caused by the computer science framing of activities in terms of input->output, as-if the point is to obtain the output "by any means". It would not matter if the LLM gave exactly the same output as you had written, and always did. Because you stil…
> It's not about getting it right, its about having thought about it. Authoring means thinking-thru, owning, etc.
So much this.
At my current workplace, I was asked to write up a design doc for a software system. The contents of the document itself weren't very relevant as the design deviated significantly based on constraints and feedback that could be discovered only after beginning the implementation, but it was the act of putting together that document, thinking about the various cases, etc. that lead to the formation of a mental model that helped me work towards delivering that system.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#215Earlier quoted context omitted.
It's not about getting it right, its about having thought about it. Authoring means thinking-thru, owning, etc. There's a tremendous hollowing-out of our mental capacities caused by the computer science framing of activities in terms of input->output, as-if the point is to obtain the output "by any means". It would not matter if the LLM gave exactly the same output as you had written, and always did. Because you stil…
“Plans are worthless, planning is invaluable”
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#216Love this study because it reinforces my own biases but also love that a study was done to actually check it. With that said, would be a study that finds out that people using motorcycles or cars to move around exclusively gets their leg and body atrophied in comparison to people who walk all the day to do their things. Totally. It's just plain obvious. The gist is in the trade-offs: can I do more things or things I…
The motorcycle/car metaphor here is really interesting. We really don't know yet, but it could indeed be that lack of access to AI would be similar to how teenagers growing up in a small town without good public transport or access to a car or motorcycle would have a different adolescence experience from those growing up with a convenient mode of travel. You can argue that either experience is "better" but they are inarguably different.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#217Earlier quoted context omitted.
I mean, they found brain atrophy. If this doesn't get someone worried, I don't know what would. I joked that "I don't do drugs" when someone asked me whether I played MMORPGs, but this joke becomes just too real when we apply it to generative AI of any soırt.
> I mean, they found brain atrophy. Where did you get that from? While the article mentions the word "atrophy" twice, it's not something that they found. They just saw less neural activation in regards to essay writing in those people who didn't write the essay themselves. I don't anything there in regards to the brain as a whole.
If physical exercise builds muscle mass, mental work and exercise builds more connections in your brain.
Like everything, not using something causes that thing to atrophy. IOW, if you depend on something too much, you'll grow dependency on it, because that part of your body doesn't do the work that much anymore.
Brain is an interconnected jungle. Improvement in any ability will improve other, adjacent abilities. You need to think faster to type faster. If you can't think faster, you'll stagnate, for example.
Also, human body always tries to optimize itself to reduce its energy consumption. If you get a chemical from outside, it'll not produce it anymore, assuming the supply will be there. Brain will reduce its connections in some region if that function is augmented by something else.
Same for skill atrophies. If you lose one skill, you lose the connections in your brain, and that'll degrade adjacent skills, too. As a result, skill atrophy is brain atrophy in the long run.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#218It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…
> against the credulous lovers of "mediocrity" (cf. https://fly.io/blog/youre-all-nuts/ ) I read that article when it was posted on HN, and it's full of bad faith interpretations of the various objections to using LLM-assisted coding. Given that the article comes from a person whose expertise and viewpoints I respected, I had to run it through a friend; who suggested a more cynical interpretation that the article mig…
Now I'm a fairly cynical person by trade but that feels like it's straying into conspiracy theory territory.
And of course the key point is that the author of that article isn't (IMO) working in the security research field any more, they work at fly.io on the security of that platform.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#219I worry about the adverse effects of LLM on already disfranchised populations - you know the poor etc - that usually would have to pull themselves up using hard work etc studying n reading hard. now if you don't have a mentor to tell you in the age of LLM you still have to do things the hard / old school way to develop critical thinking - you might end up taking shortcuts and have the LLMs "think" for you. hence agai…
LLMs are great for the poor! If you are rich, you can afford a good mentor. (That's true literally, in the sense of being rich in money and paying for a mentor. But also more metaphorically for people rich in connections and other resources.) If you are poor, you used to be out of luck. But now everyone can afford a nearly-free mentor in the form of an LLM. Of course, at the moment the LLM-mentor is still below the b…
If people are using it to critically question their beliefs and thinking, that is.
However, most of the hype around LLMs is that they take out the difficult task of thinking and allow the creation of the artifact (documents, code or something else) that is really dangerous.
Re: Accumulation of cognitive debt when using an AI assistant for essay writing task
#220It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…
It's all already there. When you converse with a junior-engineer about their latest and greatest idea (over a chat platform), and they start giving you real-time responses which are a page long and structured into bullet points...it's not even that they are using chatgpt to avoid thinking, it is the fact that they think either no-one will notice, or that this is how grown-ups actually converse with each other, is terrifying.