Live data from Hacker News

Accumulation of cognitive debt when using an AI assistant for essay writing task

arxiv.org

221–230 of 442 posts

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#221
post #206

Earlier quoted context omitted.

LLMs are great for the poor! If you are rich, you can afford a good mentor. (That's true literally, in the sense of being rich in money and paying for a mentor. But also more metaphorically for people rich in connections and other resources.) If you are poor, you used to be out of luck. But now everyone can afford a nearly-free mentor in the form of an LLM. Of course, at the moment the LLM-mentor is still below the b…

If people are using it to critically question their beliefs and thinking, that is. However, most of the hype around LLMs is that they take out the difficult task of thinking and allow the creation of the artifact (documents, code or something else) that is really dangerous.

How is it any worse than the status quo for the disenfranchised?

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#222

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

> against the credulous lovers of "mediocrity" (cf. https://fly.io/blog/youre-all-nuts/ ) I read that article when it was posted on HN, and it's full of bad faith interpretations of the various objections to using LLM-assisted coding. Given that the article comes from a person whose expertise and viewpoints I respected, I had to run it through a friend; who suggested a more cynical interpretation that the article mig…

More charitably, it's a person yet to feel the disabling phase of using an LLM.

If he's a security researcher, then I'd imagine much of his LLM use is outside his area of expertise. He's probably not using it to replace his security research.

I think the revulsion to LLMs by experts is during that phase when its clearly mentally disabling you.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#223

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

> In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people I believe that one of the most underappreciated skills in business is the ability to string a coherent narrative together. I attend many meetings with extremely-talented engineers who are incapable of presenting their arguments in a manner that others (both technical and non-techni…

> I believe that one of the most underappreciated skills in business is the ability to string a coherent narrative together. I attend many meetings with extremely-talented engineers who are incapable of presenting their arguments in a manner that others (both technical and non-technical) can follow them.

Yes, but the arguments they need to present are not necessarily the ones they used to convince themselves, or their own reasoning history that made them arrive at their proposal. Usually that is an overly boring graph search like "we could do X but that would require Y which has disadvantage Z that theoretically could be salvaged by W, but we've seen W fail in project Q and especially Y would make such a failure more likely due to reason T, so Y isn't viable and therefore X is not a good choice even if some people argue that Y isn't a strict requirement, but actually it is if we think in a timeline of several years and blabla" especially if the decision makers have no time and no understanding of what the words X, Y, Z, W, Q, T etc. truly mean. Especially if the true reason also involves some kind of unspeakable office politics like wanting to push the tools developed by a particular team as opposed to another or wanting to use some tech for CV reasons.

The narrative to be crafted has to be tailored for the point of view of the decision maker. How can you make your proposal look attractive relative to their incentives, their career goals, how will it make them look good and avoid risks of trouble or bad optics. Is it faster? Is it allowing them to use sexy buzzwords? Does it line up nicely with the corporate slogan this quarter? For these you have to understand their context as well. People rarely announce these things, and a clueless engineer can step over people's toes, who will not squarely explain the real reason for their pushback, they will make up some nonsense, and the clueless guy will think the other person is just too dumb to follow the reasoning.

It's not simply about language use skills, as in wordsmithing, it's also strategizing and putting yourself in other people's shoes, trying to understand social dynamics and how it interacts with the detailed technical aspects.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#224
post #201

Earlier quoted context omitted.

The next stage of this issue is: how do you explain something you didn't write? The LLM-optimist view at the moment, which takes on board the need to review LLMs, assumes that this review capability will exist. I cannot review LLM output on areas outside of my expertise. I cannot develop the expertise I need if I use an LLM in-the-large. I first encountered this issue ~year-ago when using an LLM to prototype a progra…

IMO we are likely in a golden era of coding LLM productivity, one in which the people using them are also experts. Once there are no coding experts left, will we still see better productivity?

Yea, and how are people going to learn when the answer is just a chat away. I know it would have been hard for me to learn programming if I knew I could just ask for the solution everytime (no stackoverflow does not count because most people dont ask a question for every single issue they encounter like with AI)

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#225
One thing that is also truly unappreciated is most of us humans actually enjoy thinking, and people are trying to make llms strip us from a fundamental thing we enjoy doing. Look at all the people that enjoy solving problems for the sake of it

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#226

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

Sad reality is that most people are not smart. They’re not creative, original, or profound. Think back to all the empty and pointless convos you had prior to AI or the web.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#227

Earlier quoted context omitted.

> This is a system for substituting thinking itself with non-thinking One of my favorite developments on the internet in the past few years is the rise of the “I don’t think/won’t think/didn’t think” brag posts On its own it would be a funny internet culture phenomenon but paired with the fact that you can’t confidently assume that anybody even wrote what you’re reading it is hilarious

> One of my favorite developments on the internet in the past few years is the rise of the “I don’t think/won’t think/didn’t think” brag posts Sorry, I can't immediately think of what you're talking about. Could you link to an example so I can get a feel for it?

Every time you have ever seen somebody weigh into an evenly slightly complex topic with a post that starts with “I asked chatgpt and”

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#228

It feels, more and more, that LLMs will be another technology that society will inoculate itself against. It's already starting to happen in education: teachers conversing with students, observing them learn, observing them demonstrate their skills. In business, quickly I'd say, we will realise that the vast majority of worthwhile communication necessarily must be produced by people -- as authors of what they want to…

It’s been my experience that most people opinions on AI is inversely proportional to the timescale they have been using it.

Using AI is kind of like having a Monika Closet. You just push all the stuff you don’t know to the side until it’s out of view. You then think everything is clean, and can fool yourself into thinking so for a while.

But then you need to find something in that closet and just weep for days.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#229
post #201

Earlier quoted context omitted.

IMO we are likely in a golden era of coding LLM productivity, one in which the people using them are also experts. Once there are no coding experts left, will we still see better productivity?

Yea, and how are people going to learn when the answer is just a chat away. I know it would have been hard for me to learn programming if I knew I could just ask for the solution everytime (no stackoverflow does not count because most people dont ask a question for every single issue they encounter like with AI)

this was the same criticism against SO at the time too. people who want to learn and put in the effort will learn even faster with AI (asking, exploring the answers, etc.) and those who use it as a crutch will be left behind as always. we're just in the confusing staging.

Re: Accumulation of cognitive debt when using an AI assistant for essay writing task

#230

Earlier quoted context omitted.

Didn't realise the pedigree of the idea went back to 1983. I read about this in a book "Our Robots, Ourselves". That talked about airline pilots' experience with auto-land systems introduced in the late 1990s/ early 2000s. As you'd expect after having read Ironies of Automation, after a few near misses and not misses, auto-land is not used any more. Instead, pilot augmentation with head-up displays is used. What is t…

>What is the programming equivalent of a head-up display? Syntax highlighting, Intellisense, and the millions of other little features built into modern editors.

We should be able to do a lot more than that. I for one would love to have UML as the basis of for system design and architecture, have "pseudo-code repositories" that can be used as a "pattern book" and leave that as the context for LLM-based code generation tools. We could then define a bunch of constraints (maximum cyclomatic complexity, strict type checking, acceptance tests that must pass, removal of dead code) to reduce the chances of the LLM going rampant and hallucinating.

This way I'd still be forced to think about the system, without having to waste time with the tedious part of writing code, fixing typos, etc.

Bonus point: this could become a two-way system between different programming languages and UML as the intermediate representation, which would make a lot easier to port applications to different languages, and would eliminate concerns about premature optimizations. People could still experiment with new ideas in languages that are more accessible (Python/Javascript) and later on port them to more performant systems (Rust/D/C/C++).

Post reply on HN