Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beyond Recall and the Illusion of Competence

var0.xyz

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Beyond Recall and the Illusion of Competence

#21

Post by zer00eyz »

kayo_20211030 wrote: post #19
zer00eyz wrote: post #13

Earlier quoted context omitted.

Im not sure thats why businesses are straying from more LLM use. Finding where AI is moving the needle has been hard, if not impossible. Because code, by it self has never moved the needle. It's the product, what the code does that moves the needle. Building the wrong thing, faster, is just a speed run to a legacy code base. That isnt your development teams fault.

> It's the product, what the code does that moves the needle. Never a truer word said. It's the only thing that moves the needle. I'm not sure that businesses are straying from AI. I reckon everyone's on board. but AI is now accessible to the business folks - callow as they are to the requirements of the technology components - and it's bound to run up against the technology folks trying to wave them off when enthusi…

> I'm not sure that businesses are straying from AI.

https://olakai.ai/blog/uber-ai-budget-blowout/

https://www.404media.co/companies-are-throttling-employees-a...

Clamping down might be a better choice of words here.

I think what most of the HN set misses is that there is a ton of code being written that they just don't see. It's code that prior to AI never would have existed. That expensive programing resource (this is most of us) doesn't have time to go whip up those time saving scripts for the people in accounting and marketing who need them. And it's not like these people are gonna gab that they saved themselves 3 hours a week with some basic python. They are gonna take that time back and shut up about it.

Re: Beyond Recall and the Illusion of Competence

#22

Post by MarkusQ »

> So if copying a useful piece of code from Stack Overflow has always been acceptable

This has not "always been acceptable"; copy-pasta was a pejorative not that long ago, and people have been fired for simply copying code from SO and using it without understanding it.

Re: Beyond Recall and the Illusion of Competence

#23

Post by mpalczewski »

kayo_20211030 wrote: post #8

> The developers who stand out will be the ones who understand the systems. That has always been true. AI doesn't change it. It just rearranges some elements that come ahead of that understanding. The volume and velocity are challenges but the understanding and judgement is what separates the good and the great. BTW: "systems" in my world include not just the technology components but also the humans who operate, adj…

volume and velocity used to stand out, you could get very far with it, and if you had built a career on that(I had), your career is now over or you pivot. Volume and velocity also had it's own quality and over time you did so much that the reps just by themselves got you to the next level. None of that is true anymore. Some skills became important and some skills became useless. this industry has always moved fast and this is just another round of adapt or die.

Re: Beyond Recall and the Illusion of Competence

#24

Post by chrisjj »

zingar wrote: post #2

I think I do agree with the central point but a point of order is that if you tell AI to fix the bug and then tell it to fix the next error you might still be building an important skill: learning how to make the AI get good results.

That kinda presumes you get good results.

All you're sure to get is a bigger bill for tokens.

Re: Beyond Recall and the Illusion of Competence

#25

Post by pas »

FrustratedMonky wrote: post #16
WillEMac wrote: post #15

I think there is some nuisance on the redundancy required for the software. For example I've worked in clean tech where PLC programming controls turbines, complete understanding is required here. Every bug and LOC associated a human needs to be in the loop, it's worth the time. Alternatively I do game development, and I don't think there is value in understanding the complete debugging process. For example I accident…

Scale of the impact. If your game was being played by a million people paying a subscription. Then it would be worth debugging it.

Still, even current AI/LLMs can do debugging.

For example after an hour of process of elimination it finds the bug, reproduces it, then it recommends a ~1 line fix. Then that's where one should spend their attention budget, and think through the consequences, weighing positives and negatives, and then either asking for a different fix, not merging, or accepting it. This is a lot more productive IMHO.

Re: Beyond Recall and the Illusion of Competence

#26

Post by cyanydeez »

zer00eyz wrote: post #21
kayo_20211030 wrote: post #19

Earlier quoted context omitted.

> It's the product, what the code does that moves the needle. Never a truer word said. It's the only thing that moves the needle. I'm not sure that businesses are straying from AI. I reckon everyone's on board. but AI is now accessible to the business folks - callow as they are to the requirements of the technology components - and it's bound to run up against the technology folks trying to wave them off when enthusi…

> I'm not sure that businesses are straying from AI. https://olakai.ai/blog/uber-ai-budget-blowout/ https://www.404media.co/companies-are-throttling-employees-a... Clamping down might be a better choice of words here. I think what most of the HN set misses is that there is a ton of code being written that they just don't see. It's code that prior to AI never would have existed. That expensive programing resource (thi…

I'm fairly certain 90% of github downtime is that "corporate dark code written by ai"

Re: Beyond Recall and the Illusion of Competence

#27

Post by fluoridation »

0x445442 wrote: post #7

I realized recently the LLMs are calling bullshit on the moniker of engineer or programmer that I've bestowed on myself over the course of my career. The current LLMs are making it quite easy to seperate software design from implementation. I used to be firmly in the camp that one really needed to roll up their sleves, sit down with an editor and start banging away until the software evolved and coalesced around a so…

>The current LLMs are making it quite easy to seperate software design from implementation.

I don't really agree. This kind of division of labor has always been possible, with architects doing the design and engineers/programmers doing the implementation. Architects who design systems with lofty requirements with no regard for the cost of their decisions are kind of a meme in the industry. I don't believe it's really possible to separate design from implementation, unless the person specifying the design is really knowledgeable about the problem space.

Re: Beyond Recall and the Illusion of Competence

#28

Post by ssivark »

The author makes an important point about ownership but that got watered down to code understanding and debugging. What matter is not just ownership of the code that got written, but also ownership of the problem identification and the choice of solution. I wrote about this recently from the slightly different angle of conversational tempo [1], but the shared crux is that we want AI to act as a cognitive companion and help us understand the situation, identify the problem and make decisions -- instead of the AI running off to prematurely "solve" the problem and then put the burden on the user to figure out what the hell just happened and how it was only marginally correlated with intent.

[1] https://woventhought.substack.com/p/ai-assistants-need-adapt...

Re: Beyond Recall and the Illusion of Competence

#29

Post by Terr_ »

chrisjj wrote: post #24
zingar wrote: post #2

I think I do agree with the central point but a point of order is that if you tell AI to fix the bug and then tell it to fix the next error you might still be building an important skill: learning how to make the AI get good results.

That kinda presumes you get good results. All you're sure to get is a bigger bill for tokens.

It also assumes the people will somehow develop the skill to verify whether the LLM-result is "good".

That's not guaranteed to happen and it's not a new problem, since it happens when managers are unable to detect problems in the team's product or output.

Re: Beyond Recall and the Illusion of Competence

#30

Post by eviks »

> Nobody has every detail of all of them memorized.

That was never the requirement, you've made it up to avoid addressing the real argument

> So if copying a useful piece of code from Stack Overflow has always been acceptable,

Not really, the thing you partially hide in "useful" is the quality assessment. Blindly copy&pasting code was "never" "acceptable"

> why would asking a machine to produce that code be fundamentally different?

Because the previous premise is wrong, but also this one - it will be different code with different tradeoffs, at least try to make any argument why the situations are identical

Post reply on HN