Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The peril of laziness lost

bcantrill.dtrace.org

Post reply on HN
Print view View original

101–110 of 148 posts

Re: The peril of laziness lost

#101

Post by HarHarVeryFunny »

WatchDog wrote: post #98
HarHarVeryFunny wrote: post #90

Earlier quoted context omitted.

Meta apparently now has a "leaderboard" for who is using the most AI - consuming the most tokens. Must make Anthropic happy, since Meta is using Claude, and accounts for some significant percentage (10%? 20%?) of their total volume.

Token usage is a different and more sympathetic heuristic than LOC produced. The metric by itself tells you nothing about what value those tokens produced, but to some extent it represents the amount of thinking you are able to offload to the computer for you. Wide breadth problems seem to scale well with usage, like scanning millions of LOC of code for vulnerabilities, such as the recent claude mythos results.

The trouble with rewarding token usage is the same as rewarding LOC written/generated - if that's what you are asking for then that is what you will get. Asking the AI to "scan the entire codebase for vulnerabilities" would certainly be a good way to climb the leaderboard!

Re: The peril of laziness lost

#102

Post by ignoramous »

reenorap wrote: post #87

As someone who has switched to exclusively coded using AI after 30 years of coding by myself, I find it really weird when people take credit for the lines of code ad features that AI generates. Flexing that one "coded" tens of hundreds of thousands of lines per day is a bit cringe, seeing as though it's really just the prompt that one typed.

Some argue, LoC is irrelevant as a quality/complexity metric as (in this new software product development lifecycle) implementation + testing + maintainance is wholly overseen by agents.

It has never been possible to code & deploy software with all but specs. Whatever software Garry is building are products he couldn't otherwise. LoC, in that context, serves as a reminder of the capabilities of the agents to power/slog through reqs/specs (quite incredibly so).

Besides, critical human review can always be fed back as instructions to agents.

Re: The peril of laziness lost

#103

Post by btrettel »

colechristensen wrote: post #100
btrettel wrote: post #70

Earlier quoted context omitted.

What I've observed in computational fluid dynamics is that LLMs seem to grab common validation cases used often in the literature, regardless of the relevance to the problem at hand. "Lid-driven cavity" cases were used by the two vibe coded simulators I commented on at r/cfd, for instance. I never liked the lid-driven cavity problem because it rarely ever resembles an actual use case. A way better validation case wou…

Both of these points seem like they would be easy to instruct an LLM to shape its testing strategy.

I think so too. If unclear, I don't use LLMs for coding at the moment and was just commenting on what I've seen from others who do in computational fluid dynamics.

Edit: Let me add that while I think it would be easy to instruct a LLM to do what I'd like, LLMs don't do these things by default despite them being recognized as best practices, and I'm not confident in LLMs getting the data or references right for validation tests. My own experience is that LLMs are pretty bad when it comes to reproducing citations, and they tend to miss a lot of the literature.

Re: The peril of laziness lost

#104

Post by WatchDog »

HarHarVeryFunny wrote: post #101
WatchDog wrote: post #98

Earlier quoted context omitted.

Token usage is a different and more sympathetic heuristic than LOC produced. The metric by itself tells you nothing about what value those tokens produced, but to some extent it represents the amount of thinking you are able to offload to the computer for you. Wide breadth problems seem to scale well with usage, like scanning millions of LOC of code for vulnerabilities, such as the recent claude mythos results.

The trouble with rewarding token usage is the same as rewarding LOC written/generated - if that's what you are asking for then that is what you will get. Asking the AI to "scan the entire codebase for vulnerabilities" would certainly be a good way to climb the leaderboard!

Absolutely, no one should be rewarded for either tokens used or LOC generated. I just think in the absence of any incentives, token usage is a better heuristic as to value produced than LOC generated.

Re: The peril of laziness lost

#105

Post by LelouBil »

Tuna-Fish wrote: post #83
CJefferson wrote: post #74

Earlier quoted context omitted.

Yes, I've found tests are the one thing I need to write. I then also need to be sure to keep 'git diff'ing the tests, to make sure claude doesn't decide to 'fix' the tests when it's code doesn't work. When I am rigourous about the tests, Claude has done an amazing job implementing some tricky algorithms from some difficult academic papers, saving me time overall, but it does require more babysitting than I would like…

Give claude a separate user, make the tests not writable for it. Generally you should limit claude to only have write access to the specific things it needs to edit, this will save you tokens because it will fail faster when it goes off the rails.

Don't even need a separate user if you're on linux (or wsl), just use the sandbox feature, you can specify allowed directories for read and/or write.

The sandbox is powered by bubblewrap (used by Flatpaks) so I trust it.

Re: The peril of laziness lost

#106

Post by marcus_holmes »

suzzer99 wrote: post #3

> Generally, though, most of us need to think about using more abstraction rather than less. Maybe this was true when Programming Perl was written, but I see the opposite much more often now. I'm a big fan of WET - Write Everything Twice (stolen from comments here), then the third time think about maybe creating a new abstraction.

"Duplication is far cheaper than the wrong abstraction"

Sandi Metz https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction

Re: The peril of laziness lost

#107

Post by ocrow »

arthurjj wrote: post #9

LLMs not being lazy enough definitely feels true. But it's unclear to me if it a permanent issue, one that will be fixed in the next model upgrade or just one your agent framework/CICD framework takes care of. e.g. Right now when using agents after I'm "done" with the feature and I commit I usually prompt "Check for any bugs or refactorings we should do" I could see a CICD step that says "Look at the last N commits a…

I've tried this approach of instructing the LLM to look for opportunities to abstract, but it's not good at finding the commonalities after the fact, when possibly related functions have already diverted unnecessarily. It writes "sloppy" code, that is to say code that is locally correct but which fails to build towards overall generalizations, but that sloppy code is a cul-de-sac: easy to write, but adding to messiness, and really tough to improve.

When a good programmer writes a new feature, they are looking for both existing and new abstractions that can be applied. They are considering their mental model of the whole system and examining whether it can be leveraged or needs to be updated. That's how they avoid compounding complications.

In order to take a big picture view like that, the LLM needs the right context. It would need to focus on what its system model is and decide when to update that system model. For now, just telling it what to write isn't enough to get good code. You have to tell it what to pay attention to.

Re: The peril of laziness lost

#108

Post by marcus_holmes »

reenorap wrote: post #87

As someone who has switched to exclusively coded using AI after 30 years of coding by myself, I find it really weird when people take credit for the lines of code ad features that AI generates. Flexing that one "coded" tens of hundreds of thousands of lines per day is a bit cringe, seeing as though it's really just the prompt that one typed.

Yes!

I don't mind it so much when it's a newbie or non-techie who has never actually written code before, because bless their hearts, they did it! They got some code working!

But if you've been developing for decades, you know that counting lines of code means nothing, less than nothing. That you could probably achieve the same result in half the lines if you thought about it a bit longer.

And to claim this as an achievement when it's LLM-generated... that's not a boast. That doesn't mean what you think it means.

But I guess we hit the same old problem that we've always had - how do you measure productivity in software development? If you wanted to boast about how an LLM is making you 100x more productive, what metric could you use? LOC is the most easily measurable, really, really, terrible measure that PMs have been using since we started doing this, because everything else is hard.

Re: The peril of laziness lost

#109

Post by geodel »

wiseowise wrote: post #44
quantummagic wrote: post #25

Earlier quoted context omitted.

Where my fellow ninety-percenters at?

I’m here man. Just want to make money and support my family. Couldn’t care less what some German general thinks about me. Even less care about online clowns trying to put people in buckets.

> Just want to make money and support my family.

That'd be just fine. But you do seem to care and feel hurt enough to call people online clowns.

Re: The peril of laziness lost

#110

Post by eucyclos »

marcus_holmes wrote: post #108
reenorap wrote: post #87

As someone who has switched to exclusively coded using AI after 30 years of coding by myself, I find it really weird when people take credit for the lines of code ad features that AI generates. Flexing that one "coded" tens of hundreds of thousands of lines per day is a bit cringe, seeing as though it's really just the prompt that one typed.

Yes! I don't mind it so much when it's a newbie or non-techie who has never actually written code before, because bless their hearts, they did it! They got some code working! But if you've been developing for decades, you know that counting lines of code means nothing, less than nothing. That you could probably achieve the same result in half the lines if you thought about it a bit longer. And to claim this as an ach…

I forget who said it, but I heard the idea floated that if your work can be measured in terms of productivity at all, it can and probably should be done by software. Not sure how that applies here since as you point out, a 10x programmer probably doesn't produce 10x the code.
Post reply on HN