Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring the sloppiness of code

earendil.com

Post reply on HN
Print view View original

161–170 of 247 posts

Re: Measuring the sloppiness of code

#161

Post by ryandrake »

danbruc wrote: post #19

Coding is not solved, correctness is not a feature, it is the bare minimum. If your code does not do what it is supposed to do, you could as well have no code at all. Efficiency, security, maintainability, reliability, readability, understandability, extensibility, maintainability, observability, portability, ... this is what high quality coding is about, not that it works, that is a given. And in my experience curre…

This has been a problem in the software industry for a loooong time, though. Too many developers see "It works" as the end state, a signal that you're done. "It works" is just step 1.

Re: Measuring the sloppiness of code

#162

Post by pdimitar »

flyinglizard wrote: post #144
einrealist wrote: post #106

Earlier quoted context omitted.

And there is another problem: LLMs generating too much code, code that is doing more than was asked. And that cannot be fixed by tests. Usually, we create tests for wanted behavior and expected exceptions. But we don't create tests for undesired behavior.

Code cost is almost down to zero. If you move the point of “just leave it to the machine” from the compiler (where humans used to do the coding) to the high level logic (now with LLMs) then in most cases more code does not really matter. Like, why build and maintain an abstraction where the LLM could implement this many times over each time with different subtleties? Why use a library with its own constraints when yo…

The "exactly" part is the problem.

Just last two weeks I had to slap Fable, three times, to stop writing 1000-2000 lines of defensive code... because of DB columns I just forgot should be NOT NULL. That was it. Nothing else. I told it that, boom, -4800 coding lines: gone.

LLMs defend the status quo and they regularly lose sight of everything bigger than the current PR they are working on.

I too am gradually making peace with the fact that LLM-maintained code does not have to be 100% readable for humans.

But this is not about readability. It's about the data model. So one concession I am willing to make is: don't care too much about the code _BUT_ manually curate the data model. So far: small wins on iteration turns and code volume producing. Too early to tell but for now I am happy with the results.

Re: Measuring the sloppiness of code

#163

Post by lukeschlather »

user43928 wrote: post #122
einrealist wrote: post #106

Earlier quoted context omitted.

And there is another problem: LLMs generating too much code, code that is doing more than was asked. And that cannot be fixed by tests. Usually, we create tests for wanted behavior and expected exceptions. But we don't create tests for undesired behavior.

Have you worked with Opus 5? Its documentation about what the code does not do could fill whole books. UI copy being full of slop explaining what the software does not do is another problem. I am not convinced that a lack of negative test cases is an issue. I do agree it generates too much code most of the time.

Documentation is important. I would say Opus' propensity to write documentation that documents non-features is part of the problem being discussed.

And the problem isn't just that it says what the software doesn't do, most of the things it claims are in fact meaningless, it's not even clearly describing something the software shouldn't do.

Re: Measuring the sloppiness of code

#165

Post by anilgulecha »

LaffertyDev wrote: post #125
anilgulecha wrote: post #117

> Qwen2.5-Coder-3B Basing it's findings of LLM as judge on this model, and then proceeding to ignore it. This article can be safely ignored as well. LLM as judge in harness evals is the way to go, for any of your custom needs. Design the eval well.

I don't know where you pulled that from, its not in the article.

Follow the paper from the LLM as judge section.

Re: Measuring the sloppiness of code

#166

Post by dnikolovv »

margalabargala wrote: post #129
heaney-555 wrote: post #92

Earlier quoted context omitted.

Plenty of serious production projects are doing exactly that. Are you using GPT-6 Astra, or something older?

You and the person you are replying to are talking about different things. Agents cannot be given a high level goal and then left unsupervised, for hours, without making some dumb decisions.

That's exactly what I meant.

Re: Measuring the sloppiness of code

#167

Post by andriy_koval »

scronkfinkle wrote: post #12

There is some sense of rose-tinted glasses of pre-LLM coding. A lot of human written code, particularly at the enterprise level, was of low quality well before AI automated it.

> A lot of human written code, particularly at the enterprise level, was of low quality well before AI automated it.

its just now the same humans weaponized by AI ship 20x more of low quality code.

Re: Measuring the sloppiness of code

#168

Post by nialv7 »

avgDev wrote: post #43
scronkfinkle wrote: post #12

There is some sense of rose-tinted glasses of pre-LLM coding. A lot of human written code, particularly at the enterprise level, was of low quality well before AI automated it.

Writing code at enterprise level is insanely difficult. You are constrained by budget, staff, legacy databases/environments, business rules hiding all over the place, and people. You can't just rewrite everything. So over many years people are touching small parts of the pie. If it works it isn't low quality.

> If it works it isn't low quality.

then the same applies to AI written code?

Re: Measuring the sloppiness of code

#170

Post by denverllc »

heaney-555 wrote: post #92
dnikolovv wrote: post #81

Earlier quoted context omitted.

Right. It feels like we're living in a parallel world or something. Can agents code? Sure. Can you let them code on their own for a serious production project? Not a chance.

Plenty of serious production projects are doing exactly that. Are you using GPT-6 Astra, or something older?

Astra does exactly the same sort of things that Sol or any of the previous agents do. They duplicate code, overengineer, miss the point, etc.

I was very optimistic about it when it was announced and saw all the demos, but a week later I find it only marginally better (and in some cases worse) than before.

Post reply on HN