Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measuring the sloppiness of code

earendil.com

Post reply on HN
Print view View original

121–130 of 247 posts

Re: Measuring the sloppiness of code

#121

Post by datsci_est_2015 »

wang_li wrote: post #48
_pdp_ wrote: post #16

It will be solved when there is no more code left to write. Code is an abstract concept that is not bound to the physical world and I imagine that future will have some much more of it that it is difficult to comprehend. Everything will be code and more code will be written than ever before. Code will never going to be solved. The question is how much humans will be involved and I think the evidence is that perhaps j…

Code and software and applications are an intermediate stage. The final stage is an AI/LLM that just does the thing that is needed without any code being written, there are no applications or programs, just an AI that does everything.

So how will separate systems communicate with each other? Or are there no separate systems? That’s the digital singularity I suppose.

Re: Measuring the sloppiness of code

#122

Post by user43928 »

einrealist wrote: post #106
danbruc wrote: post #19

Coding is not solved, correctness is not a feature, it is the bare minimum. If your code does not do what it is supposed to do, you could as well have no code at all. Efficiency, security, maintainability, reliability, readability, understandability, extensibility, maintainability, observability, portability, ... this is what high quality coding is about, not that it works, that is a given. And in my experience curre…

And there is another problem: LLMs generating too much code, code that is doing more than was asked. And that cannot be fixed by tests. Usually, we create tests for wanted behavior and expected exceptions. But we don't create tests for undesired behavior.

Have you worked with Opus 5?

Its documentation about what the code does not do could fill whole books.

UI copy being full of slop explaining what the software does not do is another problem.

I am not convinced that a lack of negative test cases is an issue.

I do agree it generates too much code most of the time.

Re: Measuring the sloppiness of code

#123

Post by ACCount37 »

bla3 wrote: post #104
Supermancho wrote: post #66

Earlier quoted context omitted.

> Efficiency, security, maintainability, reliability, readability, understandability, extensibility, maintainability, observability, portability My experience is a little different. For higher abstraction languages the output is largely acceptable in my work. I always consider that LLMs don't know what I don't tell them and they have limited context to work from. Coding issues I often identify: * Efficiency. Marginal…

This sounds roughly right to me, except for "maintainability". In my experience, agents really don't like deleting code unless you explicitly ask for it. If you're not careful, you end up with new better implementations of things but with the old implementation still around in perpetuity. Humans do this too of course.

Humans had to get it drilled into them that "+12 -440" is a damn good line stat, and that keeping around dead code is bad, especially in the age of version control.

Not too surprised that LLMs also don't "get it" by default?

Re: Measuring the sloppiness of code

#124

Post by dherman »

Really glad to see folks looking into quantitative approaches to give agents feedback on code quality. This post looks like a good start!

My main feedback for the authors would be, the most important problems for sloppiness are global properties, not local ones. In my experience an agent, like a human, has finite capacity for its attention, but if it runs into local sloppiness that gets in its way, it can fix it on a by-need basis. The technical debt issues that matter are usually global issues that aren't so easy to fix: they require global analysis and global refactoring.

I don't know the answer, but I think we're going to need ways to measure architectural properties, like separation of concerns, clear architectural layering, well-defined interfaces, etc.

Re: Measuring the sloppiness of code

#125

Post by LaffertyDev »

anilgulecha wrote: post #117

> Qwen2.5-Coder-3B Basing it's findings of LLM as judge on this model, and then proceeding to ignore it. This article can be safely ignored as well. LLM as judge in harness evals is the way to go, for any of your custom needs. Design the eval well.

I don't know where you pulled that from, its not in the article.

Re: Measuring the sloppiness of code

#126

Post by hax0ron3 »

shimman wrote: post #20
scronkfinkle wrote: post #12

There is some sense of rose-tinted glasses of pre-LLM coding. A lot of human written code, particularly at the enterprise level, was of low quality well before AI automated it.

This was always due to pressures by management and the company environment, not the workers themselves. It's hard to blame the people writing code when they have to deal with nontechnical leadership that wants to have a feature factory or never given appropriate resources to solve problems. Blaming workers is always an excuse by poor management.

The pressures from management and the company environment are not always a bad thing. It really depends on whether the pressures are coming from a logical business perspective or whether they are just coming from stupidity or ignorance. In a business environment, taking a long time to ship great code can mean that the company goes out of business, and then the software developers have a lot of great code and no income.

Re: Measuring the sloppiness of code

#127

Post by pydry »

baridbelmedar wrote: post #35
softwaredoug wrote: post #22

Earlier quoted context omitted.

I don't doubt that. But humans still need to be responsible for understanding what they're shipping. And IMO you get your best understanding by actually writing some code. Even if you don't actually ship what you wrote.

Let’s not romanticize it too much... A lot of enterprise systems are built by developers copying an old AbstractBeanFactoryFactory from a 2011 stack overflow thread without really understanding it :)

Nobody is. It's the AI slop which is supposed to replace this shit which barely worked with equally shit shit which doesnt work which people are romanticizing.

Most of the human written code was slop, but the really fundamental and successful stuff we relied upon and which we didnt want to throw away? yeah, not so much. most of that was actually really good.

those EJB monstrosities were routinely swapped out by some saas written in python by somebody who did it properly and werent responsible for a lot of late and over budget projects which barely worked or didnt work.

Re: Measuring the sloppiness of code

#128

Post by softwarewright »

danielvaughn wrote: post #68
danbruc wrote: post #19

Coding is not solved, correctness is not a feature, it is the bare minimum. If your code does not do what it is supposed to do, you could as well have no code at all. Efficiency, security, maintainability, reliability, readability, understandability, extensibility, maintainability, observability, portability, ... this is what high quality coding is about, not that it works, that is a given. And in my experience curre…

Yes. I use these models day in and day out, on all sorts of tasks. I cannot believe I hear people say that coding is solved.

me too; and my coding agents are slowed down (from developing features) because I require them to refactor the code to be more readable; my code metrics tests force AIs to leverage functional programming and design patterns).

Yes agents can produce code that compiles and runs, but I had to add tools to keep them on track, document their work, follow a process, check their outputs. I also use other AIs to generate developer documentation and review code.

It is like managing a bunch of idiot savant eager-to-please interns, except unlike interns, coding agents do not (yet) learn and improve on their own.

Re: Measuring the sloppiness of code

#129

Post by margalabargala »

heaney-555 wrote: post #92
dnikolovv wrote: post #81

Earlier quoted context omitted.

Right. It feels like we're living in a parallel world or something. Can agents code? Sure. Can you let them code on their own for a serious production project? Not a chance.

Plenty of serious production projects are doing exactly that. Are you using GPT-6 Astra, or something older?

You and the person you are replying to are talking about different things.

Agents cannot be given a high level goal and then left unsupervised, for hours, without making some dumb decisions.

Re: Measuring the sloppiness of code

#130

Post by hax0ron3 »

softwaredoug wrote: post #8

Coding might be "solved" but coding still is the best way to build your own mental model of the solution space. Which is more important to you: Velocity to a solution? Or velocity to understanding?

I code to make money, and the kind of stuff I work on doesn't kill people or lose massive amounts of money if it has bugs, so to me velocity to a solution is much more important than velocity to understanding.
Post reply on HN