Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your job is to deliver code you have proven to work

simonwillison.net

Post reply on HN
Print view View original

121–130 of 699 posts

Re: Your job is to deliver code you have proven to work

#121

Post by 0x500x79 »

enraged_camel wrote: post #30

>> As software engineers we don’t just crank out code—in fact these days you could argue that’s what the LLMs are for. We need to deliver code that works—and we need to include proof that it works as well. I would go a step further: we need to deliver code that belongs . This means following existing patterns and conventions in the codebase. Without explicit instruction, LLMs are really bad at this, and it's one of t…

Agree, maintainability, security, standards, all of these are important to follow and there are usually reasons for these things existing.

I also see AI coding tools violate "Chesterton's Fence" (and the pre-Chesterton's Fence, not sure what that is called, the idea being that code is necessary otherwise it shouldn't be in the source).

Re: Your job is to deliver code you have proven to work

#122

Post by mellosouls »

Thing is, this has always been the case. One of the problems with LLM-assisted coding is the idea that just because we're in a new era (we certainly are), the old rules can all be discarded.

The title doesn't go far enough - slop (AI or otherwise) can work and pass all the tests, and still be slop.

Re: Your job is to deliver code you have proven to work

#123

Post by pydry »

snowstormsun wrote: post #119
reedf1 wrote: post #106

Earlier quoted context omitted.

it's even worse than that! non- devs are doing it as well

it's even worse than that! bots are doing it as well

Hopefully once this AI nonsense blows over they'll reach the same realisation they did after the mid 2000s outsourcing craze: that actually you gotta pay for good engineering talent.

Re: Your job is to deliver code you have proven to work

#124

Post by jennyholzer2 »

troyvit wrote: post #78
wizzwizz4 wrote: post #8

Earlier quoted context omitted.

It's not a new phenomenon. Time was, people would copy-paste from blog posts with the same effect.

I used to do that in simpler days. I'd add a link to where I copied it from so we could reference it if there were problems. This was for relatively small projects with just a few people.

> I'd add a link to where I copied it from

LLMs can't do this.

Your code is unambiguously better than any LLM code if you can comment a link to the stackoverflow post you copied it from.

Re: Your job is to deliver code you have proven to work

#125

Post by pjc50 »

tossandthrow wrote: post #85
fnands wrote: post #21

Earlier quoted context omitted.

A friend of mine is working for a small-ish startup (11 people) and he gets to work and sees the CTO push 10k loc changes straight to main at 3 am. Probs fine when you are still in the exploration phase of a startup, scary once you get to some kind of stability

The cto is ultimately responsible for the outcome and will be there at 4am to fix stuff.

Yes .. and no. Someone who does this will definitely make the staff clean up after them.

Re: Your job is to deliver code you have proven to work

#126

Post by vernrVingingIt »

reedf1 wrote: post #106
endorphine wrote: post #104

> there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. It's even worse than that: non -junior devs are doing it as well.

it's even worse than that! non- devs are doing it as well

[flagged]

Re: Your job is to deliver code you have proven to work

#127

Post by esafak »

reedf1 wrote: post #106
endorphine wrote: post #104

> there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. It's even worse than that: non -junior devs are doing it as well.

it's even worse than that! non- devs are doing it as well

That's what democratization looks like. And the new participants are happy!

Re: Your job is to deliver code you have proven to work

#128

Post by simonw »

mellosouls wrote: post #122

Thing is, this has always been the case. One of the problems with LLM-assisted coding is the idea that just because we're in a new era (we certainly are), the old rules can all be discarded. The title doesn't go far enough - slop (AI or otherwise) can work and pass all the tests, and still be slop.

The difference is that if it works and passes the tests I don't feel like it's a total waste of my time to look at the PR and tell you why it's still slop.

If it doesn't even work you're absolutely wasting my time with it.

Re: Your job is to deliver code you have proven to work

#129

Post by pjc50 »

robryk wrote: post #27
simonw wrote: post #12

Earlier quoted context omitted.

Yeah that's fair, the manual testing doesn't have to sequentially go first - but it does have to get done. I've lost count of the number of times I've skipped it because the automated test passed and then found there was some dumb but obvious bug that I missed, instantly exposed when I actually exercised the feature myself.

Would automated tests that produce a transcript of what they've done allow perusing that transcript to substitute for manual testing?

That sounds harder?

There's a lot of pedantry here trying to argue that there exists some feature which doesn't need to be "manually" tested, and I think the definition of "manual" can be pushed around a lot. Is running a program that prints "OK" a manual test or not? Is running the program and seeing that it now outputs "grue" rather than "bleen" manual? Does verifying the arithmetic against an Excel spreadsheet count?

There are programs that almost can't be manual, and programs that almost have to be manual. I remember when working on PIN pad integration we looked into getting a robot to push the buttons on the pad - for security reasons there's no way of injecting input automatically.

What really matters is getting as close to a realistic end user scenario as possible.

Re: Your job is to deliver code you have proven to work

#130

Post by 9rx »

enraged_camel wrote: post #30

>> As software engineers we don’t just crank out code—in fact these days you could argue that’s what the LLMs are for. We need to deliver code that works—and we need to include proof that it works as well. I would go a step further: we need to deliver code that belongs . This means following existing patterns and conventions in the codebase. Without explicit instruction, LLMs are really bad at this, and it's one of t…

> Without explicit instruction, LLMs are really bad at this

They used to be. They have become quite good at it, even without instruction. Impressively so.

But it does require that the humans who laid the foundation also followed consistent patterns and conventions. If there is deviation to be found, the LLM will see it and be forced to choose which direction to go, and that's when things quickly fall off the rails. LLMs are not (yet) good at that, and maybe never can be as not even the humans were able to get it right.

Garbage in, garbage out, as they say.

Post reply on HN