Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your job is to deliver code you have proven to work

simonwillison.net

Post reply on HN
Print view View original

21–30 of 699 posts

Re: Your job is to deliver code you have proven to work

#21

Post by fnands »

dfxm12 wrote: post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

A friend of mine is working for a small-ish startup (11 people) and he gets to work and sees the CTO push 10k loc changes straight to main at 3 am.

Probs fine when you are still in the exploration phase of a startup, scary once you get to some kind of stability

Re: Your job is to deliver code you have proven to work

#22

Post by dfxm12 »

briliantbrandon wrote: post #18
dfxm12 wrote: post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

I'm seeing a little bit of this. However, I will add that the primary culprits are engineers that were submitting low quality PRs before they had access to LLMs, they can just submit them faster now.

What's the ratio of people who things the right way vs not? I mean, is it a matter of giving them feedback to remind them what a "quality PR" is? Does that help?

Re: Your job is to deliver code you have proven to work

#23

Post by throwuxiytayq »

zkmon wrote: post #14

How about letting LLMs maintain a vast number of product versions all available at the same, which receive multiple versions of untested versions of the same patch, from LLMs, and then let the models elect a version of the software based on probabilistic or gradient methods? This elected version could change for different assessments. No human touches or looks at the code! Just a wild thought, nothing serious.

Talk is cheap. Show me the proompt.

Re: Your job is to deliver code you have proven to work

#24

Post by evilduck »

wizzwizz4 wrote: post #8
dfxm12 wrote: post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

It's not a new phenomenon. Time was, people would copy-paste from blog posts with the same effect.

I would bet in most organizations you can find a copy-pasted version of the top SO answer for email regex in their language of choice, and if you chase down the original commit author they couldn't explain how it works.

Re: Your job is to deliver code you have proven to work

#26

Post by robgibbons »

For what it's worth, writing good PRs applies in more cases than just AI generated contributions. In my PR descriptions, I usually start by describing how things currently work, then a summary of what needs to change, and why. Then I go on to describe what exactly is changing with the PR. This high level summary serves to educate the reviewer, and acts as a historical record in the git log for the benefit of those who come after you.

From there, I include explicit steps for how to test, including manual testing, and unit test/E2E test commands. If it's something visual, I try to include at least a screenshot, or sometimes even a brief screen capture demonstrating the feature.

Really go out of your way to make the reviewer's life easier. One benefit of doing all of this is that in most cases, the reviewer won't need to reach out to ask simple questions. This also helps to enable more asynchronous workflows, or distributed teams in different time zones.

Re: Your job is to deliver code you have proven to work

#27

Post by robryk »

simonw wrote: post #12
allcentury wrote: post #4

Manual testing as the first step… not very productive imo. Outside in testing is great but I typically do automated outside in testing and only manual at the end. The loop process of testing needs to be repeatable and fast, manual is too slow

Yeah that's fair, the manual testing doesn't have to sequentially go first - but it does have to get done. I've lost count of the number of times I've skipped it because the automated test passed and then found there was some dumb but obvious bug that I missed, instantly exposed when I actually exercised the feature myself.

Would automated tests that produce a transcript of what they've done allow perusing that transcript to substitute for manual testing?

Re: Your job is to deliver code you have proven to work

#28

Post by rkomorn »

throwuxiytayq wrote: post #23
zkmon wrote: post #14

How about letting LLMs maintain a vast number of product versions all available at the same, which receive multiple versions of untested versions of the same patch, from LLMs, and then let the models elect a version of the software based on probabilistic or gradient methods? This elected version could change for different assessments. No human touches or looks at the code! Just a wild thought, nothing serious.

Talk is cheap. Show me the proompt.

Had to search whether "proompt" was a new meme misspelling.

New to me, but I'm on board.

Re: Your job is to deliver code you have proven to work

#29

Post by toomuchtodo »

robgibbons wrote: post #26

For what it's worth, writing good PRs applies in more cases than just AI generated contributions. In my PR descriptions, I usually start by describing how things currently work, then a summary of what needs to change, and why. Then I go on to describe what exactly is changing with the PR. This high level summary serves to educate the reviewer, and acts as a historical record in the git log for the benefit of those wh…

This is how PRs should be, but rarely are (in my experience as a reviewer, ymmv, n=1). Keep on keepin' on.

Re: Your job is to deliver code you have proven to work

#30

Post by enraged_camel »

>> As software engineers we don’t just crank out code—in fact these days you could argue that’s what the LLMs are for. We need to deliver code that works—and we need to include proof that it works as well.

I would go a step further: we need to deliver code that belongs. This means following existing patterns and conventions in the codebase. Without explicit instruction, LLMs are really bad at this, and it's one of the things that make it incredibly obvious to reviews that a given piece of code has been generated by AI.

Post reply on HN