Live data from Hacker News

Your job is to deliver code you have proven to work

simonwillison.net

191–200 of 699 posts

Re: Your job is to deliver code you have proven to work

#191

I don't know if there's a word for this but this reads to me as like, software virtue signaling or software patronizing. It's bizarre to me to tell an engineer what their job is as a matter of fact and to claim a particular usage of a tool as mandated (a tool that no one really asked for, mind you), leveraging duty of all things. I guess to me, it's either the case that LLMs are just another tool, in which case the a…

LLMs are just another tool, but they're disruptive enough that existing best practices need to be either updated or re-explained.

A lot of people using LLMs seem not to have understood that you can't expect them to write code that works without testing it first!

If that wasn't clearly a problem I wouldn't have felt the need to write this.

Re: Your job is to deliver code you have proven to work

#192
post #21
post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

A friend of mine is working for a small-ish startup (11 people) and he gets to work and sees the CTO push 10k loc changes straight to main at 3 am. Probs fine when you are still in the exploration phase of a startup, scary once you get to some kind of stability

I'd go mental if I was a SWE having to mop that up later

Re: Your job is to deliver code you have proven to work

#194
post #64

Earlier quoted context omitted.

LLMs are tools that make mediocre devs 100x more "productive" and good devs 2x more productive

From my vantage I would argue LLMs make good devs around 0.65x more productive

I think they make good devs 2x more productive for the first month, which then slowly declines as that good dev spends less time actually writing and understanding and debugging code until it falls well below the 1x mark. It’s basically a high interest loan people take against their own skills. For some people that loan might be worth it. Maybe they’re trying to change their role in an organization and need the boost to start taking up new responsibilities they want to own. I think it’s temporary though. The slow shift into “skim mode”, where the authors just don’t quite put that same amount of effort into understanding what’s being churned out. I dunno, that’s just what I’ve seen.

Re: Your job is to deliver code you have proven to work

#195
post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

Not at all. Submitting untested PRs is a wildly outside of my experience. Having tests written to cover your code is a pre-requisite for having your PR reviewed on our team. "Does it work" aka passing manual testing, is literally the bare minimum before submitting a PR

Re: Your job is to deliver code you have proven to work

#196

> there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. It's even worse than that: non -junior devs are doing it as well.

>It's even worse than that: non-junior devs are doing it as well. This might be unpopular, but that is seeming more like an opportunity if we want to continue allowing AI to generate code. One of the annoying things engineers have to deal with is stopping whatever they're doing and doing a review. Obviously this gets worse if more total code is being produced. We could eliminate that interruption by having someone do…

As I read once: all that little stuff that feels like it stops you from doing your job is your job.

Re: Your job is to deliver code you have proven to work

#197

Actually it's more specific than that. A company pays you not just to "write code", not just to "write code that works", but to write code that works in the real world . Not on your laptop. Not in CI tests. Not on some staging environment. But in the real world . It may work fine in a theoretical environment, but deflate like a popped balloon in production. This code has no value to the business; they don't pay you t…

It gets interesting when a company assigns 2 story points to a task that requires 6 minimum. No time for writing tests, barely any time to perform code reviews and QA. Also, next year the company tells you since we have AI now, all tickets must be done 2 times quicker.

Who popped this balloon? I know I need to change my employer, but it's not so easy. And I'm not sure another employer is going to be any better.

Re: Your job is to deliver code you have proven to work

#198

For what it's worth, writing good PRs applies in more cases than just AI generated contributions. In my PR descriptions, I usually start by describing how things currently work, then a summary of what needs to change, and why. Then I go on to describe what exactly is changing with the PR. This high level summary serves to educate the reviewer, and acts as a historical record in the git log for the benefit of those wh…

> I try to include at least a screenshot

This is ~mandatory for me. Even if what I am working on is non-visual. I will take a screenshot of a new type in my IDE and put a red box around it. This conveys the focus of my attention and other important aspects of the work effort.

Re: Your job is to deliver code you have proven to work

#199
it's very similar to the verification engineering problem i wrote about on HN last week. AI is as good as we can prove their work is genuine. and we need humans in the loop to fill in the gaps between autonomous systems and ultimately be held accountable by human laws. it's kind of sad but the reality we are facing

Re: Your job is to deliver code you have proven to work

#200
post #76
post #6

there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. Is anyone else seeing this in their orgs? I'm not...

Voila: https://github.com/WireGuard/wireguard-android/pull/82 https://github.com/WireGuard/wireguard-android/pull/80 In that first one, the double pasted AI retort in the last comment is pretty wild. In both of these, look at the actual "files changed" tab for the wtf.

Yeah this guys comment here is spot on: https://github.com/WireGuard/wireguard-android/pull/80#issue...

I recently reviewed a PR that I suspect is AI generated. It added a function that doesn't appear to be called from anywhere.

It's shit because AI is absolutely not on the level of a good developer yet. So it changes the expectation. If a PR is not AI generated then there is a reasonable expectation that a vaguely competent human has actually thought about it. If it's AI generated then the expectation is that they didn't really think about it at all and are just hoping the AI got it right (which it very often doesn't). It's rude because you're essentially pawning off work that the author should have done to the reviewer.

Obviously not everyone dumps raw AI generated code straight into a PR, so I don't have any problem with using AI in general. But if I can tell that your code is AI generated (as you easily can in the cases you linked), then you've definitely done it wrong.

Post reply on HN