Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your job is to deliver code you have proven to work

simonwillison.net

Post reply on HN
Print view View original

301–310 of 699 posts

Re: Your job is to deliver code you have proven to work

#301

Post by tyleo »

endorphine wrote: post #104

> there’s one depressing anecdote that I keep on seeing: the junior engineer, empowered by some class of LLM tool, who deposits giant, untested PRs on their coworkers—or open source maintainers—and expects the “code review” process to handle the rest. It's even worse than that: non -junior devs are doing it as well.

There’s folks who perform like juniors but have just been in the business long enough to be promoted.

Title only loosely tracks skill level and with AI, that may become even more true.

Re: Your job is to deliver code you have proven to work

#302

Post by IMTDb »

maerF0x0 wrote: post #286

> The first is manual testing. If you haven’t seen the code do the right thing yourself, that code doesn’t work. If it does turn out to work, that’s honestly just pure chance. Depending on exactly what the author meant here, I disagree. Our first and default tool should be some form of lightweight automated testing. It's explicit (serves a form of spec and docs how to use the software), it's repeatable (manual testin…

> Our first and default tool should be some form of lightweight automated testing

Manual verification isn't about skipping tests, it's about validating what to test in the first place.

You need to see the code work before you know what "working" even means. Does the screen render correctly? Does the API return sensible data? Does the flow make sense to users? Automated tests can only check what you tell them to check. If you haven't verified the behavior yourself first, you're just encoding your assumptions into test cases.

I'd take "no tests, but I verified it works end-to-end" over "full test coverage, but never checked if it solves the actual problem" every time. The first developer is focused on outcomes. The second is checking boxes.

Tests are crucial: they preserve known-good behavior but you have to establish what "good" looks like first, and that requires human judgment. Automate the verification, not the discovery. So our first and default tool remains manual verification

Re: Your job is to deliver code you have proven to work

#303

Post by jjmarr »

kridsdale3 wrote: post #244
vernrVingingIt wrote: post #174

Earlier quoted context omitted.

Try almost 30 years in electrical engineering. I know exactly what those layers of abstraction are used for. Why so many? Jobs making layers of abstraction. But all of them are dev friendly means of modeling memory states for the CPU to watch and transform just so. They can all be compressed into a generic and generalized set of mathematical functions ridding ourselves of the various parser rules to manage each bespo…

Okay. Go write an operating system and suite of apps with global memory and no protections. Why are we wasting so much time on abstractions like processes and objects? Just let let everyone read and write from the giant turing machine.

Embedded systems that EEs code for are like this. I have to explicitly opt into processes and objects in Keil RTX. I also get to control memory layout.

Abstraction layers are terrible when you need to understand 100% of the code at all times. Doesn't mean they're not useful.

Heck, the language for just implementing mathematical rules about system behaviour into code exists. It's called Matlab Simulink.

Re: Your job is to deliver code you have proven to work

#305

Post by tech-ninja »

trevor-e wrote: post #159

> Your job is to deliver code you have proven to work. Strong disagree here, your job is to deliver solutions that help the business solve a problem. In _most_ cases that means delivering code that you should be able to confidently prove satisfies the requirements like the OP mentioned, but I think this is an important nitpick distinction I didn't understand until later on in my career.

> In _most_ cases that means delivering code that you should be able to confidently prove satisfies the requirements like the OP mentioned

That is an insane distinction that you are trying to do there. In which cases delivering code that doesn't satisfy the requirements would solve a business problem?

Re: Your job is to deliver code you have proven to work

#306

Post by simonw »

WhyOhWhyQ wrote: post #263

Isn't this in contradiction to your blog post from yesterday though? It's impossible to prove a complex project made in 4.5 hours works. It might have passed 9000 tests, but surely there are always going to be edge cases. I personally wouldn't be comfortable claiming I've proved it works and saying the job is done even, if the LLM did the whole thing and all existing tests passed, until I played with it for several m…

That's why I don't consider my blog post from yesterday to be production quality code. I'd need to invest a lot more work in reviewing it before I staked my reputation on it

Re: Your job is to deliver code you have proven to work

#307

Post by Nizoss »

simonw wrote: post #179
rglover wrote: post #151

"Slow the f*ck down." - Oliver Reichenstein [1] This only happens because the software industry has fallen into the Religion of Speed . I see it constantly: justified corner-cutting, rushing shit out the door, and always loading up another feature/project/whatever with absolutely zero self-awareness. AI is just an amplifier for bad behavior that was already causing chaos. What's not being said here but should be: dis…

A few years ago I embraced automated tests and comprehensive documentation for even my smallest personal projects because I found that they sped me up . https://simonwillison.net/2022/Nov/26/productivity/

I would love to hear your thoughts on TDD-Guard. An open source plugin I created to enfore Test-Driven Development practices on agents:

https://github.com/nizos/tdd-guard

Re: Your job is to deliver code you have proven to work

#308

Post by dekhn »

Prove is a strong word. There are few cases in real-world programming where you can prove anything.

I prefer to make this probabilistic: use testing to reduce the probability that your code isn't correct, for the situations in which it is expected to be deployed. In this sense, coding and testing is much like doing experimental physics: we never really prove a theory or disprove it, we just invalidate clearly wrong ones.

Re: Your job is to deliver code you have proven to work

#309

Post by vladsh »

We should get back to the basic definition of the engineering job. An engineer understands requirements, translates them into logical flows that can be automated, communicates tradeoffs across the organization, and makes tradeoff calls on maintainability, extensibility, readability, and security. Most importantly, they’re accountable for the outcome, because many tradeoffs only reveal their cost once they hit reality

None of this is covered by code generation, nor by juniors submitting random PRs. Those are symptoms of juniors (not only) missing fundamentals. When we forget what the job actually is, we create misalignment with junior engineers and end up with weird ideas like "spec-driven development"

If anything, coding agents are a wake-up call that clarify what engineering profession is really about

Re: Your job is to deliver code you have proven to work

#310

Post by codeviking »

maerF0x0 wrote: post #286

> The first is manual testing. If you haven’t seen the code do the right thing yourself, that code doesn’t work. If it does turn out to work, that’s honestly just pure chance. Depending on exactly what the author meant here, I disagree. Our first and default tool should be some form of lightweight automated testing. It's explicit (serves a form of spec and docs how to use the software), it's repeatable (manual testin…

I'm a big fan of lightweight, automated tests. Despite that, I still default to manual verification. Usually I do both.

Automated tests omit a certain type of feedback that I think remains important to the development loop. Automation doesn't care about a poor UX; it only verifies what you tell it to.

For instance, I regularly contribute to a CLI that's widely used at $WORK. I can easily write tests to verify the I/O of a command I'm working on that assert correctness. Yet if I actually try to use the command I'm changing, usually as part of verifying my changes, I tend to discover usability issues that make the program more pleasant to use and the tests would happily ignore.

Also, there's certainly cases where automation isn't worth the cost. Maybe because the resulting tests are complex, or brittle. I've often found UI tests to lie in this category (but maybe I'm doing them wrong).

Because of these things I think manual testing is the right default. Automated tests should also exist; but manual tests should _always_ be part of the process.

Post reply on HN