>
Are you doing this for client-facing production code?What's "this"? We're talking about multiple things here.
But presumably yes: I'm working through agents, and all I do is read and prompt. I don't manually write code except Markdown and occasionally Nix code because it's sufficiently high-level, and I rarely read the Rust code. The Nix code is based on templates I wrote by hand, and the Rust code is based on crate selection and some loose principles I've derived from years of programming Rust by hand.
> It seems you believe painting over APIs with some amount of documentation will guarantee the implementation is correct and well designed. Am I misreading that?
I don't believe that, you're misreading.
I'm saying:
- Making and documenting APIs well is just as necessary as before I wrote to files.
- Correctness is achieved through strong types and extensive testing of all kinds.
>
Do you have tests? How do you know they’re correct if you don’t read the code? Moreover, how do know if their coverage is sufficient if you don’t read the code?There are different tests, and I know they work for different reasons.
I'll list a few kinds that I've found actually valuable in the recent past:
Manual user tests: While expensive, they're most truthful in revealing bugs and confirming if they still exist. We do this by dogfooding our own product, and by running manual user tests before major deployments. User tests help uncover problems like delays that don't appear as errors, but can be major UX bugs (like waiting ten seconds for the effect of a button press).
TDD regression tests: A user experiences an error in the application, this gets reported, the agent analyses the code and writes one or more unit tests to confirm the problem, and writes a fix to see the light go green. I know this test works because it fails first. There's an odd chance the agent either uncovered a different bug, or wrote a test that fails synthetically. I haven't tried the latter, but I've tried many times that it wrote "a test" that passes but the app still crashes. So TDD order matters. And I've tried that it found several more bugs by analysing the code. (Most recent example: https://github.com/atuinsh/atuin/issues/3603 ) Another example of a TDD regression test is: Some async code prematurely exited, I scanned for the class of bugs and found one other case and fixed it, TDD. There's a very simple consideration to make that could be expressed as a linter rule, but is more easily expressed as a single sentence in CLAUDE.md, since it will appear again (it's a pattern I and former colleagues have made several times).
End-to-end integration tests: Wire up all parts of the system in either a semi-authentic (say, docker compose-based with some mocks) or in a fully authentic staging environment (say, Kubernetes); I do both, and it catches both bugs and performance regressions by simulating real workloads. I haven't automated the performance regression testing, only workload simulation, and then I assess the metrics myself.
Algorithmic pareto-optimality: I have an algorithm that can be measured in number of steps by measure of API calls, and number of successful calls; correctness would be "the algorithm gives the same result", and pareto-optimality would be "it doesn't use more steps in the refactored version". Computing time isn't really the cost here, the cumulative size of API payloads are. The API part can actually be mocked, or "made golden" (I've just now learned that it's called a "characterization test", https://en.wikipedia.org/wiki/Characterization_test, but I always called it a golden test from having read that.)
"Full coverage unit tests": I don't actually know how valuable these are. But for example, I went with "Find all functions that return Result or Option and test all their edge cases." Some ~87 functions, some ~150 tests. No failures. But I'd be surprised, because the type of mistake these tests generally fix for me, as a human, are typos and copy-paste errors, and big models generally don't make those.
> You say you still read code when code is « algorithmically hard ». How do you define that?
I don't have a good definition. I recently deployed a service that runs something my colleagues call "the algorithm". It's some progressive audio transcription thing. My colleagues feel uncomfortable going live with something that appears to work that nobody read, so I read what was made.
I think the value of reading algorithmically hard code is not to assess correctness, but to regain cognitive debt in case something breaks and you don't know how it's made. I've consulted for people whose main programmer left and they were stuck with a machine they couldn't comprehend, start, or start to comprehend.
Point being: Code review isn't ideal for catching hard logical bugs, it's for understanding code so that you may eventually understand hard logical bugs when they surface.
> Do you refactor code or is that not needed anymore?
The need for software architecting and refactoring are exactly the same.
Only maybe more and sooner because the amount of code increases.
> I don’t understand your answer to the question of why anyone would publish a library if no one reads code anymore. « For the exact same reasons as before ». How so? This isn’t making sense.
I don't understand why it doesn't make sense.
Agents will read code. Humans do occasionally read some code.
But the value of a well-polished, tested library is not just for humans to read.
A good library will make your code better, because vibing on the spot delivers less guarantee that it's good. You'll miss some effort spent on proper data modelling and edge case and error handling. Sometimes that's the right call.
Say you build a parser for a small language. Using a parser combinator library, you get nice error messages and some very non-trivial thoughts on resuming parsing from a failed state. A vibed parser will give you neither of those.
There are some points related to supply-chain minimization.
But it's the exact same tradeoff, where the choice-points are pushed around.