61–70 of 157 posts
Human: well-scoped argument that does just enough to get the job done with minimal risk.
AI: Extremely clever and correct legal argument that almost any lawyer would have said not to file (at least as written). It tries to burn the world and seriously risks pissing off the judge.
You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…
It can't be written down as code, that's the point. I am more familiar with taste in coding and it can at best be described—that the resulting code is too subtly different from something else in the codebase, that you're masking a different bug, that you're not following what the code tells you. The good part is that while this cannot be unit tested, you can write documentation and code comments about it that tell pe…
Earlier quoted context omitted.
This is, in short, the big current problem with AI. LLMs are built for scale so they've given up on the kind of online learning / "long term memory" processes that would individualize them. The LLM is permanently locked to being a really cracked engineer on their first day at your company, looking at your codebase for the first time. You can scaffold a bit with .md files, but at the moment they lack the ability to do…
100%. The problem with them isn't making sure they're doing the right thing, it's making sure they're not making bad assumptions. IMHO this is where code review goes until we fix the individualized model thing: you need to review the decisions the agent made, where you didn't steer. Most will be right. A few will be disastrously wrong. But decision-by-decision is a lot less to review than line-by-line of code.
Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always about plain algorithms and data structures.
yup and I find it weird that people still remain so defensive of the Church of TDD even against empirical studies that show its limited benefits https://arxiv.org/abs/2602.07900
Substitute static typing for TDD in your comment, and it will remain equally valid statement.
Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always about plain algorithms and data structures.
The biggest flaw I've seen with TDD is the fact that correctness does not compose upward. Every time two units come into contact, you've got an entirely new kind of unit. The tests from constituents do not cover emergent properties of the new things. You will repeat this same exercise the entire way up to the top, and the moment you come into contact with the customer (they want to change everything), the house of ca…
Or to put it differently: a test is an assertion that no matter what, for all time this should never change again. Even if customer requirements change in the future they won't change in such a way as to break this test (this isn't always true, but you should believe it is true).
A test is most valuable when it alerts you to a real problem when it fails. If the test fails but there isn't a real problem (either because customer requirements have changed, or it is flaky) it was needless cost to investigate it. If the test passes that gives some hope of correctness, but you can never be sure it is really correct vs a bug in the test (even if you use TDD and so the test failed when you wrote it that doesn't mean a refactoring since didn't make this an always pass test).
Part of the problem is if I tell you to write sort() or your new toy language's list type you have an intuitive idea of what it should look like and probably will get them right the first time (other than bugs you want the tests so you catch). These should have tiny micro tests. These things also are really easy to use as examples of how to do TDD - which they are, but they are not representative: this type of code is generally in your standard library already and you are not writing it.
Instead you are writing code that isn't well defined with lots of industry experience. It is not clear what the exact interface should be (or more likely it is clear customer requirements will change but you don't know how yet). You have no idea what the best implementation is. You don't know if this will be used in this one place, or if it will become a useful key part that many future projects depend on. You have to make guesses.
You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…
You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…
Is there an issue of taste when generating images with AI ? or can we relatively rapidly train people to generate beautiful images with decent amount of variety ?
Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always about plain algorithms and data structures.
The biggest flaw I've seen with TDD is the fact that correctness does not compose upward. Every time two units come into contact, you've got an entirely new kind of unit. The tests from constituents do not cover emergent properties of the new things. You will repeat this same exercise the entire way up to the top, and the moment you come into contact with the customer (they want to change everything), the house of ca…
You can't unit test for taste if you haven't written down what you mean by taste. If you can externalize it, then you can. Follow this line of thinking, and the AI-friendly answer is easy: we just have to externalize everything we know, so Claude can implement what I want. Except that I can't fully externalize myself. Debugging a system takes more resources than running the system. If I could write down everything I…
I had this experience doing a port from Big Query to Postgres using Opus. I had unit tests to guarantee parity with the original code, and Opus insisted on building this bespoke query builder (e.g. `def _where(very_complicated_params)`) on top of sqlglot.
Even with the original code being straightforward and legible and repeated instructions to match, I had to fight with it to get close.
In the end, I ended up doing things the "old fashion way" where I copied chunks code into Claude proper and gave explicit instructions for each piece.
I clearly had externalized the requirements, and yet that wasn't sufficient. The only way to unit test further would be to use an AST to evaluate the output against metrics I couldn't even encode.
Exactly one of the reasons I never went down with all the TDD dogma of only writing code to fix broken tests. There is a reason conference talks are always about plain algorithms and data structures.
The biggest flaw I've seen with TDD is the fact that correctness does not compose upward. Every time two units come into contact, you've got an entirely new kind of unit. The tests from constituents do not cover emergent properties of the new things. You will repeat this same exercise the entire way up to the top, and the moment you come into contact with the customer (they want to change everything), the house of ca…
You would have the same problem if you wrote tests like that after the code.
TDD has no opinion about the level at which you wrote your test, it just assumes it's the correct one.
This is the number one biggest misconception about TDD which I keep seeing repeated on hacker news.