Earlier quoted context omitted.
> But tests have to be correct. From my experience, ChatGPT-3 has been pretty good at exercising all meaningful branches (I look at code coverage results, I don't care for percentages at all), in the least amount of tests, on the first go. I definitely have to modify each test quite a bit, because it frequently hallucinates API calls that don't exist, but the code that it produces is an incredible blueprint. And I ha…
If you write a unit test for each branch in your method and just put in the current behavior as the expectation, all you've done is created a test that says "the method does what it currently does."
It isn't that stupid, and that was done at least a decade ago with static+control flow analysis. As for AI, I was recently writing tests for a VT push parser in Rust (which is novel code, so no parroting here) and it clearly knew enough about VT to write a, correctly, failing test. I had a bug in my parser, and the test that AI generated found it.
At the end of the day, I'm not sure why anyone would believe the critique of someone who hasn't used a tool in earnest.