Ask HN: I'm less productive with TDD. Am I doing it wrong?

I think the problem is that you have an incomplete understanding of what it means to be productive.

Your programming course seems to be discussing an extreme and possibly excessive version of TDD. I have written a lot of code, and I like TDD, but in moderation. I certainly do not test every single function using TDD. I do not work out my functions in advance. However, I do take care to design and document major interfaces between components. I will write tests against those interfaces. For non-trivial parts of my code, I will write tests for smaller components. I do not take it all the way down to every function, that's nuts.

Now these tests do take time to write. And if you measure productivity by lines of code written per unit of time, well your number will go down. But that's the wrong measure. TDD increases my confidence that the tricky bits of my code are correct. It definitely catches bugs early, which is a huge productivity benefit. (And when I encounter a bug not caught by TDD, that I catch later, I will always write a new unit test to reproduce the problem.)

TDD also frees me up to clean up and refactor my code, and add new functionality. Since I have code that passes a lot of tests, I can change code with confidence because I know that the tests will pick up nearly all breakage caused by my change.

I will say that TDD has its limits. I find that TDD has been a dismal failure when the thing I'm testing has a dependency on some complex external thing, e.g. a service or a database. If you try to "mock" that external thing, you end up wasting tons of time debugging your mock database (for example). Also, external things with state (like services and databases) don't really fit TDD which depends on fast setup and teardown. Once you have these external dependencies, you are better off writing system tests.

The problem with TDD is that if you have a lot of tests, then when you want to change your code you have to change a lot of tests. This leads to inertia against changing your code, and then ultimately when you do need to change things, you end up just rewriting your tests so that they pass to match the new code, i.e. the new code becomes the test.

You nailed it. Mocking data while testing is such a pain. Takes the fun out of writng tests.

My experience is that the advocacy of TDD in introductory programming courses is leading people to think that the way you test software is with unit tests only. It's not, by far the most important mode of testing software is manual testing: test out your changes as you make them and ensure that the thing appears, at least superficially, to be acting as it is supposed to. There is no point beavering away at unit tests when the thing doesn't even vaguely do what it's supposed to do in manual testing.

For things that have a clean interface, isn't it better to write a higher-level integration test rather than relying on manual testing?

Or at least to write an integration test to mimic the manual test so that you can refactor without having to correctly remember what manual test you did?