Why TDD isn't crap

Not necessarily. If the rewrite is only addressing issues that would have been prevented with tests, then sure, this is clearly an argument for TDD. However, if the rewrite is going beyond what could have been provided by tests, then having a large testing system could actually make the rewrites harder, which means it's an argument against TDD.

It's hard to say which is the case without knowing the details, and I'd certainly err on the side of saying a yearly rewrite is not a good sign. But if a company is undergoing rapid growth, it's not that unusual for certain systems to be rewritten frequently as fundamental new insights are gathered about how to tackle problems that are hard to scale. And if the system isn't that large to begin with, periodic rewrites could be easier than writing a single version that's supposed to last 10+ years as business requirements dramatically change and expand.

That said, testing the input/output directly doesn't work well because now a single change the output data (adding a field to the JSON for example) will call tons of tests to fail.

I usually use or create some JSON serialization/deserialization classes and use those to generate fake data and ensure that the API returns 200 responses.

That way if I add a field to the JSON objects, my tests are unaffected and will generate the new fields in the tests.

So I may do something like this (Python-ish):

    new_user = User.generate_user()
    response = api.post('/users', new_user.serialize())
    assert response.status_code == 200
    
    response = api.get('/users')
    users_json = json.loads(response)
    users = {}
    for user in users_json:
        users.add(User.unserialize(user))
    assert new_user in users

Hope that helps.

However, I think I reached my sweet spot just weeks ago. Here is my optimum workflow now:

1) Write test cases (the one sentences that say what is expected, in plain English) 2) Implement the code 3) Implement the test code

The test cases become neatly arranged in bullet-like layout in a test case file. I'm able to read through and be confident that I'm probably covering most if not all the cases that need to be covered. I'm always able to switch between the code and this file to make sure my code covered all that the tests need.

Then once the code is done, I come back to the test cases to implement them one by one, catching error by error and seeing my code coming to life. As I code now, I know I have the implementation I'm quite confident of, my tests that have business value are being covered one by one. It's been pleasure since then and I'm more confident of my code and of my time spent efficiently.

This matches much of my experience with DI on real projects. Dependency injection is, unfortunately, used to reduce the labor required to achieve massive coupling. Of course the intended and potentially useful application of DI is being able to rewire an application with different components for different purposes or different environments, and the tradeoff for this flexibility is that you lose explicitness. You can no longer see the explicit wiring in code, which is a big downside. Yet I've worked on several DI-heavy projects with seasoned engineers who get this completely backwards. They see DI as a labor-saving device that allows them to write heavily coupled systems without having to explicitly work out the dependencies between things. In fact, they even see DI as eliminating the cost of complex interdependencies because it reduces the work and cleverness required to create them.

Partly I think this reflects a desire to work on "real" "enterprise" systems. Instead of fighting to keep complexity below the level where DI actually helps, they embrace the flattering thought of, hey, we're doing big boy work here — it's going to get complex, so we'd better use DI from the outset. People who take a lot of pride in working on big systems can't help creating them.