Ask HN: Seriously, how do you TDD?

Don't let TDD destroy your architecture.

TDD encourages a form of programming where you build from the outside-in, building layer after layer of abstraction, putting off solving the actual problem until you get to the messy gooey centre where it ends up being a kludge. It's also very easy to make flawed assumptions in your original TDD spec which you don't realize until you get to that gooey centre, having wasted N days in the process.

I honestly don't tend to start anything until I have a basic idea of how it's going to work (not a fashionable idea these days), and find that projects end up with a much saner structure when they're built from the inside out.

So my preference is to start TDD once I've got a very basic version of something working, at which point you can start diving in to all the corner cases.

> The problem with this is that this is testing the “tip of the iceberg” that will be almost the whole, final app.

This is also what I've run into in practice, across numerous systems.

Actually the only cases where I've been able to do TDD effectively (and get >95% test coverage) was when I was in control over the entire codebase and its design from day one, thus being able to enforce whatever modularization was necessary by the tests, otherwise the systems quickly tended to get untestable (at least in regards to being able to start with a test). In addition, I was generally more successful in cases where I was dealing with writing libraries (that are called by other code), as opposed to web applications which need to deal with shuffling around bunches of data.

Let me give you a few examples of what didn't work. Suppose that I'm asked to help out with this pre-existing codebase that's been around for 5 years or so. I look how some request to purchase a product is processed (just a made up example, though along the lines of what I've seen): first you have some server side rendering with the full complexity of PrimeFaces and around 10-20 variables that affect whether certain buttons are visible, possibly there being a modal dialog or redirection in the middle of the process.

Then, you have the view code, which once again contains different methods that interact with services, passing data around and choosing whether to display additional messages or something else. Then, in the service layer, you might have 5-10 services that are involved in retrieving the data necessary for everything, from auditing and e-mail notifications, getting various pieces of information about users/billing/discounts and then checking all of that against what the user actually wants to purchase, perhaps some validation code thrown in along the way.

Validation code which also might need to interact with the database due to incrementally grown business requirements over the years. And from there on out, additional code for returning early in the case of any errors being present, which might need to be further processed for displaying them to the user. Oh, and if the process is successful without any errors, then you might also have some scheduled processes that would be created and would need to be handled. Almost any of these steps could have transaction rollbacks along the way and there might also be a large amount of PrimeFaces/Spring related code and workarounds in there.

And I've seen similar patterns across many projects, almost regardless of technologies used (Java is just a good example): if you give developers the chance to create tightly coupled code, they will jump at it horrifyingly often, especially if they're under time pressure or have a "good enough" mentality. Over time, any sorts of boundaries will dissolve, unless you go for a microservices based approach, but even then you'll still be dealing with "magic" code which will be hard to test (especially if you need some kind of application context to be running during the tests), be it IoC/DI, annotations/decorators, proxies for your code, or even something like ORM interactions.

Speaking of databases, good luck if your code has like 10-20 database calls along the hot path, which you'll need to mock or actually let it hit some database instance, though hopefully one with state that's either reproducible or can be rolled back (both of which will make your CI slow and more error prone). You might go for an in-memory database, but good luck trying to transpose or make Oracle-specific complex queries portable to something like that, given that for many learning JPQL is too much.

In short: there are situations where TDD is simply unsuitable because of how certain systems/domains are architected (I'm yet to see the PrimeFaces layer as something easily testable) and you're better off simply using unit tests where you can, as well as a healthy helping of integration tests (which you may or may not be able to do, depending on how your environments are laid out).

Of course, things might not look so dire for projects that are a bit newer and where you can still alter their structure. For library code, however, personally I'd recommend TDD, especially if you're ever in the need of doing anything security adjacent (e.g. wrapper code around proven security libraries). And for basically all of the cases where you can write mostly functional code with little to no side effects.

Also, here's a not so fun memory: trying to write tests that dealt with loading data from the file system. In Java, you can have a common piece of code for doing that (e.g. Paths.get), though what paths are valid will depend on the actual system and file system that JVM will be running on. In other words, when locally you have Windows and the CI server has a Linux distro or something, your tests will not run consistently. But you don't want to test that bit of code? Better be prepared to add it as an exception to any code coverage quality gate that you've set up. So annoying.

You can't "drive the design" with TDD. To write a failing test, you have to have a requirement which is encoded by the test. That requirement doesn't come from TDD, and TDD can't tell you how to procure it.

To go from a high level requirement like "command line application to track transactions and show expenses by category", you need to go through several levels of detailed design before you have anything that is implementable by a test-driven approach, which will necessarily be bottom-up.

Remember that tests can only confirm the presence of a defect, not its absence, and that not everything can be tested due to the intractability of exhaustive testing.

By blindly following test-driven design, you could end up with a useless, inflexible program that correctly categorizes expenses over only a small, fixed vocabulary of categories because it never occurred to anyone that a category must be something user-defined.

A user-defined category feature cannot be implemented with strict TDD because the space of user-defined categories is infinite. Say you have a test which confirms operation for every valid category string up to three characters long. That doesn't prove the system will not fail for a four-character-long category.

The only people who regard TDD as some kind of panacea are those with no formal background of any kind, who don't understand the role of testing in the overall software engineering context, and what its limitations are.

I'm not quite getting your categorization of the first approach as top-down and second as bottom-up, they are the opposite to me.

Your program is going to have a flow like this:

  $ rcd2-awesome-finance-app september2022.csv
  Category    Amount
  Food        $100.00
  Clothes     $200.00
  ...

Internally it'll be like this:

  main(string filename)
    csv = open(filename)
    cat->$ = new dict()
    foreach line in csv
      transaction = parse(line)
      cat = categorize(transaction)
      amount = amount(transaction) // may just be t[2] not a function
      cat->$[cat] = cat->$[cat].or_default(0) + amount // or other logic for first time seeing it
    foreach (cat, amount) in cat->$
      print("{cat}\t{amount}\n")

So your second example is the top-down one, it requires all this logic to exist. That makes TDD harder because going from a one-line CSV file and hardcoding the amount and category (first code is dumb code), to a second one-line CSV and actually categorizing is a massive jump.

Categorizing is the core, so categorizing is the thing that really needs testing. The rest doesn't even need (substantial) testing anyways, it's a bog standard CSV-data driven tool. Which leads to a point I want to make:

You don't need to do TDD on everything to do TDD. TDD is the "show your steps" of programming (analogous to the high school algebra requirement that you show and name each step in a solution). Many people despise that in their math homework, but it's important for the teacher for the same reason TDD can be important to you, the programmer.

Math teachers (ok, some may, they're poor teachers) don't ask students to show their steps just for grins. They ask it so they can observe the thought process of the student. A student tasked with solving for x with `2x = 4` may come up with `x=2`, but how? Did they divide by 2 or subtract 2 (I've done my share of math tutoring, I have seen students subtract what should've been divided). They got the answer by coincidence. So when they are given another problem where that coincidence doesn't work, they get the wrong answer and in more complex problems how they went wrong is non-obvious ( in 3x=9 => x = 6 it would be glaringly obvious, but with more terms and unobserved steps it's not; it's early the coffee hasn't kicked in yet so I can't think of good example problems to illustrate this).

TDD does the same thing, but it's there for you, not an instructor, to observe what you are thinking along the way. Each test is supposed to be small so that you can easily see that the logic is correct. However, if you can see a larger step to take (like that structure I threw up at the top or parts of it) then go for it. I've written I don't know how many CLI apps and I would never use TDD for anything but that inner categorization logic (or its equivalent for them). It would be silly for me to write a test to see that I was reading the file correctly. I can think of only one reason to do it: You don't know the language and API yet so you want to test your own knowledge, more than the program itself, to ensure you're calling it correctly. Even then, unless it has a weird file reading API I wouldn't bother, 99% of them will be the same or similar enough.

That's very false in my experience, and I did 100% TDD for years (have a more nuanced approach currently in TypeScript/React)

As a programmer with a problem, you first instinct is to start at the solution.

TDD pulls you back, and you first have to write the api and decide how you verify it.

If you're doing it well, you make both of those things as simple as possible, first. Reduce dependencies and inputs, etc.

That's actually the most important part, an easily consumable api that's easy to verify, not the implementation, and TDD forces you to do it first.

Also, a pet peeve of mine is seeing a line or more of code that doesn't do anything. With TDD, such detritus is impossible as you can't write production code unless it's making a red test turn green.