Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Composable Tests

newsletter.kentbeck.com

Post reply on HN
Print view View original

11–20 of 67 posts

Re: Composable Tests

#11

Post by skydhash »

RHSeeger wrote: post #6
OakNinja wrote: post #4

Earlier quoted context omitted.

Kent Beck is smart because he proposes fixes that work in real life, as in environments with dumb code and undisciplined people. I can very much recommend his latest book ”Tidy first?”. It’s extremely short and concise, and is perfect for a very light book club within any tech team.

I agree, both with your analysis of him and your review of "Tidy first?". We read and discussed it at work, and I enjoyed it a lot.

I enjoyed the last (and theoretical) part of "Tidy first?" as I was already doing the recommended practices in the first two. But yes, very pragmatic advice and easy to apply to get your own corner of peace in a somewhat chaotic codebase.

One term I can use is "defensive programming" (not sure if the term has been used before). People are going to do awful stuff with their code, it's up to you to draw boundary of correctness to stop it from spilling on the part that you're responsible for. It should be automated (with tests and static analysis) as well as documented. I think of those boundary as hazmat suits and I take extra care of maintaining their integrity.

Re: Composable Tests

#12

Post by aleksiy123 »

mrkeen wrote: post #7

> If a test runs by first setting up its own test fixture, creating from scratch all the data it will be using as input, then that test is guaranteed to be isolated. It doesn’t matter what order you run the tests, the results will be exactly the same. Completely backward. This definition requires isolation, rather than granting it. In reality, you (or your test framework) provides the isolation by hobbling along, exe…

I don’t think that’s right?

The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation.

If I create a new in mem dB in the test there’s nothing stopping me from running it in parallel?

Nothing about that “requires” isolation. It is isolation.

Re: Composable Tests

#13

Post by mrkeen »

troupo wrote: post #3
grim_io wrote: post #2

I'm done listening to smart people proposing fixes for my dumb code. Either I'm not smart or disciplined enough to make it work, or my colleagues are not. Mostly both.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

I think I disagree with Kent, but your explanation is clearer, so I'll object here.

There's nothing wrong with hitting the same assertion multiple times, even if it doesn't sit nicely in your gut.

From a purely philosophical point of view: If I have testFoo(), testBar(), and testFooAndBar(), and my Foo is plain wrong, then both testFoo() and testFooAndBar() must fail. Anything less is misleading/dishonest.

From a practical side: Changes happen. Someone will remove testBar(), and then you're down to 0 assertions on Bar, even though you have a test claiming to testFooAndBar(). It's not even a crazy hypothetical. Someone with a different test philosophy will think (to quote TFA) "They are redundant! Something must be wrong." and delete testBar() because obviously testFooAndBar() already covers it.

Anyway, we all know how to deal with repetition. That's what programming is!

  testFoo()
  _ = validateFoo(foo())

  testBar()
  _ = validateBar(bar())

  testFooAndBar()
  foo = validateFoo(foo())
  bar = validateBar(bar())
  _   = validateFooAndBar(fooAndBar(foo, bar))

Re: Composable Tests

#14

Post by troupo »

seanwilson wrote: post #10
troupo wrote: post #3

Earlier quoted context omitted.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

What's the problem of only keeping test 3 if it depends on test 2 and 1 passing anyway? The article mentions not to do this because "Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is." but you'll know what line it failed on. And some test runners let you break a test into steps, where gr…

> What's the problem of only keeping test 3 if it depends on test 2 and 1 passing anyway?

That depends.

Sometimes test 1 tests a combination of ways (e.g., property testing, or just going through a bunch of various inputs), and only a few of those are needed for test 2.

Sometimes you don't want your test 2 to be more complicated than it already is. Or the same things are needed checked in other tests. So you extract them into test 1.

And sometimes (and in some of code bases most of the time) test 1 is redundant and unnecessary. That's why I always advocate investing in integration tests (test 20) and skip all the intermediate tests.

Re: Composable Tests

#15

Post by troupo »

RHSeeger wrote: post #8
troupo wrote: post #3

Earlier quoted context omitted.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

I wonder if it would work do design something that was able to say test 1: do step 1, assert step 1 test 2: requires: test 1 do step 2, assert step 2 test 3: requires: test 2 do step 3, assert step 3 test 4: requires: test 2 do step 3, assert step 3 If the assertion of each test doesn't change any state, that might make things easier to read. Though, given that I haven't spent much time pondering it, I expect it coul…

Yup. I'd love to have dependency declarations for tests like this.

Re: Composable Tests

#16

Post by akoboldfrying »

> Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is.

Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is. The only thing you lose is that a test function or method may now cover several related checks (they are related by "setup dependence"), meaning their names may need to be somewhat broader. But you can still describe the specific semantics of each assert() check in a one-line comment beforehand if you want. There's no need to cram it into a legal method name.

ETA: Prefer to write tests whose "arrange" steps are as simple as possible, to minimise unnecessary overlaps. But if the simplest possible "arrange" step for a test is something that itself needs to be checked for correctness, just do that check right there, and nowhere else. Anything beyond that is ceremony that adds nothing useful.

Re: Composable Tests

#17

Post by brabel »

RHSeeger wrote: post #8
troupo wrote: post #3

Earlier quoted context omitted.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

I wonder if it would work do design something that was able to say test 1: do step 1, assert step 1 test 2: requires: test 1 do step 2, assert step 2 test 3: requires: test 2 do step 3, assert step 3 test 4: requires: test 2 do step 3, assert step 3 If the assertion of each test doesn't change any state, that might make things easier to read. Though, given that I haven't spent much time pondering it, I expect it coul…

Yes that is the way to do it, in Spock that’s what @Stepwise does.

Re: Composable Tests

#18

Post by skydhash »

akoboldfrying wrote: post #16

> Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is. Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line nu…

> Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is.

The issue with your approach is that codepath execution is more of a graph than a linear timeline. With one test, you artificially constrains it to a linear timeline even if the assertions are correct. With multiples and independent you only assert one specific node. That lets you switch up how you do the preliminary steps. I much prefer an exhaustive test unit for step 1, and just a regular call in step 2 and step 3.

Re: Composable Tests

#19

Post by mrkeen »

aleksiy123 wrote: post #12
mrkeen wrote: post #7

> If a test runs by first setting up its own test fixture, creating from scratch all the data it will be using as input, then that test is guaranteed to be isolated. It doesn’t matter what order you run the tests, the results will be exactly the same. Completely backward. This definition requires isolation, rather than granting it. In reality, you (or your test framework) provides the isolation by hobbling along, exe…

I don’t think that’s right? The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation. If I create a new in mem dB in the test there’s nothing stopping me from running it in parallel? Nothing about that “requires” isolation. It is isolation.

Maybe I misread. When I see a setUp() in a test suite, 9 times out of 10 it's to handle something shared and bulky, like a database. Otherwise you'd just do the thing - assert(expected, myService.run(input)); If I saw an extra myService.reset() or myService.setUp() I would suspect it's either in anticipation of a bad state (from a previous test) or to be a good neighbour for the next test which will run.

In the cases where each unit test does have it's own individual Postgres or whatever, sure.

Re: Composable Tests

#20

Post by natbennett »

pfdietz wrote: post #5

I had a test suite with thousands of tests. One way of running it was to take all the passing tests, and then run them repeatedly in random order. This found new bugs involving unintended persistent state.

After a bunch of experiences like this I will now basically only use test runners that automatically run the tests in randomized order.

Gotta catch that stuff early.

Post reply on HN