Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Composable Tests

newsletter.kentbeck.com

Post reply on HN
Print view View original

21–30 of 67 posts

Re: Composable Tests

#21

Post by grim_io »

OakNinja wrote: post #4
grim_io wrote: post #2

I'm done listening to smart people proposing fixes for my dumb code. Either I'm not smart or disciplined enough to make it work, or my colleagues are not. Mostly both.

Kent Beck is smart because he proposes fixes that work in real life, as in environments with dumb code and undisciplined people. I can very much recommend his latest book ”Tidy first?”. It’s extremely short and concise, and is perfect for a very light book club within any tech team.

I assure you, there is no such thing as a book club at my job, especially not one focused on technical writing :)

Re: Composable Tests

#22

Post by skydhash »

troupo wrote: post #14
seanwilson wrote: post #10

Earlier quoted context omitted.

What's the problem of only keeping test 3 if it depends on test 2 and 1 passing anyway? The article mentions not to do this because "Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is." but you'll know what line it failed on. And some test runners let you break a test into steps, where gr…

> What's the problem of only keeping test 3 if it depends on test 2 and 1 passing anyway? That depends. Sometimes test 1 tests a combination of ways (e.g., property testing, or just going through a bunch of various inputs), and only a few of those are needed for test 2 . Sometimes you don't want your test 2 to be more complicated than it already is. Or the same things are needed checked in other tests. So you extract…

> That's why I always advocate investing in integration tests (test 20) and skip all the intermediate tests.

My approach is to have acceptance tests documented for any feature. Like how it would be from the user point of view to actually use the software. Then do Integration tests for each part of that workflows. That's usually the most ROI you will get for testing. Then I invest into unit tests for particular elements that are very important. I start from the middle of the pyramid because an actual e2e is expensive to setup (easy to maintain afterwards) and having lots of unit tests (easy to setup) is expensive to maintain.

Re: Composable Tests

#24

Post by akoboldfrying »

skydhash wrote: post #18
akoboldfrying wrote: post #16

> Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is. Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line nu…

> Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is. The issue with your approach is that codepath execution is more of a graph than a linear timeline. With one test, you…

If you need test1's logic as setup for test2, then you already have that "linear timeline" -- for test2 all by itself. Additionally (that is, redundantly) running the same setup code "prefix" by itself in test1 doesn't remove test2's linear timeline.

ETA: I'm assuming your objection to a "linear timeline" is that it reduces the potential for running tests in parallel -- have I got that right? If not, what do you see as being the problem with it?

Re: Composable Tests

#25

Post by jayd16 »

troupo wrote: post #3
grim_io wrote: post #2

I'm done listening to smart people proposing fixes for my dumb code. Either I'm not smart or disciplined enough to make it work, or my colleagues are not. Mostly both.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

IMO it seems kind of terrible.

You're not going to remember what tests "test 3" relies on. They aren't actually linear progressions 123. They will be "test this", "test that". If 3 fails you're going to want to immediately go and add all those asserts back to help you debug your assumptions.

The tests _will_ drift and that should be fine. Implicitly depending on other tests doesn't really get me anything.

Re: Composable Tests

#26

Post by solarengineer »

Someone had written a comment about whether Kent Beck has heard of global and static variables. That comment seems to have been deleted.

Kent Beck is co-creator [1] of the JUnit Testing Framework and has most definitely heard of static and global variables.

[1] https://junit.org/junit4/project-info.html

Re: Composable Tests

#27

Post by aleksiy123 »

mrkeen wrote: post #19
aleksiy123 wrote: post #12

Earlier quoted context omitted.

I don’t think that’s right? The isolation comes from the test implementation not the framework. There isn’t any framework out there that can guarantee/give you isolation. If I create a new in mem dB in the test there’s nothing stopping me from running it in parallel? Nothing about that “requires” isolation. It is isolation.

Maybe I misread. When I see a setUp() in a test suite, 9 times out of 10 it's to handle something shared and bulky, like a database. Otherwise you'd just do the thing - assert(expected, myService.run(input)); If I saw an extra myService.reset() or myService.setUp() I would suspect it's either in anticipation of a bad state (from a previous test) or to be a good neighbour for the next test which will run. In the cases…

I guess it all depends, isolation isn’t binary.

you can share some things and be isolated across other dimensions.

You could share a Postgres dB connection but just isolate the data logically.

You can isolate or share across individual tests, across suites, across envs, across runs, across time.

More isolation is generally better unless the complexity or cost is too high.

But I guess I was just confused as to how a test requires isolation rather than have it.

Re: Composable Tests

#28

Post by mamcx »

RHSeeger wrote: post #8
troupo wrote: post #3

Earlier quoted context omitted.

What he wrote is basically "don't repeat in test X what you already tested in test X-1". It's not as much composition as compounding, and can work quite well. Let's say something takes 20 steps, and you want to test all 20. Instead of this: test 1: do step 1, assert step 1 test 2: do step 1, assert step 1 do step 2, assert step 2 test 3: do step 1, assert step 1 do step 2, assert step 2 do step 3, assert step 3 ... y…

I wonder if it would work do design something that was able to say test 1: do step 1, assert step 1 test 2: requires: test 1 do step 2, assert step 2 test 3: requires: test 2 do step 3, assert step 3 test 4: requires: test 2 do step 3, assert step 3 If the assertion of each test doesn't change any state, that might make things easier to read. Though, given that I haven't spent much time pondering it, I expect it coul…

This make too much sense!

In special when testing against a DB.

Re: Composable Tests

#29

Post by neuronexmachina »

pfdietz wrote: post #5

I had a test suite with thousands of tests. One way of running it was to take all the passing tests, and then run them repeatedly in random order. This found new bugs involving unintended persistent state.

Reversing the test order is also pretty effective: https://pypi.org/project/pytest-reverse/

Re: Composable Tests

#30

Post by sholladay »

pfdietz wrote: post #5

I had a test suite with thousands of tests. One way of running it was to take all the passing tests, and then run them repeatedly in random order. This found new bugs involving unintended persistent state.

Shout out to AVA for running all tests async by default, which not only finds order-dependent issues, but also race condition and other timing related issues.

https://github.com/avajs/ava

Post reply on HN