Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Questions to ask yourself when writing tests

charemza.name

Post reply on HN
Print view View original

41–50 of 98 posts

Re: Questions to ask yourself when writing tests

#41

Post by ancarda »

Ace17 wrote: post #39
discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

> ... micro level unit testing, driven by people like Uncle Bob. "The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest." "The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring.…

I really wish Uncle Bob would provide code examples with his blog posts and videos. I struggle to imagine what my tests would look like if I followed this. Possibly as my tests are so tightly coupled right now that refactoring is actually not possible in some cases.

Does anyone know of explations of this with a more hands-on approach, or is this simply a collection of ideas that can’t really be shown?

Re: Questions to ask yourself when writing tests

#42

Post by rimliu »

discreteevent wrote: post #8
ajmurmann wrote: post #5

Earlier quoted context omitted.

I'm not sure I agree with the "instead" in his recommendation. "micro level unit testing" comes with its own benefits, like faster feedback and also design feedback that integration or acceptance style tests won't give you. I like to start out with a high level acceptance tests and then write unit tests for every change to involved classes. You really need a healthy testing pyramid.

The problem is that, in my observation over a number of codebases with many developers: A lot of unit tests end up testing the implementation. When you go to refactor, these tests break which slows you down. To me it seems nearly impossible not to end up with these kinds of net negative tests when you follow a micro approach. I mean the approach outlined in books and blogs where you write a test and then write only a…

I recommend https://www.sandimetz.com/99bottles/ to everyone. Also her other book, Practical Object Oriented Design in Ruby. They both communicate the essential points very clearly. Also, after reading those books I got a better idea why OOP has a bad reputation, and why people are so bad at writing tests. Somehow these skills are presumed to be easy and already there when in reality they need practice, a lot of it.

Re: Questions to ask yourself when writing tests

#43

Post by blowski »

seanwilson wrote: post #35
discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

The argument I get against this is that when a higher level test fails, it's harder to locate the lines of code that broke the test unlike when you have lots of unit tests. I don't find this convincing though. Most of the time it's going to be pretty obvious looking at what lines of code have changed since the last working commit. Lots of projects get by without any tests at all as well. I'm not saying to skip testin…

There is no silver bullet. Personally, I let a combination of complexity and importance guide my tests.

The more likely it is that a piece of code will break, and the more business damage it will do if it does break, the more tests I wrap around it.

For self-contained algorithms that have a lot of branches or complex cases, I use more unit tests. When the complexity is in the interaction with other code, I write more high-level tests. When the system is simple but critical, I write more smoke tests.

If I’ve got simple code that’s unlikely to break and it doesn’t matter if it does break, I might have no tests at all.

Re: Questions to ask yourself when writing tests

#44

Post by lmm »

yeukhon wrote: post #38
u801e wrote: post #27

Earlier quoted context omitted.

> I am so sick of having to code in extremely brittle codebases that are impossible to refactor because of the sheer volume of UTs that are tightly coupled to very specific implementation details I think this comes from "unit tests" where they have assertions such as assertThisOtherSpecificMethodNameWasCalledWithTheseSpecificallyNamedParameters If more methods were written to not call out to disk or some external ser…

> If more methods were written to not call out to disk or some external service (e.g., a database or API) and we had a main method that handled the external communication, then the unit tests would be much less fragile. I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse…

Rather than having 3 different business logic methods that call out to the database, you have each construct a command object and then have a separate service that calls out to the database based on these commands. You can then test the 3 business logic methods by testing that they construct the correct commands based on their parameters.

Re: Questions to ask yourself when writing tests

#45

Post by crdoconnor »

KirinDave wrote: post #17
crdoconnor wrote: post #12

Earlier quoted context omitted.

>Can I do this faster if I don’t write a test? The answer is always no. I think the answer is mostly no but it's dangerous to think that it's always no. I've been given many stories in the past for which writing a realistic automated test would have taken days, manual verification took minutes and the code was fairly isolated and did not get changed very frequently. Writing a test under those circumstances is actuall…

That's a decision you can make, but all you're doing is costing yourself more time later for the benefit of less time now.

5x 5 minute manual test Automatd test fundamentalists are subject to the same kinds of folly that other fundamentalists are.

Re: Questions to ask yourself when writing tests

#46

Post by crdoconnor »

simonhamp wrote: post #19
crdoconnor wrote: post #12

Earlier quoted context omitted.

>Can I do this faster if I don’t write a test? The answer is always no. I think the answer is mostly no but it's dangerous to think that it's always no. I've been given many stories in the past for which writing a realistic automated test would have taken days, manual verification took minutes and the code was fairly isolated and did not get changed very frequently. Writing a test under those circumstances is actuall…

I would be very surprised if an automated test literally took days to write when a manual test is just minutes. Also the time savings still pay off later, as automated tests usually take seconds to run and there’s no training required - once it’s in the test suite and the test suite runs are automated, it will always run and quickly identify a failure - no “oops, we forgot to show Jim the Intern that he had to test t…

Trust me you'll come across examples like this one day if you make a concerted effort to automate a test for every story. It's a rough guess but I'd say I don't automate 1 out of 20 stories - I'm not exactly blithely unaware of the benefits an automated test suite brings.

Most of the examples that require "days" (or weeks) for the test and 5 minutes for the test would involve the building or amending an elaborate mock/stub.

Some examples where this happens include rare interactions with weird hardware, odd legacy APIs that are scheduled to be replaced, race conditions and obscure edge cases with mocked services.

I document all of these special cases and they should clearly remain special but I'm not going to blindly assume that automating the story will have a positive ROI.

Re: Questions to ask yourself when writing tests

#47

Post by arkh »

Ace17 wrote: post #40
philipodonnell wrote: post #37

Earlier quoted context omitted.

Honest question: can you expand on why this is bad? Doesn't efficiency outweigh respecting feature boundaries in tests?

> can you expand on why this is bad? Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue could be located. > Doesn't efficiency outweigh respecting feature boundaries in tests? In the end, what we're trying to minimize here is feature development time. This of course depends on the feedback loop duration (build + r…

> Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue could be located.

Tests are not a debug tool. Tests are here to tell you when you broke something.

When this happen you can get your debugging toolbox out: stack traces, profilers and things like GDB. And then follow the steps your test script did.

Re: Questions to ask yourself when writing tests

#50

Post by Vinnl »

discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

On the one hand this is true, but as someone who does prefer keeping unit tests relatively small in order to be able to pinpoint issues easier and not have tests break as often without it being clear why, I think the article does strike a nice balance of only recommending it where it really makes sense, rather than simply stimulating you to write as high-level tests as possible.
Post reply on HN