Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Write Unit Tests for Logging

principal-it.eu

Post reply on HN
Print view View original

31–38 of 38 posts

Re: How to Write Unit Tests for Logging

#31

Post by Tainnor »

the_af wrote: post #18
ThePadawan wrote: post #7

Earlier quoted context omitted.

To be fair, I worked at a company that had CI for running unit tests so nothing could be merged into remote master until the build was green. The build asserted that the number of failing unit tests was 0. It turns out that "breaking the unit test library completely such that 0 tests run instead of all thousands of them" did not break the build. When someone noticed a week later, that was not fun to clean up.

Tests that test nothing useful are surprisingly easy to write, and lead to a similar result -- even in cases where you do run them! That's why it makes some sense to test the tests (or at least, check their bug-detecting quality).

That's why you should IMHO always make sure you see your test fail at least once. TDD gives you that for free, but you don't have to use TDD: just manually breaking your implementation to see if the test breaks in the way it's supposed to should work equally well (bonus points if you make sure the error message is also precise enough to debug the issue).

Re: How to Write Unit Tests for Logging

#32

Post by the_af »

Tainnor wrote: post #31
the_af wrote: post #18

Earlier quoted context omitted.

Tests that test nothing useful are surprisingly easy to write, and lead to a similar result -- even in cases where you do run them! That's why it makes some sense to test the tests (or at least, check their bug-detecting quality).

That's why you should IMHO always make sure you see your test fail at least once. TDD gives you that for free, but you don't have to use TDD: just manually breaking your implementation to see if the test breaks in the way it's supposed to should work equally well (bonus points if you make sure the error message is also precise enough to debug the issue).

Yes. But just like we automate tests instead of manually testing -- that's what computers are for after all: automation -- we can automate the tests of tests!

Also, tests that once made sense might have become useless, so it seems relevant to have a way of automatically detecting obsolete "this doesn't test anything" tests even though they could have been useful once. Without relying too much on human intervention, that is.

Re: How to Write Unit Tests for Logging

#33

Post by gnusty_gnurc »

u801e wrote: post #29
gnusty_gnurc wrote: post #19

Earlier quoted context omitted.

Unit tests are fine for testing side effects if you've written the code properly with proper interfaces, etc. You need to test that you're using the contracts you've made, and integration tests verify that everything works as expected when both sides are complying with the contract. I'm saying that if I've abstracted out the filesystem into an interface the distinction of asserting against whether a method was called…

> I'm saying that if I've abstracted out the filesystem into an interface the distinction of asserting against whether a method was called on that interface and asserting a side-effect is somewhat trivial. But now you're enforcing a degree of coupling between your filesystem interface and your code. If you refactor the filesytem interface such that the overall logic hasn't changed, but the parameter order for one of…

Ah ok, I see what you're saying. I've seen the talk "Hoist your I/O" by Brandon Rhodes. He mentions that but I don't think he went into detail about commands/events or any of that stuff.

Re: How to Write Unit Tests for Logging

#34

Post by brlewis »

u801e wrote: post #30
brlewis wrote: post #22

Earlier quoted context omitted.

I don't think clearly defined boundaries are what's needed. People just need to keep in mind that the purpose of tests is to catch bugs. When I modify this code, what tests will tell me that it still does what it's supposed to? Keeping that question at the fore will work better than any cookbook.

> When I modify this code, what tests will tell me that it still does what it's supposed to? While that's true, you don't want to end up in a situation where the test is enforcing coupling which makes it harder to refactor your code. So, for instance, if you have tests that assert whether the method tested calls another method with certain parameters in a certain order, then you're enforcing coupling between the two…

> While that's true, you don't want

I'm confused. These opening words make it sound like you're getting ready to partially disagree, but then the rest of the comment is pure agreement. What did I miss?

Re: How to Write Unit Tests for Logging

#35

Post by woldrich »

Log4J2 has a testing library that is pretty dope, you can use it to assert on log events and parameters passed and tweak your appenders so that you're not spamming the console while the test runs.

Here's the coordinates in gradle:

compile group: 'org.apache.logging.log4j', name: 'log4j-core', classifier: 'tests'

Re: How to Write Unit Tests for Logging

#36

Post by u801e »

brlewis wrote: post #34
u801e wrote: post #30

Earlier quoted context omitted.

> When I modify this code, what tests will tell me that it still does what it's supposed to? While that's true, you don't want to end up in a situation where the test is enforcing coupling which makes it harder to refactor your code. So, for instance, if you have tests that assert whether the method tested calls another method with certain parameters in a certain order, then you're enforcing coupling between the two…

> While that's true, you don't want I'm confused. These opening words make it sound like you're getting ready to partially disagree, but then the rest of the comment is pure agreement. What did I miss?

After re-reading your original comment, there doesn't appear to be enough detail to say one way or another whether we're in agreement. Specifically, you didn't mention or address the issue of tests enforcing coupling between different components.

Re: How to Write Unit Tests for Logging

#37

Post by brlewis »

u801e wrote: post #36
brlewis wrote: post #34

Earlier quoted context omitted.

> While that's true, you don't want I'm confused. These opening words make it sound like you're getting ready to partially disagree, but then the rest of the comment is pure agreement. What did I miss?

After re-reading your original comment, there doesn't appear to be enough detail to say one way or another whether we're in agreement. Specifically, you didn't mention or address the issue of tests enforcing coupling between different components.

You applied this question:

When I modify this code, what tests will tell me that it still does what it's supposed to?

To this situation:

When I modify this code, will tests that enforce coupling between different components tell me that the code still does what it's supposed to?

You observed that no, such tests would need to be changed even though the overall logic was the same. I assume that next you would apply the same question to a test that is for the overall logic, and conclude that yes, it's a good test.

Re: How to Write Unit Tests for Logging

#38

Post by Tainnor »

the_af wrote: post #32
Tainnor wrote: post #31

Earlier quoted context omitted.

That's why you should IMHO always make sure you see your test fail at least once. TDD gives you that for free, but you don't have to use TDD: just manually breaking your implementation to see if the test breaks in the way it's supposed to should work equally well (bonus points if you make sure the error message is also precise enough to debug the issue).

Yes. But just like we automate tests instead of manually testing -- that's what computers are for after all: automation -- we can automate the tests of tests! Also, tests that once made sense might have become useless, so it seems relevant to have a way of automatically detecting obsolete "this doesn't test anything" tests even though they could have been useful once. Without relying too much on human intervention, t…

that's basically mutation testing but I've never seen a place that has been able to use it... maybe at some point the tools will get better
Post reply on HN