Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unit Tests Considered Harmful

shaiyallin.com

Post reply on HN
Print view View original

31–40 of 76 posts

Re: Unit Tests Considered Harmful

#31

Post by eesmith »

kstrauser wrote: post #25
Poefke wrote: post #2

The target audience for unit tests is not the client, it is the developer. Unit tests allow you to change code with more confidence. 100% code coverage is not a useful aim, you should aim for 100% confidence in your code. Unit tests can also function as example code, that can't get out of date, since then the tests will fail. Testing for quality assurance is a different thing, usually called acceptance testing and so…

A few years ago, I got tired of my company talking about plans to upgrade our main app from Python 2 to Python 3, until one weekend I just did it. The tests caught a million little changes that I worked through until everything passed. Come Monday, we were on Python 3 and I took a couple workdays off to play video games. I wouldn’t have dared even start if I didn’t have confidence in our test suite.

I was not so lucky! I had a good test suite, but there were a lot of places where I mixed up strings and bytes. I had to add new parameters to specify in Unicode encoding/decoding options, new tests to handle those failures, and new APIs so I could have one function return bytes and another return strings.

Plus, I had C extensions, which had to be updated to handle changes in the Python/C API, including places where Python 2.7 could handle both Unicode and bytes in the ASCII subset:

  >>> u"bbcf".decode("hex")
  '\xbb\xcf'
  >>> "bbcf".decode("hex")
  '\xbb\xcf'
  >>> b"bbcf".decode("hex")
  '\xbb\xcf'
but under Python 3 required more work:

  >>> bytes.fromhex("BBCF")
  b'\xbb\xcf'
  >>> bytes.fromhex(b"BBCF")
  Traceback (most recent call last):
    File "", line 1, in 
  TypeError: fromhex() argument must be str, not bytes
AND, I needed to support both Python 2.7 and Python 3.5+ on the same code base.

AND, I needed to support various third-party tools that had their own different migration paths for how to handle the transition.

Re: Unit Tests Considered Harmful

#33

Post by taylodl »

Poefke wrote: post #2

The target audience for unit tests is not the client, it is the developer. Unit tests allow you to change code with more confidence. 100% code coverage is not a useful aim, you should aim for 100% confidence in your code. Unit tests can also function as example code, that can't get out of date, since then the tests will fail. Testing for quality assurance is a different thing, usually called acceptance testing and so…

Depends on how your unit tests were designed. You need to avoid the Fragile Test Problem. Bob Martin wrote about this a while back: http://blog.cleancoder.com/uncle-bob/2017/10/03/TestContrava...

Re: Unit Tests Considered Harmful

#34

Post by jjice »

icedchai wrote: post #32

Excessive mocking can be problematic. I prefer to test the real thing where possible even this makes most of my unit tests actually "integration tests" in the eyes of some.

As long as there's way to identify a true unit test (no outside dependencies) vs an integration test (some dependencies like a DB or Redis), it doesn't matter to me. I'd like to be able to see which category they're in and run them separately (for CI purposes), but they're all just code that runs other code and checks the output, at the end of the day.

Re: Unit Tests Considered Harmful

#35

Post by rTX5CMRXIfFG »

taylodl wrote: post #33
Poefke wrote: post #2

The target audience for unit tests is not the client, it is the developer. Unit tests allow you to change code with more confidence. 100% code coverage is not a useful aim, you should aim for 100% confidence in your code. Unit tests can also function as example code, that can't get out of date, since then the tests will fail. Testing for quality assurance is a different thing, usually called acceptance testing and so…

Depends on how your unit tests were designed. You need to avoid the Fragile Test Problem. Bob Martin wrote about this a while back: http://blog.cleancoder.com/uncle-bob/2017/10/03/TestContrava...

There needs to be a rule against having to read entire articles or YT videos just so that a reader can understand the point that a commenter is trying to make, but which they cannot explain in their own words, no? It’s very anti-discussion, and how do we even know to what extent commenters are quoting or agreeing with their sources?

Re: Unit Tests Considered Harmful

#36

Post by globular-toast »

This the best talk I've ever seen about testing (Ian Cooper): https://www.youtube.com/watch?v=EZ05e7EMOLM

I see he's done a much more recent follow up, which I intend to watch: https://www.youtube.com/watch?v=IN9lftH0cJc Not sure if it contains new ideas.

He says the unit is not "a class" but a module. It's up to you to decide what the module does (ie. what level of abstraction it's at). More importantly, he argues against mocking and essentially "unit testing" in most cases.

Re: Unit Tests Considered Harmful

#37

Post by gregors »

The author is mostly complaining about mocking from what I can tell.

>>>> Strict coverage for classes prevents regression in these classes but does not assert that the feature actually works

literally the point of unit tests

Re: Unit Tests Considered Harmful

#39

Post by eesmith »

naasking wrote: post #16
potherca wrote: post #9

It keeps coming back to the same stupid erosion of paradigms. Once upon a time, some developers that were smarter than others started testing their code. They described ways to test smaller parts of large software systems separately before being integrated together. For instance "parameter testing" (to validate component subprograms against their specification) and "assembly testing" (for parts put together)[^1] As a…

Studies on TDD has failed to show a benefit over writing tests after the fact. The only factor that seems to matter is writing tests , the more tests the better. If you can write your code so it's amenable to more testing, or such that it requires fewer tests because it's simpler/has fewer cases to handle, great. If you have a framework that can generate tests (fuzzing, property-based testing), awesome.

I'm pretty convinced that test-first is best seen as a negotiating technique to get the time to write the tests in the first place.

"Okay, the code is done, time to write the tests" results in pressure to shorten testing, since if the golden path works then surely everything else works, right?

Your management and your sales people then say "You're a great developer, it looks like it works, and we need to deploy that code now because customers are demanding it and the competition is breathing down our necks."

It's hard to resist that pressure.

While saying that it's best practices to do TDD means you don't need to deal with that sort of negotiation.

Re: Unit Tests Considered Harmful

#40

Post by ramesh31 »

The author conflates unit testing with integration testing, although it seems the terminology is a huge source of confusion for articles like this. Anything with mocks is an integration. Units are pure functions only. Generally a mixture of unit and e2e is the best way to go. Mocked out integrations tend to be a nightmare to maintain and provide little value.
Post reply on HN