Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

121–130 of 269 posts

Re: The day I started believing in unit tests

#121

Post by __MatrixMan__ »

HideousKojima wrote: post #19
the_sleaze9 wrote: post #5

Good story. I for one do not believe in Unit Tests and try to get LLM tooling to write them for me as much as possible. Integration Tests however, (which I would argue is what this story is actually praising) are _critical components of professional software. Cypress has been my constant companion and better half these last few years.

Unit tests are useful for: 1) Cases where you have some sort of predefined specification that your code needs to conform to 2) Weird edge cases 3) Preventing reintroducing known bugs In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort.

That's what you get when you don't write the tests first.

Re: The day I started believing in unit tests

#122

Post by ska »

throwaway290 wrote: post #118
eddd-ddde wrote: post #112

Earlier quoted context omitted.

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

I don't think it is really unresolved. Static languages with sufficiently rich and modifiable type systems avoid a fraction of cases where you may well want a unit test, but it's not the overwhelming majority. Merely static helps too but not all that much. So while there is a reduction, it's a stretch to call it "dramatic".

Re: The day I started believing in unit tests

#123

Post by pfdietz »

throwaway290 wrote: post #118
eddd-ddde wrote: post #112

Earlier quoted context omitted.

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

Why isn't the question "does adequate testing dramatically reduce the benefits of static typing" asked? Why is static typing privileged by default?

Re: The day I started believing in unit tests

#124

Post by HideousKojima »

__MatrixMan__ wrote: post #121
HideousKojima wrote: post #19

Earlier quoted context omitted.

Unit tests are useful for: 1) Cases where you have some sort of predefined specification that your code needs to conform to 2) Weird edge cases 3) Preventing reintroducing known bugs In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort.

That's what you get when you don't write the tests first.

That's just doubling your work. If you don't already have a spec, your unit tests and actual code are essentially the same code, just written twice.

Re: The day I started believing in unit tests

#125

Post by BurningFrog »

troupo wrote: post #88
BurningFrog wrote: post #55

The classic test rookie mindset is to test the functionality of the whole system, because that's what really matters. But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. It took me some time to learn this, even after being told. It's the same for most people. This little post will probably convince no one. But maybe remember it when you…

> But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. To me, this is actual rookie mentality. You end up testing the same thing multiple times over different lines of code, mocking and providing various sets of testing data... When you could just test specified and/or observable behaviour of your system, and achieve the exactly same resu…

Well, I certainly don't end up doing that.

There are many ways of doing things, and I guess we do unit tests differently.

> When you could just test specified and/or observable behaviour of your system, and achieve the exactly same result with fewer tests.

In my experience, it turns out to be very difficult to test a specific behaviour 5-10 layers deep from an external interface. Also, when one of those intermediate layers changes, you tend to have to rewrite many of those tests.

Re: The day I started believing in unit tests

#127

Post by feoren »

appplication wrote: post #116
mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily. This is also where the dogma of “only test public methods” fails. If your public method requires extensive mocking but the core logic you need to protect is iso…

> If your public method requires extensive mocking but the core logic you need to protect is isolated in a private method that requires little mocking, the most effective use of developer resources may be to just test your private method.

You're looking at the tested code as immutable. If you're not allowed to touch the code being tested, then yes, you'll sometimes need to test private methods, and that is fine. "Don't test private methods" is actually more about how to architect the primary code, not a commandment on the test code. If you find that you're having to do extensive mocking to call a public method in order to test the functionality in some private method, that's a major smell indicating that your code could be organized in a better way.

Re: The day I started believing in unit tests

#128

Post by ska »

m3kw9 wrote: post #86
cfiggers wrote: post #48

Earlier quoted context omitted.

> if we want to call ourselves engineers we need to hold ourselves to engineering standards. Bridge builders do not get to skip tests. Bravo. We need more of this mindset in the world, and also more collective will to encourage it in one another. YOU are the kind of engineer I want writing the code that goes in my Dad's pacemaker or the cruise control in my wife's car.

If you have worked in places where safety is critical, you wouldn’t say something so shallow. In those places they place human verification above all else. They have a thick book where you do a full run and is double checked, they don’t f around with unit tests and say this is good to go

I don't think anyone is saying unit tests and you are good to go are they?

In any critical system work, there are multiple layers and you can't really skip any of them.

It's also sort of meaningless to talk about such testing without requirements and spec to test against. Traceability is as much a part of it as any of the testing.

By the time you get to the "thick book/full run" as you put it, there has typically been a metric crapload of testing done already.

Re: The day I started believing in unit tests

#129

Post by HideousKojima »

jkubicek wrote: post #61
HideousKojima wrote: post #19

Earlier quoted context omitted.

Unit tests are useful for: 1) Cases where you have some sort of predefined specification that your code needs to conform to 2) Weird edge cases 3) Preventing reintroducing known bugs In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort.

> In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort. If you rephrase this as, "verifying that our code does what it did yesterday" these types of tests are useful. When I'm trying to add tests to previously untested code, this is usually how I start. 1. Method outputs a big blob of JSON 2. Write test to ensure that…

I had to migrate some ancient VB.NET code to .NET 6+ and C#. The code outputs a text file, and I needed to nake sure the new output matched the old output. I could have written some sort of test program that would have been roughly equal in length to what I was rewriting to verify that any change I made didn't affect the output, and to verify that the internal data was the same at each stage. Or... I could just output the internal state st various points and the final output to files and compare them directly. I chose the latter, and it saved me far more work than writing tests.

If I need to verify that my code works the same as it did yesterday, I can just compare the output of today's code to the output of yesterday's code.

Re: The day I started believing in unit tests

#130

Post by feoren »

gorgoiler wrote: post #114
benrutter wrote: post #108

Earlier quoted context omitted.

I agree! I see a lot of stuff like "static typing is better than tests", "tests don't prove your code is bug free" etc as if tests somehow have to be a silver bullet to justify their existence. I definitely think its ok for the overall standard of test code to be lower than production code though (I guess horrible unmaintanable tests is maybe a bit much). A few reasons I can think of off the top of my head: - You can…

I often think of unit tests as being programmable types, like Eiffel pre/post conditions or functional languages with types like Even and Odd. For example, in double(x) -> y you can use types to say x belongs to the set of all integers and y must also must be in that set, but that’s about all you can say in Python. Unit testing lets you express that y must be an even number with the same sign as x. It is like formal…

But you literally cannot possibly test that assertion for all x. Let's take a slightly harder problem:

prove (or at least test conclusively) that for all integer x, the output y of the following function is always even:

    y = x^2 + x + 2
There is essentially no way to prove this for all x by simply testing all integers. If your integers are 64-bit, you don't have enough time in the lifespan of the universe.

On the other hand, you could simply reason through the cases: if x is even, then all terms are even. If x is odd, then x^2 is also odd, and x^2 + x = odd + odd = even. So you're done.

This is what people mean when they say "tests don't prove your code is correct" -- it's almost always better to be able to read code and prove (to some degree) that it's correct. It's really nothing like static types, which are also constructive proofs that your code is not incorrect in specific ways. (That is: it proves that your code is not [incorrect in specific ways], not that your code is [not incorrect].)

Once you prove your code correct, you can often write efficient tests with cases at the correct boundary points to make sure that proof stays correct as the code changes.

Post reply on HN