191–200 of 269 posts
Earlier quoted context omitted.
It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.
Why isn't the question "does adequate testing dramatically reduce the benefits of static typing" asked? Why is static typing privileged by default?
Earlier quoted context omitted.
I meant "prove" as in, "mathematically proven." That is, for all possible inputs your theorem holds. A unit test is only an example of one such input. They don't prove there are no bad inputs. There are many places in programming where you don't care to prove properties of your program to this level of rigor; that's fine -- sufficiency is an important distinction: if a handful of examples are enough to convince you t…
Well that's silly. You're not arguing against unit tests, you're arguing against testing in general. But empirically testing does improve the reliability of software. You're tossing the baby out with the bathwater in the interest of an academic ideal of proved correctness. I will add that if you are verifying the correctness of some code, you have a formal specification of what the code is supposed to do. That is, yo…
However I think folks do get dogmatic about testing and will claim things like, “all you need is unit/integration/types.”
I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won't build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me. Either I am…
1. Decide that was the goal. Start measuring. Who knows, maybe I have to do nothing.
2. Discover that I seriously underestimated the number of bugs I produce. No one is available to review my code, changing languages (to something with a stronger type system) was out of the question. Only option appears to be methodical testing of every line of code.
3. Print (on dead trees - this was decades ago) all my code. Manually test all of it, running a highlighter down the listing. Continue to the entire code listing has a solid line of code down the left. It worked! Bugs per delivered lines of code dropped off a cliff. But geezz it took a loooong time, longer than writing the code. Finding an input sequence the exercised some code was surprisingly hard. And it was boring. Still success - and people who used it immediately noticed the improvement in quality and commented.
4. Then new features have to be added. Does this mean I have to test it all again? Surely not - I'll just test the bits I changed. Result: bugs per line of code rapidly start to ramp up again.
5. So I test everything again. That works, but it's horribly inefficient. I can spend days releasing a few line change. I can't get small changes anything like a reasonable time frame.
6. The solution is obvious to a programmer: automate your work, which in this case translates to writing code to do the tests. So I write unit tests for new code. It ends up being slower than doing manual tests :( Code size doubles. It works in keeping the bug count down, but can I afford to keep doing this time wise?
7. Then I add features to new code with unit tests. Initially this is painful - I move at perhaps 1/2 the speed because now I have to change at least twice the amount of code (actual and unit-test). Still, it's success bug count wise, and running unit tests is much, much faster than manually testing.
8. Keep doing this, notice that despite me having to change twice the lines of code (actual code and tests) when I'm adding new features I'm producing more debugged lines of code than before. Even more interesting, I'm fearlessly making much larger changes now. Turns out I'm using the unit tests as guard rails. I no longer minimise my changes to reduce the odds of introducing a bug.
9. Finally, notice that unit testing has changed the way I write code. And it's for the better. Code that's easy to test is also easy to understand. For example, it's much easier to test a pure function than something with side effects, so you minimise side effects as much as possible. You make your interfaces (which are the thing you focus your testing on) as small as possible. Testing deep inside a complex module is difficult and the torturous unit test code you have to write to do that is hard to understand. So you split things into smaller modules, each of which has those clean interfaces, to give your tests greater visibility. Turns out writing code so unit tests can understand it is oddly similar to writing code so that humans can easily understand it.
So it turns out unit testing is a win in every way, when done well. Well, except for the "it's boring" bit. (Numerous comments here hint at copilot being a real help.)
But writing code that's amenable to unit tests isn't something you do naturally. Fortunately just getting practice at writing unit tests is enough to teach the skill. Sadly, that takes time and frustration. While you learn your productivity will drop for a while. And worse, when writing new code adding unit tests is always slower than the old way. The pay back only comes when you later make changes.
Earlier quoted context omitted.
> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily. This is also where the dogma of “only test public methods” fails. If your public method requires extensive mocking but the core logic you need to protect is iso…
> If your public method requires extensive mocking but the core logic you need to protect is isolated in a private method that requires little mocking, the most effective use of developer resources may be to just test your private method. When I did unit tests in C++, I found a simpler (and better) solution: Shrink the class by splitting it up into multiple classes. Often the logic in the private methods could be gro…
You can split things out decompose some things even if its just some util functions and start sending out chunks of code for review. It doesn't even necessarily have to be seperate files.
Your reviews will be faster and smoother too.
Earlier quoted context omitted.
And integration tests can also be fast
Test containers really help with this. Should still have the big system tests that run overnight but a set of integration tests using Test Containers to stand in for the infrastructure dependencies is awesome. My team has a ton of those and they run inside a reasonable time frame (5min or so) but we still allow for excluding those from test runs so you can run just the unit tests.
I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…
So that puts it at about 1 bug per 20 LoC
Some estimates are something as high as 75 bugs per 1,000 LoC but that many bugs don't make it out to customers because of QA / Developer actions. So yeah, right on the money.
Earlier quoted context omitted.
That's what you get when you don't write the tests first.
That's just doubling your work. If you don't already have a spec, your unit tests and actual code are essentially the same code, just written twice.
I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won't build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me. Either I am…
I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won't build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me. Either I am…