Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unit Tests Considered Harmful

shaiyallin.com

Post reply on HN
Print view View original

11–20 of 76 posts

Re: Unit Tests Considered Harmful

#11

Post by pfix »

pfix wrote: post #8
mewpmewp2 wrote: post #5

Earlier quoted context omitted.

How do you measure 100% confidence? Pretty sure I have never reached 100% confidence, but I haven't had a metric to track it so I have no data to prove it. The only possible way to reach near 100% is I think by not writing any code, and by not building anything. I do think 100% code coverage goal doesn't mean much, since you can potentially achieve it using 0 assertions. Or you make 1 assertion per test, leaving 95%+…

You measure how many of your deployments ship broken code. I've worked in contexts where test coverage was sufficient enough to be confident that when deploying there wouldn't be any issues apart from really novel unexpected stuff you wouldn't have tested anyway. I've also worked in contexts wher I didn't have that confidence. I would say you know when you're in that environment and directly measuring doesn't really…

Because it's linked in another post right now:

If you feel confident that renovate[1] pull requests can be merged without review on a green pipeline, you have 100% confidence in your tests ;-)

My first context was that way.

[1]: https://www.jvt.me/posts/2024/04/12/use-renovate/

Re: Unit Tests Considered Harmful

#13

Post by osigurdson »

Poefke wrote: post #2

The target audience for unit tests is not the client, it is the developer. Unit tests allow you to change code with more confidence. 100% code coverage is not a useful aim, you should aim for 100% confidence in your code. Unit tests can also function as example code, that can't get out of date, since then the tests will fail. Testing for quality assurance is a different thing, usually called acceptance testing and so…

Agree. The "what about the client" trope is a complete non-argument. If unit tests make code better, the client maybe wants that. The argument should purely be based on if they are helpful.

Re: Unit Tests Considered Harmful

#14

Post by toolz »

logro wrote: post #4

If your not Dijkstra then using title like that negatively affects people's perception of your post.

I usually assume it's going to be a controversial take, but controversy shouldn't be inherently negative to anyone. Plenty of the best things we have today caused and were propelled by controversy.

Re: Unit Tests Considered Harmful

#15

Post by mewpmewp2 »

pfix wrote: post #8
mewpmewp2 wrote: post #5

Earlier quoted context omitted.

How do you measure 100% confidence? Pretty sure I have never reached 100% confidence, but I haven't had a metric to track it so I have no data to prove it. The only possible way to reach near 100% is I think by not writing any code, and by not building anything. I do think 100% code coverage goal doesn't mean much, since you can potentially achieve it using 0 assertions. Or you make 1 assertion per test, leaving 95%+…

You measure how many of your deployments ship broken code. I've worked in contexts where test coverage was sufficient enough to be confident that when deploying there wouldn't be any issues apart from really novel unexpected stuff you wouldn't have tested anyway. I've also worked in contexts wher I didn't have that confidence. I would say you know when you're in that environment and directly measuring doesn't really…

What do you mean by "broken code"? Any sort of code that yields in a bug? Any sort of bug? Because interpreting directly broken code implies to me some sort of compilation error. Also measuring that seems very complicated, depending on how you interpret "broken code". E.g. a bug could influence 100% of the customers, 5% of the customers, 0.1% of the customers and only when certain conditions happen, it could be a combination of deployments introducing this. It could be a bug that can easily be worked around by a customer. It could be a bug that only arrives 6 months down the line, because some dependency changed, and you didn't have proper handling for it, even though it was expected that you had. E.g. you are calling a third party API that alerts you in advance they are planning to add extra error code in the system in 6 months, so you should probably also include it, but you don't and then system bugs due to that.

The problem could arrive 2 years in the future when scale increases and the code wasn't built to handle it, even though it was expected.

You take 100 of your past deployments and see how many resulted in some sort of a bug? E.g. if there's 0, then you have 100% confidence?

> any issues apart from really novel unexpected stuff you wouldn't have tested anyway.

I mean that's one thing that makes it impossible to have it 100% for me at least.

> I would say you know when you're in that environment and directly measuring doesn't really help as it's more a matter of culture than technical metrics.

I may have been in wrong environments throughout my whole life, but I also have trouble imagining a perfect environment where there's 100% confidence.

Re: Unit Tests Considered Harmful

#16

Post by naasking »

potherca wrote: post #9

It keeps coming back to the same stupid erosion of paradigms. Once upon a time, some developers that were smarter than others started testing their code. They described ways to test smaller parts of large software systems separately before being integrated together. For instance "parameter testing" (to validate component subprograms against their specification) and "assembly testing" (for parts put together)[^1] As a…

Studies on TDD has failed to show a benefit over writing tests after the fact. The only factor that seems to matter is writing tests, the more tests the better. If you can write your code so it's amenable to more testing, or such that it requires fewer tests because it's simpler/has fewer cases to handle, great. If you have a framework that can generate tests (fuzzing, property-based testing), awesome.

Re: Unit Tests Considered Harmful

#17

Post by nu11ptr »

Everyone talking about unit tests, and testing in general as a thing to achieve are missing it entirely. Tests aren't a thing to achieve, and talking about whether unit testing, mocks etc. are better or worse without context is pointless. The thing to achieve is having a code base that 1) works 2) can be changed easily and proved to be still working. Now that we know that we can start deciding "how" we can achieve that and it will be different for every code base (and often different strategies for different parts of the code base). The key thing to realize in a test strategy is that your code has two interfaces, not one. The first interface is the user, the second is some form of automated test harness. Hence, testing becomes a function of two points above and a design required to achieve that.

In summary, don't do this:

- I need 100% code coverage

- Everything needs a mock

- Ensuring unit tests for everything

Do this:

- Ask how can I ensure my code works?

- How can I ensure my code keeps working?

- What sort of testing strategy can achieve the above?

- What sort of testing strategy is easily maintained without causing tons of excess burden?

- What interfaces do I need on each module of my software to achieve this?

Re: Unit Tests Considered Harmful

#20

Post by kirth_gersen »

Testing is difficult. It has both the problems of perverse metrics as well as all the problems inherent in coding itself. Unit tests aren't rubbish, they just aren't the only type of testing needed for complex projects. They are a good starting point and I suspect their popularity comes from that. This article was mostly fluff and I regret the 3 min I spent testing it out. Save yourself a read, it's mostly a hot take with a clickbait title.
Post reply on HN