Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

11–20 of 269 posts

Re: The day I started believing in unit tests

#11
post #3

unfortunately legitimate use cases for unit tests (like this) are pretty rare in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method/function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests such tests are not just useless, they're positively harmful https://…

The point of unit tests is not to CYA during refactors, but to confirm that the implementation is consistent between small changes without weird side effects.

A coworker once thought unit tests were dumb, and ended up writing code that repeated the call to an application 10x for the same info. This didn’t result in a changed UI because it was a read, but it’s not good to just suddenly 10x your reads for no good reason.

TFA also describes discovering weird side effect race conditions as a result of unit tests.

Re: The day I started believing in unit tests

#12
I don't believe in unit tests as they are practiced. This, unfortunately, is the kind of thing that can work in principle, but the realities make it unusable.

There are multiple problems with unit tests, as they are implemented in the industry. And to make the unit tests usable and productive you need to make them so productive that it can offset those problems.

First of all, for unit tests to work everybody has to contribute quality unit tests. One team member writing unit tests well for his part of functionality is not going to move the needle -- everybody has to do this.

Unfortunately, it is rarely the case that all team members are able to write quality code this is the case for unit tests.

Usually, the reality is that given deadlines and scope, some developers will deprioritize focusing on writing good unit tests to instead deliver what business people do really care about -- functionality. Give it enough time and unit tests can no longer be trusted to perform its job.

Second, it is my opinion that refactoring is extremely important. Being able to take some imperfect code from somebody else and improve it should be an important tool in preventing code rot.

Unfortunately, unit tests tend to calcify existing code making it more expensive to change the functionality. Yes, more, not less expensive. To move a lot of stuff around, change APIs, etc. you will usually invalidate all of the unit tests that work around this code. And fixing those unit tests in my experience takes more effort than refactoring the code itself.

Unit tests are good for catching errors AFTER you have made the error. But my personal workflow is to prevent the errors in the first place. This means reading the code diligently, understanding what it does, figuring out how to refactor code without breaking it. Over the years I invested a lot of effort into this ability to the point where I am not scared to edit large swaths of code without ever running it, and then have everything work correctly on the first try. Unit tests are usually standing in the way.

I think where unit tests shine is small library code, utilities, where things are not really supposed to change much. But on the other hand, if they are not really supposed to change much there also isn't much need to have unit tests...

The most paradoxical thing about unit tests is that teams that can write unit tests well can usually produce code of good enough quality that they have relatively little use of unit tests in the first place.

What I do instead of unit tests? I do unit tests. Yes, you read that correctly.

The trouble with unit tests is that everybody gets the part of what unit is wrong. Unit does not have to mean "a class". Units can be modules or even whole services.

What I do is I test a functionality that matters to the client -- things I would have to renegotiate with the client anyway if I was to ever change it. These tests make sense because once they are written -- they do not need to change even as the functionality behind them is being completely rewritten. These test for what clients really care about and for this they bring a lot of bang for the buck.

Re: The day I started believing in unit tests

#13
post #7

I barely ever have unit tests flagging real issues. It's always a chore to update them. Feature/end to end tests though... Plenty of real issue flagged.

> I barely ever have unit tests flagging real issues

That sounds like you work alone and haven't worked for a long time on a code base with unit tests. Or the unit tests are bad.

Re: The day I started believing in unit tests

#14
post #9

I believe in them. But unit tests are useless around useless humans. And there’s lots of those. A fine example is that time I wrote a test suite for a domain specific language parser. Someone wanted to break the language so they deleted the tests. New stuff was added without tests. They confidently broke everything historically and looking forward. Then blamed it on me because it was my test suite that didn’t catch i…

In my opinion this is because we don't teach Chesterton's Fence early enough (or often enough) to internalize it at a societal level.

Re: The day I started believing in unit tests

#15
post #3

unfortunately legitimate use cases for unit tests (like this) are pretty rare in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method/function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests such tests are not just useless, they're positively harmful https://…

Well-written breaking tests represent something changing in a code base. You can be intentional about breaking a test, but then at least you can be very explicit about what you are changing.

Have seen all to many times I've broken a unit test in a code base that I did not intend to break, just to have an aha moment that I would have introduced a bug had that test not been present.

Unit tests are a trade off between development speed and stability (putting aside other factors, such as integration tests, etc). In large corporate settings, that stability could mean millions of dollars saved per bug.

That example you provided is a poor one and not really consistent with your point that unit tests are useless - the point is being made that that specific test of UserResource is useless, which I also agree with. Testing at the Resource level via integration test and Service level via unit test is probably sufficient.

Re: The day I started believing in unit tests

#16
post #3

unfortunately legitimate use cases for unit tests (like this) are pretty rare in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method/function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests such tests are not just useless, they're positively harmful https://…

If maintained right, unit tests at edge conditions can quickly diagnose system and runtime state health.

If you work with malicious on incompetent staff at times (over 100 there is always at least 1)... it is the only way to enforce actual accountability after a dozen people touch the same files over years.

"The sculpture is already complete within the marble block, before I start my work. It is already there, I just have to chisel away the superfluous material." ( Michelangelo )

Admittedly, in-house automated testing for cosmetic things like GUI or 3D rendering pipelines is still nontrivial.

Best of luck =)

Re: The day I started believing in unit tests

#18
So at work we would run tons of tests against the real service with a real database, seeding thousands of schemas to allow for parallel testing of tests that change state.

This takes 3 minutes, 1 if you use tmpfs. It only takes These actually cover most real world use cases for a query-engine we maintain.

Unit tests have their place for pieces of code that run based on a well defined spec, but all in all this integration or component-level testing is really what brings me the most value always.

Re: The day I started believing in unit tests

#19

Good story. I for one do not believe in Unit Tests and try to get LLM tooling to write them for me as much as possible. Integration Tests however, (which I would argue is what this story is actually praising) are _critical components of professional software. Cypress has been my constant companion and better half these last few years.

Unit tests are useful for:

1) Cases where you have some sort of predefined specification that your code needs to conform to

2) Weird edge cases

3) Preventing reintroducing known bugs

In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort.

Re: The day I started believing in unit tests

#20
post #10

Earlier quoted context omitted.

In reality, unit tests and integration tests are different names for the same thing. All attempts at post facto differentiation fall flat. For example, the first result on Google states that a unit test calls one function, while an integration test may call a set of functions. But as soon as you have a function that has side effects, then it will be necessary to call other functions to observe the change in state. Th…

No. Or maybe only if you also consider 'village' and 'city' to be the same thing.

Clusters of humans cohabiting a confined space? If you squint hard enough…
Post reply on HN