Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

31–40 of 269 posts

Re: The day I started believing in unit tests

#31
post #3

unfortunately legitimate use cases for unit tests (like this) are pretty rare in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method/function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests such tests are not just useless, they're positively harmful https://…

The point of unit tests is not to CYA during refactors, but to confirm that the implementation is consistent between small changes without weird side effects. A coworker once thought unit tests were dumb, and ended up writing code that repeated the call to an application 10x for the same info. This didn’t result in a changed UI because it was a read, but it’s not good to just suddenly 10x your reads for no good reaso…

> confirm that the implementation is consistent between small changes without weird side effects

not sure what this is referring to, but I'll give an example

say you have a requirement that says if you call POST /user with a non-existing user, a user should be created and you should get a 2xx response with some basic details back

you could test this by actually hitting the endpoint with randomly generated user data known to not already exist, check that you get the expected 2xx response in the expected format, and then use the user id you got back to call the GET /user/userId endpoint and check that it's the same user that was just created

this is a great test! it enforces actual business logic while still allowing you to change literally everything about the implementation - you could change the codebase from Java Spring Boot to Python Flask if you wanted to, you could change the persistence tech from MySQL to MariaDB or Redis etc etc - the test would still pass when the endpoint behaves as expected and fail when it doesn't, and it's a single test that is cheap to write, maintain and run

OR

you could write dozens of the typical corporate style unit test i'm referring to, where you create instances of each individual layer class, mocks of every class it interacts with, mocked database calls etc etc which 1) literally enforce every single aspect of the implementation, so now you can't change anything without breaking the tests 2) pretend that things work when they actually don't (eg, it could be that the CreateUserDAO actually breaks because someone stuffs up a db call, but guess what, the CreateUserResource and CreateUserService unit tests will still pass, because they just pretend (through mocks) that CreateUserDao.createUser returns a created user

Re: The day I started believing in unit tests

#32
post #13
post #7

I barely ever have unit tests flagging real issues. It's always a chore to update them. Feature/end to end tests though... Plenty of real issue flagged.

> I barely ever have unit tests flagging real issues That sounds like you work alone and haven't worked for a long time on a code base with unit tests. Or the unit tests are bad.

Dont take this the wrong way, but this is the answer i would get from enterprise devs usually when pointing this out.

Then i would realize that their definition of a real issue was completely removed from any business or user impact, but geared more towards their understanding of the process detail in question.

I would argue that there certainly are some good places for unit tests, like if you have some domain-driven design going and can have well defined unit-tests for your business logic, but this usually is the smallest part of the codebase.

Mocking things that talk to databases etc. usually gives a false sense of security while that thing could break for a whole number of reasons in the real world. So just dropping the mock here and testing the whole stack of the application can really do wonders here in my experience.

Re: The day I started believing in unit tests

#33
post #21

I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…

That reminds me of this time I wrote some code to add a method to Python string objects. The first reply to my issue on it in the bug tracker was "We shouldn't accept this, it's trivial to implement in your own code, see: XXXX". The second reply was "You have a bug in your implementation in the first reply."

It took a couple years to be accepted.

Re: The day I started believing in unit tests

#34
I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling!

I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but if we want to call ourselves engineers we need to hold ourselves to engineering standards. Bridge builders do not get to skip tests.

Re: The day I started believing in unit tests

#35
The gem of this story is the author is not running unit test in what most folks understand a unit test is. As he also pointed out, he is executing the tests on target so it is more of an integration tests rather than unit tests. In the test that he is doing, it brings in new categories of potential faults. ie scheduling issues, memory constraints, interrupts servicing,

Re: The day I started believing in unit tests

#37
post #30
post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

That's a great rule of thumb.

Re: The day I started believing in unit tests

#38
post #30
post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

My "unit tests" do hit the database and file system, and I have found and fixed many many problems during testing by doing so. I have found many other problems with those calls in production when I didn't do so. Yes, they make testing a lot slower. Our main app takes around 40 minutes to build which isn't good. I'd like it to be faster. But writing a bunch of separate integration tests to cover those functions would be a steep price. I can understand reasonable people choosing either approach.

Re: The day I started believing in unit tests

#39
It's all degrees. Unit tests are great at finding examples of errors or correct behaviours. However they prove nothing and they definitely do not demonstrate the absence of errors.

They are often sufficient for a great deal of projects. If all it takes to convince you it's "good enough," are a handful of examples then that's it. As much as you need and no less.

However I find we programmers tend to be a dogmatic bunch and many of us out there like to cling to our favoured practices and tools. Unit tests aren't the only testing method. Integration tests are fine. Some times testing is not sufficient: you need proof. Static types are great but fast-and-loose reasoning is also useful and so you still need a few tests.

What's important is that we sit down to think about specifying what it means for our programs to be, "correct." Because when someone asks, "is it correct?" You need to as, "with respect to what?" If all you have are some hastily written notes from a bunch of meetings and long-lost whiteboard sessions... then you don't really have an answer. Any behaviour is, "correct," if you haven't specified what it should be.

Re: The day I started believing in unit tests

#40
post #8
post #4

We can't even have a consensus on what "unit" tests really are... Every company i work for has a different meaning for it. Some places consider a test "unit" when all the dependencies are mocked, some places consider a whole feature a "unit".

Kent Beck (the originator of test-driven development) defines a unit test as "a test that runs in isolation from other tests". This is very different from the popular and completely misguided definition of a unit test as "a test that tests a class/method in isolation from other classes/methods". But it doesn't really matter if you want to call a given test an "integration" test or a "unit" test. The point of any test…

> The point of any test is to fail when something breaks and pass when something works

The point of any test is to document API expectations for future developers (which may include you).

That the documentation happens to be self-validating is merely a nice side effect.

Post reply on HN