Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

171–180 of 269 posts

Re: The day I started believing in unit tests

#171

Post by pfdietz »

agentultra wrote: post #145
pfdietz wrote: post #73

Earlier quoted context omitted.

> they prove nothing If they fail, they prove there's a bug (in either the test or the code.) This is like literally any other kind of test.

I meant "prove" as in, "mathematically proven." That is, for all possible inputs your theorem holds. A unit test is only an example of one such input. They don't prove there are no bad inputs. There are many places in programming where you don't care to prove properties of your program to this level of rigor; that's fine -- sufficiency is an important distinction: if a handful of examples are enough to convince you t…

Well that's silly. You're not arguing against unit tests, you're arguing against testing in general. But empirically testing does improve the reliability of software. You're tossing the baby out with the bathwater in the interest of an academic ideal of proved correctness.

I will add that if you are verifying the correctness of some code, you have a formal specification of what the code is supposed to do. That is, you have a description of valid inputs, and a formula determining if the output is correct, given those inputs. But if you have those, you can also do property-based testing: generate random inputs that satisfy the input properties, and check that the output satisfies the output condition. This is all easier that proving correctness (it requires little or no manual intervention) and gives much of the same benefit.

Re: The day I started believing in unit tests

#172

Post by therealdrag0 »

m3kw9 wrote: post #80
corndoge wrote: post #34

I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling! I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but…

You could have watch the program and observed the failure, why need to write a test to be “surprised” it failed

A huge benefit of tests is their regression protection against future changes, often by other engineers. You don’t get that from ad hoc manual execution.

Re: The day I started believing in unit tests

#173

Post by noselasd »

throwaway290 wrote: post #118
eddd-ddde wrote: post #112

Earlier quoted context omitted.

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

I'm writing unit tests in rust and c++, I'm in the same boat as the op, often finding logical errors while writing the tests.

Not to mention peace of mind when you go and mess around with code you wrote 9 months ago - if you mess up or didn't think of a corner case, there's decent chance it'll get caught by existing tests.

Re: The day I started believing in unit tests

#174

Post by BeetleB »

teeray wrote: post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

> they are fast and cheap to run.

But expensive to write. Especially if you want them to be fast and cheap to run.

Re: The day I started believing in unit tests

#175

Post by closeparen »

gbacon wrote: post #30
teeray wrote: post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

It is pretty excruciating (and IMO useless) to write true, DB-isolated unit tests for DB access layer code.

Re: The day I started believing in unit tests

#176

Post by BeetleB »

appplication wrote: post #116
mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily. This is also where the dogma of “only test public methods” fails. If your public method requires extensive mocking but the core logic you need to protect is iso…

> If your public method requires extensive mocking but the core logic you need to protect is isolated in a private method that requires little mocking, the most effective use of developer resources may be to just test your private method.

When I did unit tests in C++, I found a simpler (and better) solution: Shrink the class by splitting it up into multiple classes. Often the logic in the private methods could be grouped into 1-3 concepts, and it was quite logical to create classes for each of them, give them public methods, and then have an instantiation of that class as a private member.

Now all you need to do is write unit tests for those new classes.

Really, it led to code that was easier to read - the benefit was not just "easier to test". Not a single colleague (most of whom did not write unit tests) complained.

I've yet to run into a case where it was hard to test private behavior via only public methods that couldn't be solved this way.

Re: The day I started believing in unit tests

#177

Post by qup »

__MatrixMan__ wrote: post #29
the_sleaze9 wrote: post #5

Good story. I for one do not believe in Unit Tests and try to get LLM tooling to write them for me as much as possible. Integration Tests however, (which I would argue is what this story is actually praising) are _critical components of professional software. Cypress has been my constant companion and better half these last few years.

You should try switching it up. Write the tests and then ask the LLM to write the code that makes them pass. I find I'm more likely to learn something in this mode.

How effective is the LLM when used this way, compared to normally?

Re: The day I started believing in unit tests

#178

Post by kreeben »

I don't see how you can either believe or not believe, in a unit test. A unit test is what it is. It's a real thing. It exists. Use it, or don't.

How this topic can sometimes be about belief is beyond me. It's like if a person found a screw driver and says, I now believe in screw drivers.

The topic of how people believe in unit tests, to me is proof that the world is screwed. We're all screwed and everything is a screw driver.

Re: The day I started believing in unit tests

#179

Post by qup »

ejb999 wrote: post #41

I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won't build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me. Either I am…

try writing the test first

Re: The day I started believing in unit tests

#180

Post by BeetleB »

BurningFrog wrote: post #55

The classic test rookie mindset is to test the functionality of the whole system, because that's what really matters. But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details really matter. It took me some time to learn this, even after being told. It's the same for most people. This little post will probably convince no one. But maybe remember it when you…

I'll mirror a sibling comment. For me your mentality is the rookie mentality. I too once believed in strict unit tests, as well as a strict differentiation between them and other kinds of tests (end to end, integration, etc).

Then I joined a project where they were just starting to add tests to an existing project, and the lead developer was adamant on the following philosophy: "Virtually all tests will run the full program, and we'll mock out whatever slows the tests down (e.g. network access, etc)". I whined but I had to comply. After a year of this, I give him credit for changing my mindset. The majority of bugs we found simply would not have been found with unit tests. On the flip side, almost none of the unit test failures were false alarms (function signature changed, etc).

Since then, I've dropped categorizing tests as unit tests vs other types of tests. Examine the project at hand, and write automated tests - preferably fast ones. Focus on testing features, and not functions.

Post reply on HN