Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

201–210 of 269 posts

Re: The day I started believing in unit tests

#201

Post by diarrhea »

corndoge wrote: post #34

I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling! I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but…

Interesting take. I find structuring code to be testable to make the code much clearer: mainly, by making dependencies explicit via dependency injection. I do that even if I don’t end up testing the code.

Re: The day I started believing in unit tests

#202

Post by solraph »

ejb999 wrote: post #41

I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won't build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me. Either I am…

> Either I am writing really good code so there are no bugs, or I am really bad a writing unit testing code to find those bugs.

Honestly, having literally had a scenario 20 minutes ago where I wrote a test for what I figured was absolutely trivial code, and having it _fail_ on me and pick up a bug that I hadn't considered (and this is not the first time this has happened) I would strongly suggest it's the latter.

Do your unit tests the output and side effects exactly, or do they just make sure the function returned without error?

Just because function/method/whatever has 100% coverage, doesn't mean you have tested all the potential scenarios.

Re: The day I started believing in unit tests

#203

Post by makeitdouble »

_dain_ wrote: post #184
eddd-ddde wrote: post #112

Earlier quoted context omitted.

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

>"this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases. Depends on the language and the business logic. Types are a way of specifying preconditions and postconditions; a more expressive type system lets you rule out more edge cases by making illegal states unrepresentable. In particular, I'm pretty sure…

Speaking only for me, but one can get used to the limits of the system and adapt (put another, I'd tend to introduce as many logical bugs, but in different ways)

For instance in a method that absolutely requires a specific type of object as a return, setting up a sacrificial default value to have the compiler happy, and actually build the inners of the function from there would be a normal course of action. That lets us run the code as we build it. But at the end if you forgot a case, it will still be a bug, except instead of having a wrong return type you get a wrong value. Weither it's better or not is up for debate.

Re: The day I started believing in unit tests

#204

Post by aleksiy123 »

corey wrote: post #44
davnicwil wrote: post #28

I think about unit tests being useful for getting more confidence that some deterministic, pure (mathematically speaking) and stateless piece of code that's data in data out actually works, particularly when you change it. If any of those conditions doesn't hold the cost/benefit certainly and even sometimes the absolute utility goes way down. If I have to mock anything, in particular, or more generally care at all ab…

I agree that mocks are brittle and nearly useless. If you follow SOLID principles to the extreme, you'll find that your code is separated into logic code that is pure and easy to unit test, and IO code that is very simple and can be tested by a relatively few number of integration tests.

To some extent this is pretty much the same as mocking. You are still injecting fake data into your pure logic functions whether its through their parameters or by them calling a mock.

I agree preferable but sometimes you want to test the logic of the code thats actually making decisions about how and when the IO is called.

You can do it with integration tests of course but in more complex environments with lots of complex IO dependencies mocking is cheaper. Its also hard to simulate specific failures in integration tests like a specific request failing. Pretty much mocking with extra steps.

So mocking has its place as well.

Re: The day I started believing in unit tests

#205

Post by newZWhoDis »

eddd-ddde wrote: post #112
throwaway290 wrote: post #100

Earlier quoted context omitted.

Was this statically or dynamically typed language?

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

Oh it absolutely matters. Rail apps in particular are full of “tests” a basic compiler would catch.

So you save a “ton of time” writing the code without “cognitive overhead of types”, then you spend 3x as long writing the tests

Re: The day I started believing in unit tests

#206

Post by icedchai »

corndoge wrote: post #34

I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling! I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but…

I work on a team with 75% of the developers don't write any tests. You never know what you're going to run into. Did I cause a new bug, or did I discover an old one? It's embarrassing when you discover completely non-working code paths.

I'm not even looking for a particularly high level of test coverage, just a basic "I wrote an API, here's a test (integration, unit, doesn't matter) for the happy path"-level of coverage would be great.

On the opposite end, I worked at places that wanted unit tests for every new function, even if it was something simple (like a getter or setter) used elsewhere. That's also terrible.

Re: The day I started believing in unit tests

#207

Post by MoreQARespect »

randomdata wrote: post #162
MoreQARespect wrote: post #146

Earlier quoted context omitted.

I've evangelized against unit testing at most companies I work at, except in one specific circumstance. That circumstance is complex logic in stateless code behind a stable API where unit testing is fine. I find this usually represents between 5-30% of most code bases. The idea that unit testing should be the default go to test I find to be horrifying. I find that unit test believers struggle with the following: 1) T…

> The idea that unit testing should be the default go to test I find to be horrifying. Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests. In practice, this means that a unit test won't break other tests. I am not sure why you would want anything other than unit tests? Surely everyone agrees that one test being able to break another test is a…

>Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests

I vaguely remember him also complaining that there were too many conflicting definitions of unit tests.

Maybe that can be solved with another definition?

https://xkcd.com/927/

or maybe not.

I dont know many people who would describe a test that uses playwright and hits a database as a unit test just because it is self contained. If Kent Beck does then he has a highly personalized definition of the term that conflicts with its common usage.

The most common usage is, I think, an xUnit style test which interacts with an app's code API and mocks out, at a minimum, interactions with systems external to the app under test (e.g. database, API calls).

He may have coined the term but that does not mean he owns it. If I were him Id pick a different name for his idiosyncratic meaning than unit test - one that isnt overburdened with too much baggage already.

Re: The day I started believing in unit tests

#208

Post by makeitdouble »

__MatrixMan__ wrote: post #29
the_sleaze9 wrote: post #5

Good story. I for one do not believe in Unit Tests and try to get LLM tooling to write them for me as much as possible. Integration Tests however, (which I would argue is what this story is actually praising) are _critical components of professional software. Cypress has been my constant companion and better half these last few years.

You should try switching it up. Write the tests and then ask the LLM to write the code that makes them pass. I find I'm more likely to learn something in this mode.

I'd argue having useable LLMs kind of brings out how problematic TDD is.

Imagine the dumbest function you have to write: a product A and a street address as input, and the shipping cost as an output.

How many test cases would you write to be absolutely sure that function actually does what you want it to do, and be confident it doesn't have weird exceptions that the LLM injected randomly ? I'd assume you'd still vet the code written by the LLM, but if it's hundreds of rambling lines doing weird stuff to get the right result, is it really faster than writing it yourself ?

Re: The day I started believing in unit tests

#209

Post by mayoff »

pfdietz wrote: post #123
throwaway290 wrote: post #118

Earlier quoted context omitted.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

Why isn't the question "does adequate testing dramatically reduce the benefits of static typing" asked? Why is static typing privileged by default?

I wouldn’t say static typing is privileged, but that testing is disadvantaged, because, in the words of Edsger Dijkstra, “Program testing can be used to show the presence of bugs, but never to show their absence!”

https://www.cs.utexas.edu/users/EWD/transcriptions/EWD02xx/E...

Re: The day I started believing in unit tests

#210

Post by icedchai »

_dain_ wrote: post #187
gbacon wrote: post #30

Earlier quoted context omitted.

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

>1. It talks to a database. >3. It touches the file system. These are BS. Maybe they made sense in the beforetimes when when we didn't have Docker containers or SSDs, but nowadays there's no reason you can't stand up a mini test database as part of your unit test suite. It's way simpler than mocking.

100% this. A guy I work with just rebuilt our CI/CD pipeline and we're spinning up a database and all dependent services in containers. There are no mocks and it works great.

In previous lives, I worked on tests that mocked everything. We spent more time creating and maintaining mocks than writing the actual tests.

Post reply on HN