Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

221–230 of 269 posts

Re: The day I started believing in unit tests

#221

Post by nerdponx »

linsomniac wrote: post #33
donatj wrote: post #21

I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…

That reminds me of this time I wrote some code to add a method to Python string objects. The first reply to my issue on it in the bug tracker was "We shouldn't accept this, it's trivial to implement in your own code, see: XXXX". The second reply was "You have a bug in your implementation in the first reply." It took a couple years to be accepted.

Sounds familiar. Was that str.removeprefix?

Re: The day I started believing in unit tests

#222

Post by nerdponx »

therealdrag0 wrote: post #172
m3kw9 wrote: post #80

Earlier quoted context omitted.

You could have watch the program and observed the failure, why need to write a test to be “surprised” it failed

A huge benefit of tests is their regression protection against future changes, often by other engineers. You don’t get that from ad hoc manual execution.

This. I like tests because it's hard to know if I accidentally broke something other than the thing I'm working on. Even software of modest size is at a level of complexity beyond what a human can QA in a reasonable amount of time for every revision. If you're at the point of needing a checklist with even 3 items on it, you're past the point of needing tests.

Re: The day I started believing in unit tests

#223

Post by nerdponx »

icedchai wrote: post #210
_dain_ wrote: post #187

Earlier quoted context omitted.

>1. It talks to a database. >3. It touches the file system. These are BS. Maybe they made sense in the beforetimes when when we didn't have Docker containers or SSDs, but nowadays there's no reason you can't stand up a mini test database as part of your unit test suite. It's way simpler than mocking.

100% this. A guy I work with just rebuilt our CI/CD pipeline and we're spinning up a database and all dependent services in containers. There are no mocks and it works great. In previous lives, I worked on tests that mocked everything. We spent more time creating and maintaining mocks than writing the actual tests.

I think those are still considered "integration" tests in the traditional way of thinking. The problem is that a lot of applications don't do much other than interact with external resources, to the point where isolated unit tests are rare or only cover non-critical code, while just about any substantive test is an "integration" test.

Re: The day I started believing in unit tests

#224

Post by icedchai »

nerdponx wrote: post #223
icedchai wrote: post #210

Earlier quoted context omitted.

100% this. A guy I work with just rebuilt our CI/CD pipeline and we're spinning up a database and all dependent services in containers. There are no mocks and it works great. In previous lives, I worked on tests that mocked everything. We spent more time creating and maintaining mocks than writing the actual tests.

I think those are still considered "integration" tests in the traditional way of thinking. The problem is that a lot of applications don't do much other than interact with external resources, to the point where isolated unit tests are rare or only cover non-critical code, while just about any substantive test is an "integration" test.

Yes, you are correct. I never cared much for the traditional way of thinking about tests. I've had more arguments with people claiming this is an integration test, not a unit test, write some mocks, etc. Outside of very specific pieces of code, you generally get more value from integration tests.

Re: The day I started believing in unit tests

#225

Post by [deleted] »

kibwen wrote: post #140
throwaway290 wrote: post #118

Earlier quoted context omitted.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

Having static types is like having an automatic, exhaustively comprehensive, efficient unit test generator for the classes of bugs whose units tests are the most boring and annoying to write and maintain. Static types don't prevent you from needing unit tests, they free you up to focus on writing tests that are actually interesting.

[deleted]

Re: The day I started believing in unit tests

#226

Post by RaftPeople »

randomdata wrote: post #219
RaftPeople wrote: post #216

Earlier quoted context omitted.

> End-to-end tests are unit tests, generally speaking. Generally, in the software industry, those terms are not considered the same thing, they are at opposite ends of a spectrum. Unit tests are testing more isolated/individual functionality while the end to end test is testing an entire business flow. Here's an example of one end to end test (with validations happening at each step): 1-System A sends Inventory avail…

> Here's an example of one end to end test Bad example, perhaps, but that's also a unit test[1]. Step 8 is dependent on the state of step 1, and everything else in between, so it cannot be reduced any further (at last not without doing stupid things). That is your minimum viable unit; the individual, isolated functionality. [1] At least so long as you don't do something that couples it with other tests, like modifyin…

Every step updates shared databases (frequently plural). In the case of the fulfillment step, the following systems+databases were involved: ERP, WMS, Shipping.

Typically, in end to end testing, tests are run within the same shared QA system and are semi-isolated based on choice of specific data (e.g. customers, products, orders, vendors, etc.). If this test causes a different test to fail, or vice-versa, then you have found a bug.

If we call that entire sequence of steps a "unit" test, would you start with testing the entire sequence of steps, or would you recommend testing the individual steps first?

And if we did test the individual steps first, we would give that testing a different name? Like maybe "sub-unit" testing?

Re: The day I started believing in unit tests

#227

Post by donatj »

throwaway290 wrote: post #100
donatj wrote: post #21

I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…

Was this statically or dynamically typed language?

It was PHP, for what it's worth. I've had similar experiences in Go though.

Re: The day I started believing in unit tests

#228

Post by randomdata »

RaftPeople wrote: post #226
randomdata wrote: post #219

Earlier quoted context omitted.

> Here's an example of one end to end test Bad example, perhaps, but that's also a unit test[1]. Step 8 is dependent on the state of step 1, and everything else in between, so it cannot be reduced any further (at last not without doing stupid things). That is your minimum viable unit; the individual, isolated functionality. [1] At least so long as you don't do something that couples it with other tests, like modifyin…

Every step updates shared databases (frequently plural). In the case of the fulfillment step, the following systems+databases were involved: ERP, WMS, Shipping. Typically, in end to end testing, tests are run within the same shared QA system and are semi-isolated based on choice of specific data (e.g. customers, products, orders, vendors, etc.). If this test causes a different test to fail, or vice-versa, then you ha…

> Every step updates shared databases (frequently plural).

That's fine. It all happens within a single unit. A unit should mutate shared state within the unit. Testing would be pretty much useless without.

> If we call that entire sequence of steps a "unit" test, would you start with testing the entire sequence of steps, or would you recommend testing the individual steps first?

For all intents and purposes, you can't test the individual steps. All subsequent steps are dependent on the change in inventory state in step 1. And the product of step one is undoubtedly internal state, so there is no way for the test to observe the state change in isolation (unless you do something stupid). You have to carry out the subsequent steps to be able to infer that the inventory was, in fact, updated appropriately.

After all, the whole reason you are testing those steps together is because you recognize that they represent a single instance of functionality. You don't really get to choose (unless you choose to do something stupid, I suppose).

> And if we did test the individual steps first, we would give that testing a different name?

If the individual steps can be tested individually (ignoring a case of you doing something stupid), it's not actually and end-to-end process, so your example would make no sense. Granted, we have already questioned if it is a bad example.

Re: The day I started believing in unit tests

#229

Post by lmm »

eddd-ddde wrote: post #112
throwaway290 wrote: post #100

Earlier quoted context omitted.

Was this statically or dynamically typed language?

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

> Major bugs are not "oh this can be null"

Not true. Even today the majority of security bugs are simple buffer overflows, for example. The subtle bugs are more memorable, but that doesn't mean they're actually more common.

> major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

It absolutely does, if you put a little bit of effort into actually using it. Represent your business logic invariants in the type system, then the compiler won't let you accidentally violate them.

Re: The day I started believing in unit tests

#230

Post by fijiaarone »

_dain_ wrote: post #184
eddd-ddde wrote: post #112

Earlier quoted context omitted.

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

>"this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases. Depends on the language and the business logic. Types are a way of specifying preconditions and postconditions; a more expressive type system lets you rule out more edge cases by making illegal states unrepresentable. In particular, I'm pretty sure…

It’s also not possible to write a non-trivial program in Rust.
Post reply on HN