Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

111–120 of 269 posts

Re: The day I started believing in unit tests

#111

Post by slothtrop »

erikpukinskis wrote: post #83
pfdietz wrote: post #73

Earlier quoted context omitted.

> they prove nothing If they fail, they prove there's a bug (in either the test or the code.) This is like literally any other kind of test.

A bug in test code is not a real bug. It’s just a test that’s not giving you useful information. Lots of tests don’t give you useful information. Some that fail and some that pass. It’s easy to write a test that doesn’t provide useful information across time. Harder to write a test that does.

> A bug in test code is not a real bug.

The bug is in the library that the test code invokes. Tests themselves should be simple, if there are bugs in tests they are trivial once you have a framework figured out.

Re: The day I started believing in unit tests

#112

Post by eddd-ddde »

throwaway290 wrote: post #100
donatj wrote: post #21

I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs. I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it? By the time I was done writing the test I had found three major bugs in tha…

Was this statically or dynamically typed language?

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

Re: The day I started believing in unit tests

#113

Post by appplication »

JonChesterfield wrote: post #107

Most software doesn't work the moment you stray from the expected path. Whether that's because most software isn't tested competently or because software testing practices don't deliver robust software is not yet clear. I suspect that unit tests, and tests in general, will be considered a historical artifact from the time before we worked out how to write software properly. For example, we don't generally unit test t…

I think it’s a little over optimistic to think that we will ever work out how to properly write software. Some new patterns may help, but we will always have a need for unit tests and other tests.

Wrt typing, that’s a very narrow set of errors, and I would dare say even a small minority of the things that can and do go wrong in software are type related. That said, effective typing is another orthogonal tool to unit tests that can help create robust software. On that front, what we are missing is a language with robust typing that catches these type errors, but also gets out of developers way the rest of the time.

Re: The day I started believing in unit tests

#114

Post by gorgoiler »

benrutter wrote: post #108
mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

I agree! I see a lot of stuff like "static typing is better than tests", "tests don't prove your code is bug free" etc as if tests somehow have to be a silver bullet to justify their existence. I definitely think its ok for the overall standard of test code to be lower than production code though (I guess horrible unmaintanable tests is maybe a bit much). A few reasons I can think of off the top of my head: - You can…

I often think of unit tests as being programmable types, like Eiffel pre/post conditions or functional languages with types like Even and Odd.

For example, in double(x) -> y you can use types to say x belongs to the set of all integers and y must also must be in that set, but that’s about all you can say in Python.

Unit testing lets you express that y must be an even number with the same sign as x. It is like formal verification for the great unwashed, myself included.

Re: The day I started believing in unit tests

#115

Post by [deleted] »

troupo wrote: post #78
randomdata wrote: post #6

Earlier quoted context omitted.

In reality, unit tests and integration tests are different names for the same thing. All attempts at post facto differentiation fall flat. For example, the first result on Google states that a unit test calls one function, while an integration test may call a set of functions. But as soon as you have a function that has side effects, then it will be necessary to call other functions to observe the change in state. Th…

You should not be downvoted as heavily as you are now. I feel like we did testing a disservice by specifying the unit to be too granular. So in most systems you end up with hundreds of useless tests testing very specific parts of code in complete isolation. In my opinion a unit should be a "full unit of functionality as observed by the user of the system". What most people call integration tests. Instead of testing N…

[deleted]

Re: The day I started believing in unit tests

#116

Post by appplication »

mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily.

This is also where the dogma of “only test public methods” fails. If your public method requires extensive mocking but the core logic you need to protect is isolated in a private method that requires little mocking, the most effective use of developer resources may be to just test your private method.

> On caveat is that some developers write pretty nasty unit tests. Their production code is nice and readable, but then they just went nuts in the unit tests and created a horrible unmaintainable mess, I don't get why you'd do that.

I have also seen this a lot and usually it’s when people try to add too much DRY to their unit tests. I recall being as a junior dev told by our lead that boilerplate and duplication in tests is not strictly a bad thing, and I have generally found this to be true over the years. Tests are inherently messy and each one is unique. Trying to get clever with custom test harnesses to reduce duplication is more likely to lead to maintainability issues than it is test nirvana. And if your code requires so much setup to test, that is an indicator of complexity issues in the code, not the test.

Re: The day I started believing in unit tests

#117

Post by jcranmer »

gbacon wrote: post #30
teeray wrote: post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

I'd actually quibble a lot on this definition--if you want to unit test code that needs to do any of the first three things, well, you have to do them to test the code. I would say that a test for a network protocol that works by spinning up an echo server on a random port and having the code connect to that echo server is still a unit test for that network protocol.

In my definition, it would still be a unit test if it is fast and it talks to a database, a network server, or the file system, so long as such communication is done entirely in a "mock" fashion (it only does such communication as the test sets up, and it's done in such a fashion that tests can be run in parallel with no issues).

Re: The day I started believing in unit tests

#118

Post by throwaway290 »

eddd-ddde wrote: post #112
throwaway290 wrote: post #100

Earlier quoted context omitted.

Was this statically or dynamically typed language?

I don't think it really matters. Major bugs are not "oh this can be null", major bugs are "this combination of preconditions yield a business logic edge case that wasn't accounted for". Static typing doesn't really help more than dynamic typing in these cases.

It matters enough that the question "does static typing dramatically reduce the benefits of unit testing" is an open question or at least seriously discussed in the industry. All other replies are about dynamic languages.

Re: The day I started believing in unit tests

#119

Post by randomdata »

appplication wrote: post #116
mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily. This is also where the dogma of “only test public methods” fails. If your public method requires extensive mocking but the core logic you need to protect is iso…

> the most effective use of developer resources may be to just test your private method.

While there is nothing wrong with testing an internal function if it helps with development, so long as it clearly identifiable as such, you still need the public interface tests to ensure that the documented API is still conformant when the internals are modified. Remember that public tests are not for you, they are for future developers.

This is where Go did a nice job with testing. It provides native language support for "public" and "private" tests, identifying to future developers which can be deleted as implementation evolves and which must remain no matter what happens to the underlying implementation.

Re: The day I started believing in unit tests

#120

Post by m3kw9 »

coldbrewed wrote: post #94
m3kw9 wrote: post #74

How do you all feel about the need to rewrite a unit test when code gets refactored or business logic changes, isn’t that like a huge pita?

I treat unit tests like double-entry bookkeeping; I wouldn't describe it as a particular pain and consider it more of a matter of due diligence. Not everything needs this level of rigor but there are plenty of cases where the tests are very cheap to write and reason about (for many pure functions) or are worth the cost as they validate critical behavior. Unit tests also add some design pressure to keep more logic pur…

Thanks for that response, it was helpful esp about the side effects
Post reply on HN