Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The day I started believing in unit tests

mental-reverb.com

Post reply on HN
Print view View original

161–170 of 269 posts

Re: The day I started believing in unit tests

#161

Post by troupo »

BurningFrog wrote: post #153
troupo wrote: post #143

Earlier quoted context omitted.

> I don't know how my ways differs from yours, but I don't have much of the problems you mention. You haven't answered "How else are you "unit testing every single function and method"?" Given a medium-sized project and at least a passing and a failing test case for each function and method, you end up if not with hundreds, but with dozens of tests largely doing the same thing. > We do have some integration test, of…

> Given a medium-sized project and at least a passing and a failing test case for each function and method, you end up if not with hundreds, but with dozens of tests largely doing the same thing. I don't understand this comment. If I have one unit test for each function/method, that's just one test doing the same thing.

1. Your unit test probably shouldn't be testing several conditions at once [1]

2. Even if its just one unit test per function/method, even in a medium-sized project it's dozens of tests, many if them overlapping, with no idea if those functions/methods even work together correctly

[1] Depends on function/test

Re: The day I started believing in unit tests

#162

Post by randomdata »

MoreQARespect wrote: post #146
6DM wrote: post #106

Earlier quoted context omitted.

I have tried to evangelize unit testing at each company I've worked at and most engineers struggle with two things. The first is getting over the hurdle of trusting that a unit test is good enough, a lot of them only trust an end-to-end test which are usually very brittle. The second reason is, I think, a lot of them don't know how to systematically breakdown test into pieces to validate e.g. I'll do a test for null,…

I've evangelized against unit testing at most companies I work at, except in one specific circumstance. That circumstance is complex logic in stateless code behind a stable API where unit testing is fine. I find this usually represents between 5-30% of most code bases. The idea that unit testing should be the default go to test I find to be horrifying. I find that unit test believers struggle with the following: 1) T…

> The idea that unit testing should be the default go to test I find to be horrifying.

Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests. In practice, this means that a unit test won't break other tests.

I am not sure why you would want anything other than unit tests? Surely everyone agrees that one test being able to break another test is a bad practice that will turn your life into a nightmare?

I expect we find all of these nonsensical definitions for unit testing appearing these days because nobody is writing anything other than unit tests anymore, and therefore the term has lost all meaning. Maybe it's simply time to just drop it from our lexicon instead of desperately grasping at straws to redefine it?

> It seems to be tied up with some sort of idea that unit testability/DI just makes code inherently better.

DI does not make testing or code better if used without purpose (and will probably make it worse), but in my experience when a test will genuinely benefit from DI, so too will the actual code down the line as requirements change. Testing can be a pretty good place for you to discover where it is likely that DI will be beneficial to your codebase.

> The idea that test realism might actually matter more than test speed.

Beck has also been abundantly clear that unit tests should not resort to mocking, or similar, to the greatest extent that is reasonable (testing for a case of hardware failure might be place to simulate a failure condition rather than actually damaging your hardware). "Realism" is inherit to unit tests. Whatever it is you are talking about, it is certainly not unit testing.

It seems it isn't anything... other than yet another contrived attempt to try and find new life for the term that really should just go out to pasture. It served its purpose of rallying developers around the idea of individual tests being independent of each other – something that wasn't always a given. But I think we're all on the same page now.

Re: The day I started believing in unit tests

#163

Post by servaldeneptuno »

tarmon wrote: post #103
servaldeneptuno wrote: post #93

I have an unrelated (and most likely dumb) question about the article. When they talk about the inheritance relationship between 'Thread' and 'MyThread' in the example code in reference to the destructor methods, particularly here: > Now, what happens when MyThread::singlepassThreadWork() uses a member variable of MyThread like foobar and we delete the MyThread object while the thread is still running? The destructio…

https://en.cppreference.com/w/cpp/language/destructor Take a look at "Destruction sequence" but basically the destructors are chained together and called one after another to free all resources rather than forming one destructor for the derived object. That being said it is still effectively one object in memory.

Thank you for the explanation and reference.

Re: The day I started believing in unit tests

#164

Post by servaldeneptuno »

OvbiousError wrote: post #101
servaldeneptuno wrote: post #93

I have an unrelated (and most likely dumb) question about the article. When they talk about the inheritance relationship between 'Thread' and 'MyThread' in the example code in reference to the destructor methods, particularly here: > Now, what happens when MyThread::singlepassThreadWork() uses a member variable of MyThread like foobar and we delete the MyThread object while the thread is still running? The destructio…

You're right, the wording is confusing. It should be "parent class". There is only one object, a MyThread object. In C++ when an object is destroyed, all the destructors in the hiearchy run, from bottom to top. So first ~MyThread and then ~Thread. Anyway I think it is odd design to stop the thread in the destructor. You'd normally stop the thread first and then destroy the object, not the other way around?

I see now, thank you very much

Re: The day I started believing in unit tests

#165

Post by lamontcg »

mrweasel wrote: post #63

It is a little sad to see so many be so dismissive of unit tests. They aren't a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases. If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in i…

> If you need to mock out 80% of a system to make your unit test work, then yes, it's potentially pointless. In that case I'd argue that you should consider rewriting the code so that it's more testable in isolation, that will also help you debug more easily.

demands that people rewrite all their production code in service of unit tests are probably a big reason of why a lot of programmers don't unit test.

> On caveat is that some developers write pretty nasty unit tests. Their production code is nice and readable, but then they just went nuts in the unit tests and created a horrible unmaintainable mess, I don't get why you'd do that.

probably they write bad unit tests because they can't rewrite all their code but they have a mandate that all changes must be unit tested.

if strict purity could be relaxed and programmers were allowed to write more functionalish unit tests with multiple collaborators under test then there would likely be less resistance to testing and there shouldn't be any mocking-hell tests written.

higher level functional/integration tests also shouldn't be missed since your unit tests are only as good as your understanding of the interfaces of the objects and people write buggy unit tests that allow real bugs to slip between the cracks.

Re: The day I started believing in unit tests

#166

Post by randomdata »

DontchaKnowit wrote: post #102
jkubicek wrote: post #61

Earlier quoted context omitted.

> In actual practice, about 99% of unit tests I see amount to "verifying that our code does what our code does" and are a useless waste of time and effort. If you rephrase this as, "verifying that our code does what it did yesterday" these types of tests are useful. When I'm trying to add tests to previously untested code, this is usually how I start. 1. Method outputs a big blob of JSON 2. Write test to ensure that…

The problem with this for me is that most of the time "verifying that our ccode does what it did yesterday" is not a useful condition : if you make no change to code, its going to do what it did yesterday. If you do make a change to the code, then you are probably intending for it to do something different, so now you have to change the test accordingly. It usually just means you have to make the same change in 2 dif…

> then you are probably intending for it to do something different

If you have decided that your software is going to do something different, you probably want to deprecate the legacy functionality to give the users some time to adapt, not change how things work from beneath them. If you eventually remove what is deprecated, the tests can be deleted along with it. There should be no need for them to change except maybe in extreme circumstances (e.g. a feature under test has a security vulnerability that necessitates a breaking change).

If you are testing internal implementation details, where things are likely to change often... Don't do that. It's not particularly useful. Test as if you are the user. That is what you want to be consistent and well documented.

Re: The day I started believing in unit tests

#167

Post by RaftPeople »

randomdata wrote: post #162
MoreQARespect wrote: post #146

Earlier quoted context omitted.

I've evangelized against unit testing at most companies I work at, except in one specific circumstance. That circumstance is complex logic in stateless code behind a stable API where unit testing is fine. I find this usually represents between 5-30% of most code bases. The idea that unit testing should be the default go to test I find to be horrifying. I find that unit test believers struggle with the following: 1) T…

> The idea that unit testing should be the default go to test I find to be horrifying. Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests. In practice, this means that a unit test won't break other tests. I am not sure why you would want anything other than unit tests? Surely everyone agrees that one test being able to break another test is a…

> Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests

Kent Beck didn't invent the term "unit test", it's been used since the 70's (at minimum).

> I am not sure why you would want anything other than unit tests?

The reason is to produce higher quality code than if you rely on unit tests only. Generally, unit tests catch a minority of bugs, other tests like end to end testing help catch the remainder.

Re: The day I started believing in unit tests

#168

Post by randomdata »

RaftPeople wrote: post #167
randomdata wrote: post #162

Earlier quoted context omitted.

> The idea that unit testing should be the default go to test I find to be horrifying. Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests. In practice, this means that a unit test won't break other tests. I am not sure why you would want anything other than unit tests? Surely everyone agrees that one test being able to break another test is a…

> Kent Beck, who invented the term unit test, was quite clear that a unit test is a test that exists independent of other tests Kent Beck didn't invent the term "unit test", it's been used since the 70's (at minimum). > I am not sure why you would want anything other than unit tests? The reason is to produce higher quality code than if you rely on unit tests only. Generally, unit tests catch a minority of bugs, other…

> other tests like end to end testing help catch the remainder.

End-to-end tests are unit tests, generally speaking. Something end-to-end can be captured within a unit. The divide you are trying to invent doesn't exist, and, frankly, is nonsensical.

Re: The day I started believing in unit tests

#169

Post by randomdata »

feoren wrote: post #130
gorgoiler wrote: post #114

Earlier quoted context omitted.

I often think of unit tests as being programmable types, like Eiffel pre/post conditions or functional languages with types like Even and Odd. For example, in double(x) -> y you can use types to say x belongs to the set of all integers and y must also must be in that set, but that’s about all you can say in Python. Unit testing lets you express that y must be an even number with the same sign as x. It is like formal…

But you literally cannot possibly test that assertion for all x . Let's take a slightly harder problem: prove (or at least test conclusively) that for all integer x , the output y of the following function is always even: y = x^2 + x + 2 There is essentially no way to prove this for all x by simply testing all integers. If your integers are 64-bit, you don't have enough time in the lifespan of the universe. On the ot…

> But you literally cannot possibly test that assertion for all x.

Hence why he states it is formal verification for the "unwashed masses". The "washed" will use a language with a type system that is advanced enough to express a formal proof, but most people can't hack it, and thus use languages with incomplete type systems and use testing to try and fill in the gaps.

Re: The day I started believing in unit tests

#170

Post by dllthomas »

gbacon wrote: post #30
teeray wrote: post #2

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly…

From Working Effectively With Legacy Code by Feathers, p. 14[0]: Unit tests run fast. If they don’t run fast, they aren’t unit tests. Other kinds of tests often masquerade as unit tests. A test is not a unit test if: 1. It talks to a database. 2. It communicates across a network. 3. It touches the file system. 4. You have to do special things to your environment (such as editing configuration files) to run it. Tests…

I've never liked the conflating of target size and test time constraints.

I very much agree that there's benefit to considering the pace of feedback and where it falls in your workflow; immediate feedback is hugely valuable, but it can come from unit tests, other tests, or things which are not tests.

Meanwhile, some tests of a single unit might have to take a long time. Exhaustive tests are rarely applicable, but when they are it's going to be for something small and it's likely to be slow to run. That should not be in your tightest loop, but it is probably clearer to evict it simply because it is slow, rather than because it is not a unit test for being slow.

Post reply on HN