Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Write tests. Not too many. Mostly integration

blog.kentcdodds.com

Post reply on HN
Print view View original

211–220 of 338 posts

Re: Write tests. Not too many. Mostly integration

#211

Post by Jach »

I don't practice TDD (see Norvig, Jeffers, and Sudoku) but I do like unit tests. I also like integration, e2e, property, mutation, and fuzz tests. I like proving specs with TLA+. Sometimes type theory proofs are useful. There's a lot of ways to improve quality. Some of those I've only been able to toy with personally, though, because the dirty secret of software engineering is that for most things we don't need very high levels, we just need most customers to not be mad. So that leads to articles like this, where people claim their experience showed the most bang for buck with a certain approach, despite not always even trying other approaches...

> One thing that it doesn’t show though is that as you move up the pyramid, the confidence quotient of each form of testing increases. You get more bang for your buck. So while E2E tests may be slower and more expensive than unit tests, they bring you much more confidence that your application is working as intended.

In my experience this is wrong. Working with an inverse pyramid, you'd think confidence would be high, bugs few. It's the exact opposite.

> just stop mocking so much stuff

Here here. But this gets into subtle arguments over "is this really a unit test if it's using RealFoo even though we're trying to test Bar, such that if RealFoo breaks this test will also break but not because of Bar?" I'm not too strict on my definition of unit test, my best attempt is something like "relatively small, isolated from the broader module/library/application, runs fast, easy to find root failure when test fails, avoids testing the implementation rather than behavior (often hard), and asserts something." It leaves open the possibility for technical 'integrations' but there's a pretty big space of possibilities between a minor integration to avoid mostly useless mocking and suddenly requiring the whole application server to have started up before you can do anything.

Re: Write tests. Not too many. Mostly integration

#212

Post by laurent123456 »

On a hobby project recently I've only written backend integration tests. I found these to be extremely useful though since they touch directly or indirectly to all the critical parts of the software, so if something's broken it will eventually be caught there. Also since it's relatively high level there's also rarely a need to change them whenever I change something to the backend. All in all it's definitely a time saver, both in terms of catching bugs and maintaining the tests.

I think there's still a value in detailed unit tests but mostly for library code, when you want to test each function properly with various inputs.

Re: Write tests. Not too many. Mostly integration

#215

Post by titzer »

fersab wrote: post #160
kartan wrote: post #146

Earlier quoted context omitted.

> What in the. serious. fuck. Of course tests test implementation details. Because _implementation details_ are where the goddamn bugs are. Testing is about testing inputs and outputs, not implementations details. Bad testing checks if you called this function or if you accessed this data. Good test checks that for a given input you get the expected output. > Please don't follow the advice of this. It's total crap. T…

I totally agree with you on this one! The tone is absolutely immature and frankly disgusting. I would never hire a person with this kind of attitude, frankly because it is the ultimate creativity-killer in a team. This type of attitude is creating a culture where everyone is "to scared to learn" by being to scared to comment on things, or even to ask a simple question. So sad to see this...

Sorry for the harsh tone--probably an overreaction--but I think creativity is not the primary variable to optimize for in software development. I recognize that this opinion is a product of where I've gotten stuck in the software stack.

However the OP's advice was basically "don't write tests because they slow you down." They even said that 100% code coverage was "a really bad idea". The OP's attitude was flippant, dangerous, and IMO, stupid. Don't follow this advice.

Re: Write tests. Not too many. Mostly integration

#216

Post by 3pt14159 »

kartan wrote: post #149

So much people in this thread is talking about different domains and are not able to see that they need different rules. It is not the same creating a library that is going to be used to launch a multi-billion rocket to Mars than developing a mostly graphical mobile app where requirements are changing daily as you A/B test your way into better business value. The article has really good points and the reasons why the…

I've programmed in many domains. There are only three places where I don't use heavy testing:

1. Very, very graphical programming - like SVG charting with animations. If it were static generation it would be easy, but throwing time into the mix makes the tests really hard plus if things go wrong in the future people can literally see it going wrong and complain, so I don't think it is worth the trouble.

2. Data analysis meant for static reporting. You know, those 2000 line SQL queries that barf out data that you pop into excel to munge through before typing up a 20 pager for upper management.

3. Small personal tools, like a CLI script that spits out equivalent yearly interest rates or what have you.

Everything else I test. Libraries, backend web apps, machine learning shit, compiled, whatever. It is too easy for codebases to turn into a hellscape without tests. You get too afraid to change things.

Should it change the API to the codebase? I usually don't think so, but occasionally I'll, say, make something an instance variable that I'd normally keep as a locally scoped variable. So I model test what I can and I integration test the rest. I think that he's right that integration tests cover a lot of functionality and I think he's right that mocks aren't usually great, but I think he's wrong about how much testing we should be doing. Most things should be tested most of the time.

It's cheaper. Why?

Because it costs money to hire support people and it costs money to validate their tickets and it costs money to fix the bugs and the bugs are harder to fix when the data is already in the system and the data is wrong.

It's also more profitable to write more tests. Why?

Because bugs mean lost customers and even when you keep the customer the feedback that you get from them is which problems you need to fix, not which functionality could be made better.

Re: Write tests. Not too many. Mostly integration

#217

Post by hacker_9 »

vertex-four wrote: post #205
hacker_9 wrote: post #182

Earlier quoted context omitted.

That's exactly my point, mock your external dependencies. Static calls don't allow you to do that.

So how exactly do you test that your SQL query does the right thing? That you're using the Twitter API correctly?

Testing a database, or an external web service, is an integration test. They can be as simple as:

    void TestCreateUser() {
        var repo = new UsersRepository();
        var mockUser = new User("John", "Smith");
        repo.AddUser(mockUser); // db call
        var addedUser = repo.GetUsers().Single(); // db call
        Assert.StructureIsEqual(mockUser, addedUser);
    }
For the Twitter web service, you might test that you successfully get a response, as you don't have control of what exactly comes back.

Re: Write tests. Not too many. Mostly integration

#218

Post by hacker_9 »

seanmcdirmid wrote: post #198
hacker_9 wrote: post #194

Earlier quoted context omitted.

To be fair, TDD is quite good for exploratory programming, as it makes you think about the intent or your API up front.

Not really. It makes you commit to an API upfront, this is the exact opposite of what exploratory programming should be (noncommittal, keep everything open).

No with TDD you don't need to go in with a structure in mind, the structures arise as you write more tests and get a proper understanding of what components you'll require. Red, green, refactor - each refactor brings you closer to the final design.

Re: Write tests. Not too many. Mostly integration

#219

Post by toasterlovin »

oweiler wrote: post #12

As always it depends. There are projects where integration is the hard part and others where the business logic is impossible to get right without unit tests. Unit tests also have the added benefit that they tell you exactly the reason they failed (because there can only be one) whereas integration tests can fail for multiple reasons.

Yeah, but usually an integration test tells you what line it failed on, which gets you fairly close to knowing the exact reason it failed.

Re: Write tests. Not too many. Mostly integration

#220

Post by whack »

hinkley wrote: post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

You've missed the point. Sure, code can always be written better to facilitate testing, but ultimately, each component of the code still has to correctly call/be-called by other components. No class exists in a vacuum. Suppose you have class-A which interacts with class-B. I've seen people put a ton of effort into unit-testing A and B in isolation, and writing very elaborate mocks/fakes/stubs for A and B. Only to end up with bugs anyway because they made a mistake in their mock/fake assumptions. Instead, an integration test that allows A and B to interact directly, and tests their resulting behavior, would avoid all this wasted effort and bugs that come from mocking.

You suggest that instead of writing integration tests, this problem can be avoided by "writing better code". But how exactly would you rewrite the code to avoid the above problem? Declare that A and B should not interact at all, and move all their interactions into class-C? Now you've just given a new name to the same problem: "How do we adequately test class-C?" And once again, the correct answer is to ease up on the mocks and just write some integration tests.

Post reply on HN