Live data from Hacker News

Write tests. Not too many. Mostly integration

blog.kentcdodds.com

21–30 of 338 posts

Re: Write tests. Not too many. Mostly integration

#21
post #4
post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

Sounds like Tableau in Seattle.

Re: Write tests. Not too many. Mostly integration

#22
post #4
post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

For some reason, unit tests vs. integration tests reminds me of this image: https://pbs.twimg.com/media/CZX0O-tWQAAeaLi.jpg The unit tests past, but why bother with integration tests?

I wrote a component at work. Sure, there are some unit tests (the SIP parser working? Check. But for that component, unit tests only go so far as I need to query another program that actually implements the business logic (it goes SIP -> my program -> custom protocol [1] -> business logic program and back again). To mock the business logic program (I need to make sure to return the proper information per the SIP request) is to reimplement the business logic program, so when testing my component, we also use the business logic unit. The business logic also requires two more programs to run. At this point, there is no difference between a unit test and an integration test as I'm running five programs ... no, ... six, I do have to mock a cell phone (basically, respond to a custom protocol and make a web request), to test the "unit" that is my program.

Oh, and to make it even nicer, this is legacy C and C++ (C with classes) code.

[1] Legacy code. It works. At the rate of production deployments we (our team) gets, it would be around two years [2] to remove the custom protocol. So it stays.

[2] Have I mentioned the very scary SLAs?

Re: Write tests. Not too many. Mostly integration

#23
post #18

Earlier quoted context omitted.

> Good lord. Why integration tests? Because they can find bugs and errors which unit tests cannot.

This is true. Areas where I have full unit test coverage tend not to have any bugs. What a waste of time to have written all these tests!

Yeah, assuming you think it's not a bug when the backend API that you're mocking is accidentally removed or changes its response slightly.

Re: Write tests. Not too many. Mostly integration

#24
post #13

Why does everyone rethink a working strategy. Write lots of unit tests that are fast. Write a good amount of integration tests that are relatively fast. Write fewer system integration tests that are slower. The testing pyramid works. He even talks about it in this post, and then ignores the point of it. You write lots of unit tests because you can run them inline pre-commit or in a component build. If you integration…

Because for certain kind of project the strategy stops working. I work as QE on a fairly large, ~7 years old project. Micorservice architecture has been attempted. We always merge to master, which means that everything more-or-less is a feature-branch merge. We have too many repositories to count. And what we learned is, that most of the components we have are just too thin to allow for useful unit-test coverage. Alm…

So, if I understand it correctly, Middleware and Backend should have been single component since it's one bounded context and splitting it makes one of those feature envy? Is there some benefit keeping these separate or is the cost of change too high at this point? If it's not about features, but more about API, have you tried Consumer-driven contract testing approach?

Re: Write tests. Not too many. Mostly integration

#26
post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

No true Scotsman.

You might be speaking about those cases where people write large god classes, pervasively side-effectful code, zero API design, and a general lack of pure abstractions - then their tests would equally be bad. Tests are code, so one's ability to design programs would reflect on their tests and vice versa.

But a reasonable programmer who cares enough about what they do can still end up with a brittle test suite because of other factors.

Unit tests written in a dynamic language is a major drag on refactoring. It is not so much that we test the internals of the system or get tied up with the shape of code internal to an object. Even if you follow Sandi Metz's wonderful guidelines around testing: i) do not test internals, ii) test incoming queries by asserting on their return value, iii) test incoming commands by asserting on their side effects, iv) mock outgoing commands, and v) stub outgoing queries, you end up with a brittle test suite that is hard to refactor thanks to connascence.

Whenever you refactor your names, or shuffle the hierarchy of your domain elements, you are now left with a thankless chore of hunting and pecking your unit tests and making them reflect the new reality of your code. Here integration tests help you know that your system still works end to end, and unit tests simply remain that one thing that refuses to budge until you pay it its respects.

Unit testing complex views is still a hard problem. There are no well-defined stable "units" to speak of in an ever changing HTML UI. We have snapshot tests, we try to extract simple components on whom we can assert presence/absence of data, and we have integration tests that blindly railroads over everything and make sure the damn thing worked.

But in a different context unit testing is the one true answer. If your system is statically typed (say in Haskell or OCaml), and your functions are pure and compositional, you don't so much worry about mocking and stubbing. You can make simple assertions on pure functions and as the granularity of your functions increase, they end up covering more parts of your system and get closer to an integration test. Static types form another sort of guarantee, the most basic one being that it takes a class of bugs away in form of undefined data types, the system going into invalid states, and of course the clerical mistake of named connascence. We often abuse unit tests in dynamic languages to cover these scenarios, leading to huge test suites with very brittle tests.

I think it is important to call out that the value of unit tests are still contextual - "it depends" like everything in the world, and despite our best efforts, they can become hard to refactor. There is a case to be made for writing integration tests because they deliver more business value at a cheaper price than a pervasive set of unit tests when dealing with a highly effectful dynamic system. This lets us also think about other forms of testing like generative testing and snapshot testing that come to the same problem from different angles.

Re: Write tests. Not too many. Mostly integration

#27
post #4
post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

It depends on what you unit test and why.

If it's for 100% test coverage: forget about it.

If you test private methods: you're doing something wrong.

What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Then you do some refactoring and you have to rewrite tons of tests. And as those tests are just made to get 100% coverage you end-up with logic bugs because most unit tests have been written to go through the code, not check limits and edge-case. When you stumble upon this kind of test harness you can only think this as only cons (more to write upfront, less willingness to refactor) and no pro (the code is still brittle). Then your integration tests feel like your real harness: you can change anything in your code it'll tell you what has been broken when used.

Now if you consider your unit tests as a kind of integration tests for the API your classes present then you get the benefits of unit tests. But this mean testing only public methods. And mutation testing resilience is a better metric than test coverage.

Also: those tests do not replace a real documentation which can be a lot faster to read and understand than code.

Re: Write tests. Not too many. Mostly integration

#28
I feel like there's definitely an art to mocking. Rather than mocking out the module being called (A calls B, A_test mocks B), you want to mock out its dependencies (A calls B calls C, A_test mocks C). But if you swap out said dependency, now you have to go update other modules' integration tests to mock out the new service.

Re: Write tests. Not too many. Mostly integration

#29
post #4

Earlier quoted context omitted.

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

Perhaps a well written, easy to follow guide on how to structure different types of code to simplify testing would be helpful. Know of any?

Working Effectively with Legacy Code is a good read: it presents what kind of code you want to attain and methods to get there from a crappy code base.

The definition of legacy code for the author (which I like) is: untested code. So the book is more about getting code in a testable state than random refactoring to get to Clean Code level.

Re: Write tests. Not too many. Mostly integration

#30
post #23

Earlier quoted context omitted.

This is true. Areas where I have full unit test coverage tend not to have any bugs. What a waste of time to have written all these tests!

Yeah, assuming you think it's not a bug when the backend API that you're mocking is accidentally removed or changes its response slightly.

You seem to be implying that I prefer all unit tests and no integration or system tests. Far from it.

A system test catches the backend API issue. A unit test can demonstrate that my component degrades gracefully if the backend API is not available (because I used a mock to provoke a timeout response).

Post reply on HN