Mocking is useful for testing small parts of your programs/libraries. For full-scale testing you really need to not emulate because any emulation will be woefully incomplete, so you're going to have to spin up a virtual network with all the services you need including DNS.
How to test without mocking
51–60 of 225 posts
Re: How to test without mocking
#52> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
I think writing tests as a form of documentation is a waste of time. If I'm using a component I don't want to read unit tests to figure out what it should do. Unit tests are most often used to cover a few more lines that need coverage. That's the value they provide.
Documentation for other developers of that code, not users of that code.
Re: How to test without mocking
#53> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
I think writing tests as a form of documentation is a waste of time. If I'm using a component I don't want to read unit tests to figure out what it should do. Unit tests are most often used to cover a few more lines that need coverage. That's the value they provide.
A well designed API will generally allow users to understand usage without any additional documentation, sure. However, those who modify the API in the future will want to know every last detail that you knew when you were writing it originally. That must be documented to ensure that they don't get something wrong and break things – and for their general sanity. That is, unless you hate future developers for some reason.
You could do it in Word instead, I suppose, but if you write it in code then a computer can validate that the documentation you wrote is true. That brings tremendous value.
Re: How to test without mocking
#54> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
> Unit tests aren't meant to find bugs, they're meant to protect against regressions
That hasn't been the general consensus on unit tests for at least 30 years now. Regression tests are a small subset of tests, typically named for an ID in some bug tracker, and are about validating a fix. The majority of unit tests catch issues before a bug is even opened, and pretty much any random developer you talk to will consider that to be the point.
Re: How to test without mocking
#55Earlier quoted context omitted.
Testing both your code and the message system is exactly what you want, since if the message system is broken in a way that upstream didn't catch, you want to learn about it during testing and not production, if possible.
Keep in mind that there are different kinds of testing. What Beck called unit tests and integration tests. Unit tests are really for purposes of documentation . They show future programmers the intent and usage of a function/interface so that others can figure out what you were trying to do. Mocking is fine here as future programmers are not looking to learn about the message system here. They will refer to the messa…
Unit tests as a form of example code based documentation is where I could see unit tests complimenting documentation, yes.
However, depending on the industry, code coverage is a valuable tool to gauge the maturity of the software baseline and burning down software execution risk. One example of this is Airworthiness or Safety Critical Code.
Re: How to test without mocking
#56The catch-22 with refactoring to be able to write unit tests is that refactoring introduces risk as you are changing code, and you need tests to help reduce that risk. But you can't easily write tests without refactoring. This has been a very difficult problem for the team I'm currently on. The only strategy I'm aware of is described in `Working Effectively With Legacy Code`, where you start by writing throwaway unit…
Why get rid of working e2e tests? IMO they are more useful than unit tests at finding the kinds of problems that stop a release/deployment.
You can attack from both directions: e2e tests make sure that certain processes work in fairly ordinary situations, then look for little things that you can unit test without huge refactoring. When you've pushed these as far as you can, section off some area and start refactoring it. Do your best to limit your refactoring to single aspects or areas so that you are never biting off more than you can chew. Don't expect everything to become wonderful in one PR.
Your e2e tests will catch some errors and when you look at what those commonly are then you can see how to best improve your tests to catch them earlier and save yourself time. In python I had stupid errors often - syntax errors in try-catch blocks or other things like that. If I used a linter first then I caught many of those errors very quickly.
I was working on a build system so I mocked the build - created a much simpler and shorter build - so I could catch dumb errors fast, before I ran the longer e2e test on the full build.
IMO you need to progress to your vision but trying to reach it in one step is very dangerous. Make life better piece by piece.
You can even do PRs where you only add comments to the existing files and classes (not too much detail but answering questions like "why" is this file/class here). This helps to make sure you really understand the current system is doing before you change it.
I once added type hints everywhere to a legacy python program - it wasn't as helpful as I'd hoped but it did prevent some issues while I was refactoring.
Re: How to test without mocking
#57I don't think mocking is an anti-pattern. Using only unit tests and then mocking everything probably is. Mocks have a perfectly viable place in testing. They help establish boundaries and avoid side effects that are not pertinent to the logic being tested. I would reference the testing pyramid when thinking about where to be spending time in unit tests vs. integration tests vs. end to end tests. What introduces risk…
I think the better advice that goes with the spirit of the article is, prioritize integration testing over unit testing if you're constrained on time.
Re: How to test without mocking
#58Go ahead. Don't mock that external service that you rely on for an API. Now you need to have multiple keys, one for each developer, or share keys separate from various environments? Does it not offer dev/test/staging/prod keys? Well, now you need to share those keys. Does it only offer Prod keys? Now you are stuck sharing that. API request limits? Now you are eating through that just to run tests. And let's not forge…
Also the intermittent failures of your tests relying on unstable dependencies.
Re: How to test without mocking
#59If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Also, don't create any interfaces you wouldn't mock. I've seen too many people waste months creating some kind of "database wrapper."
Re: How to test without mocking
#60> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
There's a few issues with this IMO:
1. Changes often require changing the functionality of a component, which means many of the current unit tests are bunk and need to be updated. Changes that are simply refactoring but should retain the same behavior, need to update/rewrite the tests, in which case again often requires significant refactoring of the existing tests.
2. Small isolated changes usually require testing everything which in a big org is very time consuming and slows down builds and deploys unnecessarily.
3. A lot of false confidence is instilled by passing unit tests. The tests passed, were good! Most of the production bugs I've seen are things you'd never catch in a unit test.
I really can't imagine a large refactor where we wouldn't end up rewriting all the tests. Integration tests are much better for that imo, "units" should be flexible.