31–40 of 178 posts
This blog post has tell us that we shouldn't use mocks but should instead "send data" to the code we are testing. I think this is supposed to mean using dependency injection instead. But the case isn't really made. Instead, with a waving of hands and wild assertions such as that I'm lying to myself, I'm left wondering what I just read. The previous post was on cargo cult programming. It warns that CCG is bad but can'…
This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…
https://github.com/adobe/S3Mock
It's more realistic than using mock objects/function calls and requires less maintenance.
This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…
For AWS specifically, I prefer to have an AWS account specifically dedicated to each service + stage. For example, if I have an image service that handles s3 uploads (say Lambda, S3, Cloudfront and API Gateway), then I'd deploy a "test" environment to a dedicated AWS account and run tests against that. Since it's fully serverless, it only costs a few pennies to test (or free). I try not to develop locally at all anym…
Not only that, but you now need network access to run tests. A network blip or a third party service outage now makes your tests fail.
There is also the possibility that an aborted test run might leave state in s3 that you are now paying for. Someone hits Ctrl-c during a test run and now you have a huge AWS bill.
The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally. Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a fe…
>The author seems to believe people either mock everything or don't mock anything. The author is saying that people frequently mock things that it would be more economic to just run because you've got the real thing right there. Building a model for it is an expensive waste that probably won't even match reality anyway and will demand constant maintenance to sync up with reality. If you're overtly concerned with the…
Having a set of very fast running tests is absolutely necessary in my opinion.
Once I have validated that the piece of code I wrote is doing what I intended, then I want to run other tests that do not use mocks/fakes, e2e tests that can possibly take a whole day to complete and will allow me to see if the whole system still works fine with my new feature plugged in. But this comes AFTER fast unit tests, and definitely cannot REPLACE those.
Earlier quoted context omitted.
>The author seems to believe people either mock everything or don't mock anything. The author is saying that people frequently mock things that it would be more economic to just run because you've got the real thing right there. Building a model for it is an expensive waste that probably won't even match reality anyway and will demand constant maintenance to sync up with reality. If you're overtly concerned with the…
When I am developing a feature, I want to know very fast whether or not my code's logic is correct. It is not rare during the development cycle to run the same test dozens of times because I made a silly mistake (or a few), and obviously if the test takes 30 minutes to complete it completely wastes my day of work. Having a set of very fast running tests is absolutely necessary in my opinion. Once I have validated tha…
This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…
For AWS specifically, I prefer to have an AWS account specifically dedicated to each service + stage. For example, if I have an image service that handles s3 uploads (say Lambda, S3, Cloudfront and API Gateway), then I'd deploy a "test" environment to a dedicated AWS account and run tests against that. Since it's fully serverless, it only costs a few pennies to test (or free). I try not to develop locally at all anym…
Local development is the easiest way to avoid wasting money and resources on debugging/development.
This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…
Yes, use stubs. But then also have integration tests. The point is to have easier to write lower overhead unit tests, and then have a few full fat integration tests that put everything together. Mocking is a terrible middle ground.
Earlier quoted context omitted.
When I am developing a feature, I want to know very fast whether or not my code's logic is correct. It is not rare during the development cycle to run the same test dozens of times because I made a silly mistake (or a few), and obviously if the test takes 30 minutes to complete it completely wastes my day of work. Having a set of very fast running tests is absolutely necessary in my opinion. Once I have validated tha…
Yeah, he's talking to you for sure.
The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally. Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a fe…
I've seen a lot of tests where people just mock everything by default without thinking. Smart programmers at a good company. It's an issue that does deserve more recognition. Abuse of mocks is bad for tests.
This blog post has tell us that we shouldn't use mocks but should instead "send data" to the code we are testing. I think this is supposed to mean using dependency injection instead. But the case isn't really made. Instead, with a waving of hands and wild assertions such as that I'm lying to myself, I'm left wondering what I just read. The previous post was on cargo cult programming. It warns that CCG is bad but can'…
>I think this is supposed to mean using dependency injection instead. I'm pretty sure he means "just use integration tests".
Unfortunately, integration tests are too slow, so the practice doesn't scale if one is trying to TDD.
Insinuating integration testing into every user story will lead to friction. Test run times will balloon, cycle times will get extended and resentment will grow for the test suite and the team's testing regime.
If your test suites cannot complete quickly (seconds), then they cost more than they're worth. I've learned this about outside-in TDD at-scale. Our code quality is glorious. But our test run times are untenable.
I'm experimenting with sociable tests to curb my appetite for integration tests or at the very least keep on writing them but make it safe/productive to run the vast majority of them on the CI/CD server only.