When to Mock

Panic closed the tab when a giant form popup appeared asking for my email. Hate to add noise to the discussion, but I hope the author sees this and understands the market doesn't want this and reduces readership.

> Retrieving data from the database is an incoming interaction — it doesn’t result in a side effect.

Unless you count the time taken to retrieve the data as a side effect. If you are implementing a cache with a well specified behavior then you might want to test incoming interactions.

Anyway, this just shows that having a side effect can be a matter of perspective.

Right, the testing pyramid. Your largest set of tests should be a foundation of unit tests that run quickly. Then integration and or functional tests, which is the section this is talking about.

They are still necessary, then move up the stack and you get into the end-to-end tests of a fully running system.

The reason isn’t only speed but also signal to noise ratio. The further up the testing pyramid you go, the less clear it becomes where errors were introduced.

Same. I wish HN had a filter I could set up to hide entries form domains that do this. Won't catch one offs like this but I can add it along with NYT and other paywalls so I don't waste my time.

> Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test?

So what we do where I work is for unit tests, we mock services and repositories, unit tests don't go anywhere near the database.

For integration tests, we use an in memory database.

BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when running the application. There are SO many pitfalls, nasty bugs, and other warts along that road. EntityFramework alone has quite some weirdness there. Expect these kinds of integration tests to cost a lot of developer effort to build and maintain. For us, it took months of effort to get these kinds of tests working usefully.

Personally after working on applications that have a solid test pyramid, I would recommend:

* Write unit tests as per the standard advice (bottom of pyramid kind of quantity), but try and keep it sane (don't go for 90% coverage just for the sake of hitting an arbitrary number; don't pointlessly unit test your framework/libraries/other dependencies)

* Write some integration tests where they really add value (interaction between 2 or a few complicated components in your system, for example places where the state of a component changes a lot depending on input from another component). Make sure when you start out writing an integration test that it doesn't turn into an "almost" end-to-end test along the way. They have a habit of doing this and it can really cost you later. Integration tests should still be focused.

* Write end-to-end tests that test as much of your system as possible, including a database (preferably the same vendor; one approach is to truncate all tables before each run). IME it is very good here to have one e2e test that covers a big chunk of functionality in one run, than lots of separate tests covering different things. Why? Because no matter how "different" the things being tested by the e2e test there will still be LOTS of overlap by its very nature, and changing anything where that overlap is will tend to break ALL your e2e tests. Not a fun workflow.

One last comment on E2E tests. This is more opinionated. But try and limit E2E tests to things that are really business critical (e.g. your "sign up" process and your "renew subscription" process, but not every single form in your dashboard). This is just a cost-benefit thing. E2E tests help, but they also slow you down, and the more you have, the more slowly you will be able to change your software. Sometimes minor bugs slipping through is an acceptable cost if it means you release 1 week sooner.