The testing pyramid should look more like a crab

Everyone wants to be a crab https://www.popularmechanics.com/science/a34389129/crab-evol...

> Well, the unit tests can find you a few logical errors, which is great. I write unit tests for that all the time. But all possible sources of error are discovered by end-to-end tests.

Citation needed, sounds anecdotal. If this really is how your app behaves then you're going to run into problems eventually because end-to-end tests are slow as well as difficult to write+maintain, and you'll need hundreds (if not thousands) of them to cover all the important business logic in your app. This will slow down your developer productivity (PRs take long to merge because people wait for CI to finish) and ability to deploy quickly.

If your code is well-structured, your important business logic should be easily testable with unit and/or integration tests, and you don't need an end-to-end test to discover that, for example, a calculation is wrong. The pattern of functional core/imperative shell embodies this really well, I think.

I’ve seen the same at several different companies. There seems to be just too much that can go wrong during an e2e test like this: network failures, slow CPU causing timeouts, etc. At one company, the entire suite of 700 tests was re-run 3 times nightly, taking 5+ hours, and some tests still failed randomly.

I have found that the majority of errors can be caught by unit and integration tests on a single service, and that the extra burden of maintaining Cypress just isn’t worth the small amount of coverage.

End to end tests are less precise to triage when they break, and they break more often than unit tests. As the author said, they show all bugs, so they break a lot.

When the test is broken, it's not providing coverage until it's fixed. Which is why I advocate for less assertive testing in that case: https://assertless.org/

The priority of fixing end to end tests becomes critical, and so test maintenance is more important. With a large test suite of functional tests, that effort is not sustainable in my experience. I've been there. An image comparison suite of tests has the same issue, except it may defer all assertions, which is good, but it offloads a huge amount of analysis to the engineer consuming all the output. You really do need a human to check those tiny style changes it finds, a human with extensive knowledge of the product and it's ever-changing visual quirks. I also maintain a large visual testing suite like that.

Cypress is not different from Selenium in that aspect. (There's not much real difference at all that I can see.)

I think end to end tests have more of a value than the traditional testing pyramid would indicate, but there's pitfalls and benefits to each type. A suite of concise unit tests will find obvious bugs with precise granularity incredibly quick on the code they cover, which can never be complete coverage, but it doesn't need to be. This can be used for things like commit hooks that end to end tests just can't.

Don't forget that generally e2e tests take longer to run and typically require more test assets.

For example, if e2e tests involve simulations running in real time, then by increasing the number of distinct e2e tests you also increase the requirement on test assets and/or overall CI duration.