The testing pyramid should look more like a crab

This got so long I decided to split it in two:

It is important to note you can't pull all of the tests down. I tend to use a plumbing analogy that I probably have stolen from the literature or a mentor - It's likely that QA already tested all of the pipes at the factory before they were installed in your house. But every plumber will turn on the taps and flush the toilet at once before they leave, to make sure the stuff going in one end actually comes out the other (and only the other). Test all the bits in isolation, but always leave at least one happy and one unhappy path test going through the whole stack to make sure good and bad answers end up in the right place. I think here the pyramid analogy is a bit wrong, because in the pyramid there's a geometric reduction at each level, but I have a suspicion that in a best case scenario, you can achieve a log scale reduction (because your E2E tests can potentially reflect the height of the decision tree, rather than the surface area of a module)

Tests higher up the pyramid rarely if ever suggest, let alone demand API improvements. If the first person writes elaborate mocks instead of simplifying the design to not need them, they've locked in the implementation in a way that unit tests don't (Sunk Cost). This can turn a masochistic programmer into a sadist.

And that is the trap most of us fall into the first few times, and some never outgrow. It's easier to write greenfield E2E tests, but they're hell to maintain, and the create a codependent testing relationship, where the existence of the tests dissuades you from engaging in healthier activities. I almost think that you should ban E2E testing frameworks until the team has gotten comfortable with Unit Tests because of this. With E2E, you can get really far down the wrong road and find yourself in a blind alley that you now lack the imagination or conviction to escape, because everything else feels remedial. That is the main problem with unit tests - they feel like some sort of grade school activity. Doing something so simple feels like doing multiplication tables, which is kids work. I am a smart and functional adult, I shouldn't be doing something so boring. This is making me feel dumb.

It takes a special disposition to enjoy a simple task for what it is, and many software developers - myself included - were lured into software by the idea of automating any task that is so simple. Rarely do I let production code be so dominated by idioms with so little meat, so why do I enjoy that in tests? Honestly, I don't have an answer, except "better in the tests than in the production code," which is hardly an answer at all.

And importantly, all of these uneasy feelings are used as evidence by anti-testing people that it's too painful and too little gain and we should stop doing it. A form of learned helplessness.

I write a bunch of e2e tests, and not many unit tests. I'm on the fence about if this is a good idea or not. I feel like when you are starting at zero, you need more e2e tests, before writing more unit/service tests as your product is solidified.

I wish e2e tests were faster. I feel like this should be achievable, but it's a problem way out of my depths. I feel like you could run a browser within a docker container at 2x or 5x speed with enough virtualization. I've been told this would not work in practice.

e2e testing is the way to go. But our current tech stack is really bad for testing. Testing in browsers seems an afterthought, browsers first were designed for features, later on debugging was added, quite good nowadays, but testing is out of the scope. Until we get easy to test browsers everything will continue being fragile and bug prone.

Pyramid approach, inverted pyramid, crab shaped does not make sense because it tries to be catch all guideline. Same with any 'maturity models' for software projects.

People are forgetting there is something like 'risk based' approach to testing. This is the only way to allocate budget effectively, where stuff that can drop half of your database should be tested well and stuff that will make your UI look funny probably less.

I'm co-founder of an end-to-end testing product, but I don't actually agree that better end-to-end testing coverage means you should create less unit tests. The reason why is because there's a class of things you want to test that are much better served at the unit testing level. Unit tests are always going to be faster to execute and by definition have a much smaller scope, which is helpful in reasoning about what you're testing.

Our product is really focused on making end-to-end tests much easier to create and maintain. Here's an example video I recorded yesterday of automating Monday.com's sign-up flow using our product: https://www.loom.com/share/1336a5bfa9f54f2190269961991802b5 - looking at this I don't really see what unit tests this would replace, rather I think it gives coverage for things that you'd often have no automated coverage for - i.e. the interplay between disparate components along with interacting with email - basically the steps that form a complete workflow from the user's perspective.