The testing pyramid should look more like a crab • Hacker News Board

Re: The testing pyramid should look more like a crab

#61

Post by nullsense » Wed, Nov 04, 2020, 5:59 AM UTC

hinkley wrote: post #20

By moving the 'meat' of the tests up high into the hierarchy, the author has just re-invented the testing ice cream cone with a different flavor. In software, we can make pretty much any process work for 18 months, before it starts to fall apart. If you don't stay at a company for at least a couple of years after they start doing testing in earnest, you won't really see that what you're doing doesn't scale/isn't resi…

What we really need is a giant flow chart that factors in every single thing that should go in to the calculus of how to structure your testing.

Blanket strategies and approaches are just never going to be right for all codebases and companies. There probably are aggregate rules which kind of generalize, but I think the actual system under test and the organization it sits inside is an incredibly important consideration that needs to be taken into account.

I'd love to see how everyone's different advice pans out for all possible permutations of org and codebase they could be tried against.

Re: The testing pyramid should look more like a crab

#62

Post by contravariant » Wed, Nov 04, 2020, 8:03 AM UTC

nullsense wrote: post #60

contravariant wrote: post #42

Earlier quoted context omitted.

Is that a benefit of end-to-end tests? Seems more like encouraging more unit tests would discourage bad coding habits if anything.

As always in these discussions I like to remind everyone that you can have good and bad implementations of everything, and just doing unit testing doesn't make your engineers suddenly understand how to write testable code. So, you can well end up in a situation with a reasonable amount of code coverage, but not quite as high as you might like and when someone invariably digs into why they find while parts of the syst…

Yeah there's no silver bullet for any of these problems probably.

That said an end-to-end test seems just as likely to merely test implementation details. You could have all kinds of changes in behaviour without necessarily doing anything incorrect.

I suppose the point is that you need to test where you can define what correct behaviour is. And if this is only possible for end-to-end tests then something is screwy.

Re: The testing pyramid should look more like a crab

#63

Post by ozim » Wed, Nov 04, 2020, 9:26 AM UTC

jniedrauer wrote: post #33

ozim wrote: post #24

Earlier quoted context omitted.

I would agree because I see my devs writing unit tests for plumbing code that does not have any logic. I am annoyed by that. So I am 'party pooper' when it comes to unit testing. But I would like to have unit tests for something that has calculations or non trivial logic only. Other thing I would really like 'risk based' approach, like what is the worst thing that can happen with that code. If it can drop half of the…

There is some value in unit tests for plumbing. Something I see a lot is a set of values being transformed across several domains, losing data at each step. A very simple test that inputs values on one end of the pipe, and checks the values on the other end of the pipe, makes it very easy to be sure that you didn't drop something.

If you transform values, then it has logic, so it does not fit in my definition of simple plumbing.

If you check that integer value AgencyId is passed to another object as integer value with the same name of AgencyId that is basic plumbing that should not be covered by unit test.

Of course someone might forget to wire that up somewhere, but it is not something I would say writing test and then maintaining it costs more than just fixing it when you find it and once it is fixed there is no real reason to keep regression on it.

Re: The testing pyramid should look more like a crab

#64

Post by bluntfang » Wed, Nov 04, 2020, 3:49 PM UTC

btilly wrote: post #58

mpweiher wrote: post #50

Earlier quoted context omitted.

If it's talking to the database, it's not a unit test. https://blog.metaobject.com/2014/05/why-i-don-mock.html

I'm aware of this. If it is talking to a database, it's not a unit test. If you've mocked out the database, it is a unit test. My point is that the unit test version is a lot of work to create and tends to be useless. (Which is similar to the point that your blog link made.)

If you write your code in such a way where business logic and database access are separated you won't have to mock out the database. Business logic is where the Good Stuff is. We can trust that the database access package is working because we probably didn't write it and it probably has it's own set of tests.

Re: The testing pyramid should look more like a crab

#65

Post by btilly » Fri, Nov 06, 2020, 2:21 AM UTC

bluntfang wrote: post #64

btilly wrote: post #58

Earlier quoted context omitted.

I'm aware of this. If it is talking to a database, it's not a unit test. If you've mocked out the database, it is a unit test. My point is that the unit test version is a lot of work to create and tends to be useless. (Which is similar to the point that your blog link made.)

If you write your code in such a way where business logic and database access are separated you won't have to mock out the database. Business logic is where the Good Stuff is. We can trust that the database access package is working because we probably didn't write it and it probably has it's own set of tests.

If your business application deals with large volumes of data, trying to abstract out the database is a recipe for performance problems. Instead you want to push logic to the database.

To give a real example, I'm working with time series generated by sensors on factory equipment. They can easily have millions of records per day.