Live data from Hacker News

Write tests. Not too many. Mostly integration

blog.kentcdodds.com

231–240 of 338 posts

Re: Write tests. Not too many. Mostly integration

#231

Earlier quoted context omitted.

I don't see why TDD requires ruling out static methods and insisting on hiding everything behind an interface. Static methods are straightforward to test, certainly more than a class with multiple dependencies which need to be mocked. Usually the complaint is about coupling when calling static methods but these can be wrapped in a delegate if required.

Simply because you can't mock the static dependency, therefore that method is now dependent on the static class and you don't have any control over it. This is problematic - what if at some point later another developer adds a database call into the static method to do some logging? Now your testing will dirty whatever database you're using, as well as run 10x slower - and yet the test will still pass and everyone wi…

“What if ...” doesn’t pass YAGNI.

Re: Write tests. Not too many. Mostly integration

#232
Testing.

My current theory for why there is so much confusion about testing is because developers are not often taught the difference between specification (theorems) and implementation (proofs) and why you want to separate the two.

It seems like business and investors want us to write code, more of it, and faster. Value, value, value!

So my question is: what is valuable?

Do you value your customers' data? Do you value their time? Their safety? Your brand and reputation? If you answered yes to any of these (and the other questions I may of forgotten or elided) then you should be encouraging your developers to write specifications.

One such specification, and a weak one that developers can write and maintain on their own without involving stakeholders, are unit tests. It's a weak form of specification for a library/module/component you would like to have because it specifies properties and behaviors by example. The spec gives an example of use and the expected outcomes. A good test is a verbose Hoare triple: given some context, when this method is called, then this result is expected.

Your implementation of that specification is what you're after and it's the reason why you should write the tests first. Writing tests first has little to do with productivity or design. You should write them first because your implementation should prove the theorems in your specification. Theory first. Proof after.

Sometimes you have to revise your theories after attempting the proof.. but that's a story for another day.

But we can write better specifications! Unit tests are weak because each test only demonstrates a single expectation. It doesn't quantify over the space of possible inputs! If you want to write a better specification for your parser or transformation try property based testing. Use a library like QuickCheck. You give it a theorem that quantifies over the input space of your function under test and it will find out for you if your proof holds (limited only by how many examples you want to try... say 10000). It's not a proof that your implementation is correct but its a much stronger guarantee than a suite of unit tests and it doesn't cost you that much more to use it.

Integration tests though. I'm not sure if I agree with the advice. You should definitely write them but they can become a time sink and cost quite a bit to run if you need to test a non-trivial system. Where they lack is at level of quantification and this is where the real errors lie in software systems with many components. Integration tests, like unit tests, are proof by example. You write a specification for how a given configuration of components should interact and you supply a context and run the test and see if your assertions hold. It's wise to be aware that it won't catch most safety errors and it will never catch liveness errors.

Safety errors having to do with correctness of expected values over the lifetime of a computation.

Liveness having to do with maintaining invariants over the lifetime of a computation.

So integration tests are useful, do write them, but I wouldn't advise spending most of your time on them if you're dealing with more than 2 or 3 components.

Once you have messaging and co-ordination you're going to want a stronger specification and that would probably look more like a theorem written in a language that can be verified by something called a model checker. Something like TLA+ is making good progress breaking into industry.

... this is getting long. To summarize: developers should be taught and given time to write specifications. Most errors in software arise from poor, incorrect, or missing specifications. Weak specifications are better than none. Think of tests as specifications, write them first, and prove your software meets those specifications. Then you can change your implementation as you refine your specification and write better, faster, more reliable software.

Re: Write tests. Not too many. Mostly integration

#233

Earlier quoted context omitted.

>That said, if you go for functional style in OOP, i.e. shoving as much as you can into static helper functions and most of the rest into dumb private stateless functions, you suddenly gain both a clean architecture and lots of test points to use in unit tests. So you can have testable code, but you have to chill out with the OOP thing a bit. Wow, this is exactely totally opposite of how one can achieve testability i…

It seems the encouraged method is IoC these days, and that's just dreadful. IoC/Dependency resolution all over make it insanely hard to reason about code without running circles through the codebase. For me, IoC seems invented almost entirely to make up for how difficult testing can be in particular languages. Which, sure, making up for shortcomings as good, but the necessity to use IoC for it feels bad.

This is how I felt about IoC as a junior dev 15 years ago, before I'd actually used it myself.

Nowadays the benefits are clear to me: more modular, testable code, and also lifecycle management.

Give it a try, you may well change your views.

Re: Write tests. Not too many. Mostly integration

#234

Earlier quoted context omitted.

Assuming you are a developer, start writing some. Next bug you find that needs a unit/functional test (e.g. it is caused by a simple error in transformation in one function), write the test first as a table of inputs vs outputs, find it fails, fix the function, and leave the test in. Gradually, the code base will contain unit tests which are useful, people will see they are useful, and other people might start using…

I agree. Tests for bug fixes are extremely valuable. Of such tests, unit tests are often very feasible. A test accompanying a bug fix holds value in many ways. Firstly, it demonstrates to those reviewing the change that the fix is suitable. Secondly, the presence of a test encourages reviewers to consider what a test does and doesn't cover, sometimes resulting in comments regarding improvements that had not otherwise…

I’m confident you know this, but just for the peanut gallery:

Tests that go along with bug fixes are some of the highest value tests, but they must be previously failing tests.

I can’t tell you how many times I’ve reviewed fixes with “tests” that are basically affirming the consequent; they assert something that was already true and it turns out they’re not actually fixing the reported bug.

Re: Write tests. Not too many. Mostly integration

#235

Earlier quoted context omitted.

> The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. The biggest problem I see with people advocating for tests and employing TDD is that they do change how they write code to accommodate tests. This leads to inclusion of lots of…

The abstraction is consistent though, and familiarity is a good thing when navigating a codebase which has N amount of other devs pushing to it every day. I practise TDD for peace of mind - if I add new functionality to existing code I can be 99.9% sure I haven't made any regressions. When a client's system goes down on a friday, I can 99.9% guarantee it wasn't my code that is at fault. If I have to work at the weeke…

I also practice TDD, but with a different 'T' - Type Driven Design. I find them much easier to reason about with types and safer (you can't compile your code if it doesn't pass the type check). Just model your data as ADT and pattern matching accordingly.

Of course, types alone can't represent every error cases out there (especially the one related to number or string), so I still write Unit Test for those cases. But the number of Unit Tests needed is much lower.

Re: Write tests. Not too many. Mostly integration

#236
post #2

"Eat food. Not too much. Mostly plants." -Michael Pollan

So much for "deep" & "profound".

What? Does it take away from it somehow that it's a twist on someone else's famous phrase? I immediately recognized the reference despite hardly being a Pollan-head, so I doubt anyone's idea was to conceal the connection.

Re: Write tests. Not too many. Mostly integration

#237
post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

I think many devs build applications only testing manually for a long time, and taking shortcuts (hacks) when something appears wrong. When they later want to write some unit tests, there are no proper units and that ball-of-mud-y code is hard to test. Of course, integration tests are still feasible because they're agnostic of the internal mess.

I've run into this a couple times and noticed at some point that writing unit tests while developing (not necessarily TDD) helps a lot in clarifying boundaries early on, and generally improves code quality.

Re: Write tests. Not too many. Mostly integration

#238

Earlier quoted context omitted.

> The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. The biggest problem I see with people advocating for tests and employing TDD is that they do change how they write code to accommodate tests. This leads to inclusion of lots of…

> as much as you can into static helper functions and most of the rest into dumb private stateless functions In our work we use C# and it is very hard, even next to impossible to make a static class pass a code review - given it's not for extension methods (which I hate... why not be explicit about the first parameter and stop acting as a part of the class ). They just tell us to use IoC and move to the next point. I…

Why hate extension methods? Do you really want to write Enumerable.ToList(Enumerable.Select(Enumerable.Where(someList, e => e.someBool), e => new {a = e.x, b = e.y)) and so on?

Re: Write tests. Not too many. Mostly integration

#239
post #200
post #152

Earlier quoted context omitted.

Why not just mock the UI drawing library? (I find this a very interesting question.)

Because you would be implementing a 100% of the UI features and still cannot prove if it meets the UI/UX design specs.

I think a lot of that is just the poverty of UI APIs and especially the imperative drawing paradigm. There's no reason in principle why we can't programmatically verify that the basics of the UI spec are fulfilled. If the whole UI layer is just impossible to verify then if we're at all serious about correctness then we should (hyperbolically) stop making UIs until we figure it out.

Re: Write tests. Not too many. Mostly integration

#240

Earlier quoted context omitted.

> as much as you can into static helper functions and most of the rest into dumb private stateless functions In our work we use C# and it is very hard, even next to impossible to make a static class pass a code review - given it's not for extension methods (which I hate... why not be explicit about the first parameter and stop acting as a part of the class ). They just tell us to use IoC and move to the next point. I…

Extension methods are useful for only one reason: they trigger code completion for browsing what this object can do. Static methods suffer from FP code completion problems (you can’t complete easily in the first arg of a function/procedure).

I think I am not mistaken in saying extension methods, like lambda functions, were invented primarily for the use case of Linq. Even if they weren't, that's how Linq is implemented, so extension methods serve more than that "one purpose" if you don't insist on writing C# in the style of C# 2.0.
Post reply on HN