99% code coverage (2017)
rachelcarmena.github.io
99% code coverage (2017)
1–10 of 99 posts
Re: 99% code coverage (2017)
#2The author suggests to add mutations to the code, like replacing 'I think this might apply when you write your own algorithms and want to test them. But if you are, like most of us, working on Business Logic, then you are probably writing the wrong tests.
We usually want to know whether a workflow or a customer story works as intended and as such we should write more Integration tests.
The idea itself has merit though and draws parallels to writing tests with variable or random input parameters, I am pretty sure there already was something on this.
Re: 99% code coverage (2017)
#3Mutation testing is a neat idea I'd not heard of. Wonder how well it works in practice.
Someone's implemented a package for doing it with Go which looks good: https://github.com/zimmski/go-mutesting
Re: 99% code coverage (2017)
#4Mutation testing is a neat idea I'd not heard of. Wonder how well it works in practice. Someone's implemented a package for doing it with Go which looks good: https://github.com/zimmski/go-mutesting
Me neither, I liked the concept of mutation testing. (I was using this performing changes manually, without knowing this technique has a nice name).
I would appreciate if somebody points out a mutation framework /tools for .net
Re: 99% code coverage (2017)
#5Mutation testing is a neat idea I'd not heard of. Wonder how well it works in practice. Someone's implemented a package for doing it with Go which looks good: https://github.com/zimmski/go-mutesting
I haven't worked with it extensively, but from what I've seen, it's far beyond what most teams should consider. It's basically a more thorough method of measuring test coverage, pointing out cases that have not been covered by your tests yet. However, the number of tests that it would have you write to reach 100% mutation coverage is not justified by the number of bugs you'll catch, unless the impact of any bug is very high (I suppose at NASA?). In fact, the amount of work required to even check which tests you missed is already not justified.
I usually use coverage as a tool to remind me of pieces of code I intended but forgot to test [1], so setting it to an arbitrary percentage is not that useful, in my opinion. If 100% is not feasible usually as well, that makes Mutation Testing in general not worth the effort.
Again, take my opinion with a grain of salt.
Re: 99% code coverage (2017)
#6Mutation testing is a neat idea I'd not heard of. Wonder how well it works in practice. Someone's implemented a package for doing it with Go which looks good: https://github.com/zimmski/go-mutesting
I haven't worked with it extensively, but from what I've seen, it's far beyond what most teams should consider. It's basically a more thorough method of measuring test coverage, pointing out cases that have not been covered by your tests yet. However, the number of tests that it would have you write to reach 100% mutation coverage is not justified by the number of bugs you'll catch, unless the impact of any bug is ve…
I think mutation testing really shines in code bases that are already heavily tested, because it let's you discover test cases that you don't actually need. Tests are a burden since you have to adapt them when you change the behavior. With mutation testing you can prune your test code by identifying tests that test very similar behavior.
Re: 99% code coverage (2017)
#7The author suggests to add mutations to the code, like replacing ' I think this might apply when you write your own algorithms and want to test them. But if you are, like most of us, working on Business Logic, then you are probably writing the wrong tests. We usually want to know whether a workflow or a customer story works as intended and as such we should write more Integration tests. The idea itself has merit thou…
You think customers aren't affected if you mistakenly replace a < with a <= in your code?
Re: 99% code coverage (2017)
#8I had a similar experience, you always get what you measure.
IMO the best a approach is to first integrate the test coverage in the code reviews, cause there is no hard rule [0] and second write property-based tests [1].
[0] http://www.se-radio.net/2018/05/se-radio-episode-324-marc-ho...
[1] Sample framework for JavaScript https://github.com/jsverify/jsverify
Re: 99% code coverage (2017)
#9The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project.
The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes.
If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful to your project. If you build square wheels for example, you may not realize that they're not designed correctly until you attach them to the car and realize that the car doesn't function well with them. It makes no sense to preemtively write unit tests for a component which has a very high likelihood of not being in its final desired state; you're just giving yourself more work to refactor the unit tests over and over; or worse, you're afraid of refactoring the tests and so you lie to yourself thinking that square wheels are fine.
Integration tests are by far the most useful tests to have at the beginning and middle stages of the project; integration tests allow you to keep your focus on the real goals of the project and not get stuck on designing the perfect square wheel.
Re: 99% code coverage (2017)
#10The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…
What is an unit? Is it the wheel or the car? When you start to see that the car is the unit the difference between an unit test and an integration test gets blurry.