What types of solutions exist for automating the creation of front-end tests? Either full automation, or partial with some user input/direction. I found https://kwola.io/ but wondering if there are other alternatives
The Myth of Code Coverage
61–70 of 115 posts
Re: The Myth of Code Coverage
#62Just because code is touched during the test process doesn’t necessarily mean it has been tested. Coverage is more useful for finding chunks of code that aren’t exercised at all by tests. Branches that never get hit that probably deserve extra scrutiny. Coverage is an interesting heuristic but 100% code coverage is not 100% bug free code. Likewise, there’s stuff that’s just not worth it to wrap in a test. The effort…
Another advantage that does not get the love it deserves in my opinion, is how useful coverage becomes when it comes to upgrading a dependency, or even the language version. But your project will have to stick around for a while to appreciate that.
Re: The Myth of Code Coverage
#63I'm a strident adherent to having 100% code coverage, but this mostly works because my history in infrastructure and a desire to sleep well. The key idea that I have found that if you can't synthetically get your process into a specific state, then life is going to be hard. The problem with a lot of people is that they view testing as a burden rather than a criticism of the code they are testing. If you can't test yo…
Totally agree with you. This really is a concept that you need to keep in mind when you write your code "testability". And i find that when you get this you also get a Buch of other properties for "free". Orthogonality and reusability for example. And really it's a force multiplier and kicks productivity into high gear when you know that your lower level components work as expected and you don't have to go ok a wilf goose hunt when something would be constantly broken. To summarize I find that writing tests early on reduce product velocity but the compound effect of good quality and regression prevention give you higher velocity later on.
Re: The Myth of Code Coverage
#64> Being dogmatic about tests and covering every line will only make it more difficult to get rid of it.
This is only true if one is lackadaisical about how they architect their code and (unit) tests. You should be able to completely smash a function and it's tests without breaking any other tests ... If not then you're testing a different unit within those broken unittests.
Re: The Myth of Code Coverage
#65Just because code is touched during the test process doesn’t necessarily mean it has been tested. Coverage is more useful for finding chunks of code that aren’t exercised at all by tests. Branches that never get hit that probably deserve extra scrutiny. Coverage is an interesting heuristic but 100% code coverage is not 100% bug free code. Likewise, there’s stuff that’s just not worth it to wrap in a test. The effort…
When “hold people accountable to metrics” is not just your company’s management philosophy but the gospel of its entire surrounding culture, this is not an argument that gets anywhere. It is always better to just cover the damn lines than make your Director explain to the VP why he has worse coverage numbers than a sister org.
The same way you should make an expected “no such object” result a 200 status code, so that she does not have to stand up and justify her error rate.
Re: The Myth of Code Coverage
#66Nothing engineers like more than evidence that they don't need to write tests!
Re: The Myth of Code Coverage
#67I always say between 60-80%, 60% for greenfield or new testing as that is generally right for covering the golden path(s) and the major error cases. It should slowly grow over time as bugs get tests written for them, but if it gets over 80% you probably want to refactor some.
>80% test coverage digs way to far into the code, testing the implementation instead of the interfaces/apis. It is a definite code smell that probably means the tests were written poorly and need some major reworking.
Re: The Myth of Code Coverage
#68Earlier quoted context omitted.
Really? You want to strive to test that logging logs and observability observes? You want to test constructors, getters and setters? Testing trivial code brings negative value, why would you do that?
I prefer to test everything. The parts you don't test are the parts that break. For example, here's a case where I incremented the wrong metric: https://github.com/jrockway/alertmanager-status/commit/fccae... I noticed the bug when I went to look at a dashboard with that metric on it, and noticed it had the wrong name. If go's Prometheus library had an easy way to run "metric.CurrentValue()", I would have tested it..…
> If go's Prometheus library had an easy way to run "metric.CurrentValue()"...
A small tip for the future: https://pkg.go.dev/github.com/prometheus/client_golang/prome....
Re: The Myth of Code Coverage
#69I always say between 60-80%, 60% for greenfield or new testing as that is generally right for covering the golden path(s) and the major error cases. It should slowly grow over time as bugs get tests written for them, but if it gets over 80% you probably want to refactor some. >80% test coverage digs way to far into the code, testing the implementation instead of the interfaces/apis. It is a definite code smell that p…
> that is generally right for covering the golden path(s) and the major error cases
The thing is, your metric of 50-80% has nothing to do with if you're actually covering the golden path(s) and major error cases. The error cases you didn't think of, wouldn't count in this metric of course, but they might still be missed.
Code coverage as a metric says as much as number of lines as a metric. Meaning nothing, it's just a number. Aiming at any sort of code coverage metric misses to think about what to actually cover. 0 code with 0 tests has 100% code coverage, doesn't mean it's actually a good program.
Re: The Myth of Code Coverage
#70I'm a strident adherent to having 100% code coverage, but this mostly works because my history in infrastructure and a desire to sleep well. The key idea that I have found that if you can't synthetically get your process into a specific state, then life is going to be hard. The problem with a lot of people is that they view testing as a burden rather than a criticism of the code they are testing. If you can't test yo…
I wrote a UDF for MySQL. It had just shy of 100% coverage. It used a malloc. I could not figure out how to trigger a malloc failure inside of a imported library running on a MySQL instance. Do I not check for malloc failure (in order to get 100%)? Or do I develop some sort of instrumented version of MySQL which lets me test malloc failure? That seemed far more overkill than warranted for the small project I worked on…
I believe you can override libc's malloc by passing in LD_PRELOAD, at which point you've got the basic building blocks to test malloc failure if you can find a way to tell your malloc call apart from the others.
Having said that, I don't disagree with your choice, because messing with LD_PRELOAD is difficult, scary, and probably overkill anyway.