Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

99% code coverage (2017)

rachelcarmena.github.io

Post reply on HN
Print view View original

51–60 of 99 posts

Re: 99% code coverage (2017)

#51

Post by CJefferson »

peterwaller wrote: post #3

Mutation testing is a neat idea I'd not heard of. Wonder how well it works in practice. Someone's implemented a package for doing it with Go which looks good: https://github.com/zimmski/go-mutesting

I tried it. If you had a super serious algorithm where you were will to spend any money to test it (a central security algorithm for example), it might be worth it.

The problem I had is that there were many many mutations which led to code which was functionally identical, just a bit slower, or took a different route. You have to manually verify every mutation to check if this.

For one mutation it took me 30 minutes to convince myself the algorithm was still doing the same thing, just in a different way.

Re: 99% code coverage (2017)

#52

Post by jeffasinger »

austincheney wrote: post #37
cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever. In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise w…

I think your argument is missing the "cost" of feature tests, which is that they necessarily must run much slower than tests that are only testing a small unit of code.

For example, I worked on payroll software at some point. After finding a bug, we'd want to ensure that could never happen again, and would want to add a test for it going forward. So for example, I may have needed to write a test around someone who worked in one city in Ohio, lived in another, previous income above some amount, and with certain tax advantaged benefits. Setting up this test via feature testing is certainly possible, but it's likely the test itself will take a significant amount of wall time to execute. It's way faster to just test the payroll calculation code, which means you can run the tests more often, and with less developer inconvenience.

Re: 99% code coverage (2017)

#53

Post by JoeAltmaier »

austincheney wrote: post #50
JoeAltmaier wrote: post #38

Earlier quoted context omitted.

In a complex system a feature test can detect a problem but not diagnose it. For that a unit test is desirable. The hole you get into is, lots of units are changed and a feature quits working and everybody says 'its not me!' and it doesn't get fixed.

> and everybody says 'its not me!' and it doesn't get fixed. Test automation is not a remedy to prop up broken leadership. I would appoint an arbitrary owner of the defect and that person would visit with other people as necessary to remove or dismiss various functionality from blame one by one. Once the appropriate collision of changes is discovered it will become more clear how to address resolution.

...and they would do that by writing individual unit tests.

Re: 99% code coverage (2017)

#54

Post by Jach »

rypskar wrote: post #29
jondubois wrote: post #14

Earlier quoted context omitted.

Unit tests should test individual subcomponents of your system in isolation. If your project is to build a car from scratch, then the unit cannot be the car. If your project is to build an autonomous fleet of self-driving cars, then from your project's perspective, the car could be a unit; the project to build the autonomous car would have different units from the project which manages the fleet of cars.

Exactly, context matter. Most cases when some is writing for or against unit testing they miss the part about finding what they define as an unit. If the class need other classes to work, the whole set can be seen as one unit. Who didn't want to test a private method before they see that a unit is more than a function?

I like to call it the 'system under test' / SUT. Then I can go on to say a unit test is a test that's fast and if it fails points to a specific problem in the SUT. Smaller SUTs obviously help satisfy both. If setting up collaborators in a bigger SUT leads to either unacceptable runtimes or too much brittleness from many things being able to go wrong, it's a signal you're not getting the value of unit testing in that place. (The test itself may or may not be valuable, other types of testing have their own value propositions.)

Re: 99% code coverage (2017)

#55

Post by androidgirl »

mabbo wrote: post #32
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

While in principle I agree with you, there is something to be said for 'explorative' programming- where I don't quite know how I'm going to build this feature, or I don't quite know just what the refactoring will look like, etc. Most often it's that I don't fully understand the codebase I'm working with yet, and the tests that exist are old and crusty. I want to play a little while first. I feel like that is a valid…

I might be wrong but aren't these called spikes?

I do exploratory coding in a similar way, usually with a repl next to my text editor, and this first single file version is usually always trash but helps me understand the problem space. Then I write it "for real" with the knowledge I've gained.

Re: 99% code coverage (2017)

#56

Post by Cpoll »

adrianN wrote: post #11
cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

IMHO unit tests are replacements for an interface contract specification language. You encode the requirements for the interface of your unit (whatever that may be) into tests. Writing unit tests early means that you think about the interface early. This way you can find awkward or wrong requirements early. Of course they're no replacement for integration tests, but they're supposed to be much cheaper to write, to ru…

> interface contract specification language

Sounds a bit like C#'s code contracts. A team I was on tried it some years ago, and I think they were a great substitute for a bulk of the unit tests we wrote. It didn't catch on, unfortunately.

Re: 99% code coverage (2017)

#57

Post by ramblerman »

josteink wrote: post #39
swish_bob wrote: post #34

Earlier quoted context omitted.

It's entirely possible to write code that passes an as yet tested desired piece of functionality. If you want to make sure that piece of functionality doesn't get rewritten out later, you probably still ought to write a test. This test never fails though, unless you deliberately force it to. You _could_ just assume your test is correctly written, but I personally prefer to be sure by seeing it red at least once.

I think you misunderstand. I'm not arguing against red tests. I'm arguing for them . What I'm arguing against is 1. writing the finished code, 2. writing green tests to "prove" the code correct, 3. trying to sabotage the finished code to "prove" that the tests works by making them red. Sounds crazy? That was what OP said he was doing!

I'm pretty sure everyone understood. There are many ways write code.

Re: 99% code coverage (2017)

#58

Post by monksy »

austincheney wrote: post #37
cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever. In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise w…

Please tell me all of the products that your code is contained in. I don't want to die in a self-driving car because has superficial test coverage. (Well it passed it's feature tests that you wrote for it)

There are very specific reasons for each level of testing and the benefits stack.

Re: 99% code coverage (2017)

#59

Post by SkyPuncher »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

I agree. The project I'm working on hovers around 70% coverage. As far as I can tell, we'd likely loose productivity pushing for any better coverage.

We get some bugs/issues that could be identified by better coverage, but most of our "bugs" stem from real world issues or differing views on functionality. Getting that right takes a ton of effort that simply isn't worth it right now.

Re: 99% code coverage (2017)

#60

Post by ApolloFortyNine »

monksy wrote: post #58
austincheney wrote: post #37

Earlier quoted context omitted.

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever. In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise w…

Please tell me all of the products that your code is contained in. I don't want to die in a self-driving car because has superficial test coverage. (Well it passed it's feature tests that you wrote for it) There are very specific reasons for each level of testing and the benefits stack.

>Well it passed it's feature tests that I wrote for it

You (the programmer) write unit tests too, and even well code with well written unit tests will have bugs in it, if simply due to logic errors, or if due to mocks.

>Please tell me all of the products that your code is contained in. I don't want to die in a self-driving car because has superficial test coverage.

I just don't think this was the best way to go about your argument. Why attack the poster? Why not link your own papers about the benefits of unit tests?

Post reply on HN