Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

99% code coverage (2017)

rachelcarmena.github.io

Post reply on HN
Print view View original

31–40 of 99 posts

Re: 99% code coverage (2017)

#31

Post by snidane »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

Very good comment about SW testing in general.

In my experience the only time when unit tests were useful was when they were integration tests at the same time.

For example when writing a data structure or algoritms or programming language/compiler. There is a clear boundary to such a system and one can reason about it in terms of input and output black box. As such the test for this system is an integration test, but due to its isolation it looks so simple that someone would call it a unit test. Second example would be all TDD blog posts, where the author tests some trivial entity with perfect isolation. That confuses integration tests with unit tests again.

The biggest mistake people do when they write tests (or library functions) too early is that they miss the target. The first part of writing code is not to produce a clean and maintainable system, but to get the requirements right. Listening to people seldomly works, it is usually better to observe people interacting with some prototype and infer requirements from that. In words of Henry Ford, the goal is to find that the requirements are for a 'car' and not just for 'faster horses'. People who fossilize the requirements too early miss the target completely in my experience.

Re: 99% code coverage (2017)

#32

Post by mabbo »

josteink wrote: post #23
sethammons wrote: post #19

I hadn't realized this mutation testing existed as automated tooling. I'll be looking more into it. Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

While in principle I agree with you, there is something to be said for 'explorative' programming- where I don't quite know how I'm going to build this feature, or I don't quite know just what the refactoring will look like, etc. Most often it's that I don't fully understand the codebase I'm working with yet, and the tests that exist are old and crusty. I want to play a little while first.

I feel like that is a valid form of our craft, and it doesn't play well with TDD. It's not the primary way that I prefer to work, but I'm certain I'll be there later today (I've got a bastard of a sprint task waiting for me).

Re: 99% code coverage (2017)

#33

Post by TravelAndFood »

sethammons wrote: post #26
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

You don't always write a test first. A bug? Sure, write a test that shows the bug, fix it, make it green. A very well defined something? Sure, TDD. Many times, you will write code (or come across pre-existing code) and tests come last.

Yep, TDD is quite hard when requirements are vague/changing... the reqs changed? OK now I have to rewrite my test AND the code that passes it.

Vague/changing reqs is of course a separate problem.

Re: 99% code coverage (2017)

#34

Post by swish_bob »

josteink wrote: post #23
sethammons wrote: post #19

I hadn't realized this mutation testing existed as automated tooling. I'll be looking more into it. Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

It's entirely possible to write code that passes an as yet tested desired piece of functionality. If you want to make sure that piece of functionality doesn't get rewritten out later, you probably still ought to write a test. This test never fails though, unless you deliberately force it to.

You _could_ just assume your test is correctly written, but I personally prefer to be sure by seeing it red at least once.

Re: 99% code coverage (2017)

#35

Post by crispyporkbites »

paulriddle wrote: post #28
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

Some people feel more comfortable writing code without tests first, gradually shaping final design of the interface. Especially if the tests involve a lot of stubbing and mocking of things like Redis and Sphinx Search, or messing with crypto tokens, parsing HTML, freezing time, setting up global config attributes like support emails, interaction between 2 different databases, etc. Tests are code as well and oftentime…

Agree, actually designing & implementing good tests requires a lot of effort. It's rarely wasted effort but if you really need something out the door now it can distract from short term delivery.

e.g. I'm using a 3rd party transpiler / build tool, which I don't know all the details of, my test runner wouldn't do exactly the same transpliation of code this build tool, it took me about an hour to figure out where the build tool's config file was and how to get it working with the test runner.

Did I learn more about my tooling and code base? Yes. Was it useful? well maybe not as I might kill this project shortly anyway.

Re: 99% code coverage (2017)

#36

Post by mannykannot »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

More generally, the idea that you can conquer complexity by writing only small components overlooks the fact that the complexity is found in the interactions of the components. Modularity is necessary, but not sufficient, for success in software development.

Re: 99% code coverage (2017)

#37

Post by austincheney »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever.

In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise way or it doesn't. Everything else is extraneous, though you may have to test the application in wildly different ways with wildly different means of automation to validate all feature support.

Feature tests are executed by running the application with a given input and configuration and comparing the result against a known expected output. Provided sufficient feature coverage this is enough to test for regression. While the feature tests are running each test runs against a clock so that dramatic swings in performance can be qualified with numbers. When that is not enough, as in the case of accessibility, have people (actual users) perform feature tests.

Other problems like bad design, redundant features, or poor code quality are qualified using code reviews and validation tools, like a linter.

Re: 99% code coverage (2017)

#38

Post by JoeAltmaier »

austincheney wrote: post #37
cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever. In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise w…

In a complex system a feature test can detect a problem but not diagnose it. For that a unit test is desirable.

The hole you get into is, lots of units are changed and a feature quits working and everybody says 'its not me!' and it doesn't get fixed.

Re: 99% code coverage (2017)

#39

Post by josteink »

swish_bob wrote: post #34
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

It's entirely possible to write code that passes an as yet tested desired piece of functionality. If you want to make sure that piece of functionality doesn't get rewritten out later, you probably still ought to write a test. This test never fails though, unless you deliberately force it to. You _could_ just assume your test is correctly written, but I personally prefer to be sure by seeing it red at least once.

I think you misunderstand.

I'm not arguing against red tests. I'm arguing for them.

What I'm arguing against is 1. writing the finished code, 2. writing green tests to "prove" the code correct, 3. trying to sabotage the finished code to "prove" that the tests works by making them red.

Sounds crazy? That was what OP said he was doing!

Re: 99% code coverage (2017)

#40

Post by humanrebar »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

> The purpose of unit tests is to lock down the project's source code once it's essentially complete

Good unit tests (like doctests) also provide superior documentation and examples.

Sure, worrying about documentation too soon is bad, but sometimes when doing early research, getting details clarified and written down is paramount and tests, maybe even unit tests, are a nice tool to have when you want it.

Post reply on HN