Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

99% code coverage (2017)

rachelcarmena.github.io

Post reply on HN
Print view View original

41–50 of 99 posts

Re: 99% code coverage (2017)

#41

Post by hopler »

rypskar wrote: post #27
twodave wrote: post #13

Earlier quoted context omitted.

The basic rule I go by is whether or not the test is self-contained. For example if you are making a trip to a database or hitting a 3rd party dll then it’s an integration test. I thought this was pretty widely accepted.

I would say that for most CRUD applications it is much easier to have the tests run against an sqlite database instead of mocking it, so would let unit tests update the database and write tests to check if the data in the database match what I would expect from the input. If you change how you store the data, it isn't a side effect of changes you are doing but the main effect so the test could be rewritten anyway. So…

Mocking is a last resort and usually the wrong choice and used incorrectly. In memory fakes like sqlite (or better, running your real database in a small local inmemory footprint) is usally the way to get fast, correct, stable tests.

Re: 99% code coverage (2017)

#42

Post by hopler »

rypskar wrote: post #27
twodave wrote: post #13

Earlier quoted context omitted.

The basic rule I go by is whether or not the test is self-contained. For example if you are making a trip to a database or hitting a 3rd party dll then it’s an integration test. I thought this was pretty widely accepted.

I would say that for most CRUD applications it is much easier to have the tests run against an sqlite database instead of mocking it, so would let unit tests update the database and write tests to check if the data in the database match what I would expect from the input. If you change how you store the data, it isn't a side effect of changes you are doing but the main effect so the test could be rewritten anyway. So…

Mocking is a last resort and usually the wrong choice and used incorrectly. In memory fakes like sqlite (or better, running your real database in a small local inmemory footprint) is usually the way to get fast, correct, stable tests.

Re: 99% code coverage (2017)

#43

Post by hopler »

paulriddle wrote: post #28
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

Some people feel more comfortable writing code without tests first, gradually shaping final design of the interface. Especially if the tests involve a lot of stubbing and mocking of things like Redis and Sphinx Search, or messing with crypto tokens, parsing HTML, freezing time, setting up global config attributes like support emails, interaction between 2 different databases, etc. Tests are code as well and oftentime…

That sounds like a complicated workaround for bad libraries (Redis does support unit tests?) and bad apply design with strong coupling between remote components.

Re: 99% code coverage (2017)

#44

Post by hopler »

josteink wrote: post #23
sethammons wrote: post #19

I hadn't realized this mutation testing existed as automated tooling. I'll be looking more into it. Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

How do you know you have full test coverage if you don't mutate your code?

Re: 99% code coverage (2017)

#45

Post by swish_bob »

josteink wrote: post #39
swish_bob wrote: post #34

Earlier quoted context omitted.

It's entirely possible to write code that passes an as yet tested desired piece of functionality. If you want to make sure that piece of functionality doesn't get rewritten out later, you probably still ought to write a test. This test never fails though, unless you deliberately force it to. You _could_ just assume your test is correctly written, but I personally prefer to be sure by seeing it red at least once.

I think you misunderstand. I'm not arguing against red tests. I'm arguing for them . What I'm arguing against is 1. writing the finished code, 2. writing green tests to "prove" the code correct, 3. trying to sabotage the finished code to "prove" that the tests works by making them red. Sounds crazy? That was what OP said he was doing!

It's quite possible that he's testing after. It's implied, but it isn't necessarily the case.

Anyway, we all agree testing is good. And that a test you've never seen fail is bad. Personally, I try to delete exploratory code and try to use tests to drive out the _real_ implementation, but I'm going to admit that at least sometimes I'll write code that's inherently "safe", and then add a test afterwards for garbage inputs, for example.

Re: 99% code coverage (2017)

#46

Post by josteink »

hopler wrote: post #44
josteink wrote: post #23

Earlier quoted context omitted.

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed. That's completely backwards compared to how you should be doing things. You either write the test: 1. to verify the existence of a bug by reproducing it, or 2. to formalize the spec for yet-to-be implemented code/feature. And…

How do you know you have full test coverage if you don't mutate your code?

That’s a fair point. I don’t.

In that youre not trying to prove the tests “correct” though, but instead trying to prove good coverage.

I’d argue there are several ways to do that, and that’s certainly an interesting approach to the problem.

In either case you’ll have trouble being 100% confident though, so I guess it’s a matter of deciding when enough is enough.

Re: 99% code coverage (2017)

#47

Post by th456789123 »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

unit tests is not only to catch regression and it does not lock down anything. I found that doing TDD actually helps you design better, it naturally force you to write easily testable code which translate into more readable and maintainable code. Also well written UT prove very useful as living documentation. When you put the same care in them as you do in production code, you can very easily add features not needing to reverse-engineer the all thing. UT is one of thing things that may seem counter-intuitive but if you stick to it for a little while you see lots of positive effects. A bit like forcing yourself to hold the neck properly when learning the guitar, it slows you down at 1st, hurts a little, but if you do it you'll play better.

Re: 99% code coverage (2017)

#48

Post by mabbo »

cryptica wrote: post #9

The industry is obsessed with getting 100% unit test code coverage even though it doesn't mean anything to the project. The purpose of unit tests is to lock down the project's source code once it's essentially completed; to avoid regressions when making minor changes. If you start writing unit tests too early in the project, you're effecively locking down units of code which haven't yet proved themseves to be useful…

> The purpose of unit tests is to lock down the project's source code

I disagree slightly. If you're writing unit tests such that the code itself can't change without breaking the tests, perhaps the tests have been too tightly coupled.

The end goal of unit tests is to secure the expected functionality of the unit of code such that the tests will fail if the functionality stops working. But if changes to the code (refactoring, changing the ordering of certain lines, etc) causes the tests to break, then what was tested wasn't the functionality but the implementation. You might as well write a test that loads each source file as a string and verifies it equals what you've written.

An example: my favorite set of tests I've written lately were for some classes to handle deduplication of a large set of records. One implementation wrote to disk (for data sets too large to fit into memory) while the other did it purely in memory. Since their functionality was supposed to be identical, I wrote one set of tests that both ran on. The tests could not be specific to the code written because it had to work on two sets of code. If we wrote a third implementation, the same tests should work for it as well.

(But what you said about integration tests: 100% agree)

Re: 99% code coverage (2017)

#49

Post by _pmf_ »

You can do that, or you can let priority of business requirements drive your testing efforts. For each project, I have a certain set of hard requirements that will never be used in the field. We deliver these half-assed and fix bugs if 2 years later someone uses the feature by accident.

Re: 99% code coverage (2017)

#50

Post by austincheney »

JoeAltmaier wrote: post #38
austincheney wrote: post #37

Earlier quoted context omitted.

It drives me nuts during interviews talking about test automation because people are very particular about the type of testing whether its unit testing, integration testing, acceptance testing, or whatever. In my mind you only need 1 kind of testing: feature tests. Does the application provide the expected output for a given input and/or configuration. The application does all that it claims to do in a very precise w…

In a complex system a feature test can detect a problem but not diagnose it. For that a unit test is desirable. The hole you get into is, lots of units are changed and a feature quits working and everybody says 'its not me!' and it doesn't get fixed.

> and everybody says 'its not me!' and it doesn't get fixed.

Test automation is not a remedy to prop up broken leadership. I would appoint an arbitrary owner of the defect and that person would visit with other people as necessary to remove or dismiss various functionality from blame one by one. Once the appropriate collision of changes is discovered it will become more clear how to address resolution.

Post reply on HN