Live data from Hacker News

Save the code or save the tests? (2020)

qntm.org

21–30 of 33 posts

Re: Save the code or save the tests? (2020)

#21

Earlier quoted context omitted.

Exactly. Verifying an algorithm is easy. Writing it is hard.

add(a, b) => a + b How would you verify that algorithm?

Something like:

    assertEquals(0, add(0,0))
    assertEquals(1, add(0,1))
    assertEquals(2, add(1,1))
    assertEquals(8, add(3,5))
    assertEquals(-3, add(2,-5))
PS: This kind of proves why a test is a lot less useful than the algorithm. Try figuring out how to implement "add" from the above test.

Re: Save the code or save the tests? (2020)

#22
post #20

I see how I am going to be downvoted to hell, but let's give it a shot: I just refuse to write tests. I think it is complete waste of time. Here, I said it. My preferred style of working is writing something and then refactoring it to my satisfaction. I toy with interfaces as I discuss meaning of various things, learning in the process. I am mercilessly cutting stuff that is not necessary, finding ways to make everyt…

I agree with you somewhat: especially when starting a project, it just makes no sense to write tests up-front (again, unless the product is the tests).

That said, when making incremental changes to an existing project, tests (and in some cases TDD) have a lot of value. There comes a point where the project changes from a creative endeavor to a product. Then you need tests.

The problem is how you create a test suite at that point in time without being negatively influenced by having written the code. Most unit tests I've seen in the wild are BS for this reason: the person writing them just repeats the code they wrote to implement the feature, or at least makes the same bad assumptions. In most cases they mock out so much that the test bears no relation to the real world, and breaks with the slightest change to the code.

For this reason I prefer to write tests from the top-down instead of bottom-up. Start with a few end-to-end tests, making sure to cover the happy paths and a few of the more probable error cases. These few initial tests are more valuable than any that will come later, because they best correspond to how the product will be used, and they are least likely to break during later refactorings since they don't depend on any internals, making them very cheap to maintain. For smaller projects, these tests may be all you need.

Every level you go below that incurs an increasingly large maintenance cost, so you have to weigh it against how often the design of that component will change.

When I do write unit tests it will be for essentially pure functions where the domain of inputs is sufficiently small that you can do fairly exhaustive testing. For example, a function that determines whether a floating point value can be losslessly converted to a 32-bit integer is perfect for unit testing. Nothing needs to be mocked - whatever the code does under test is what it will do in production.

All of this is my preference, but the only thing that really matters is that you actually use your brain when deciding what tests you write: weigh up the costs of the test against the set of potential bugs that it can prevent. The real sin of TDD is that it teaches you to write thoughtless tests.

Re: Save the code or save the tests? (2020)

#23

Earlier quoted context omitted.

Forall A: add(A, 0) == A Forall A B C: add(add(A,B), C) == add(A, add(B, C)) I think that’s sufficient. Rule 101 of useful testing is understanding the actual constraints you rely on. Maybe throw in add(A, B) == add(B, A) if you’re angling for a promotion…

add(a, b) => a == 1 && b == 1 ? 1 : a + b Kinda disappointing your test would take so long to run and still catch a simple error.

> your test would take so long to run

I think GP's test would probably be implemented with property testing (or even better, symbolic execution). It wouldn't necessarily take super long to run.

Re: Save the code or save the tests? (2020)

#24

Earlier quoted context omitted.

add(a, b) => a == 1 && b == 1 ? 1 : a + b Kinda disappointing your test would take so long to run and still catch a simple error.

> your test would take so long to run I think GP's test would probably be implemented with property testing (or even better, symbolic execution). It wouldn't necessarily take super long to run.

How difficult would it be to write a proper test with one of those systems for something more complex when even this trivial example doesn't have solution yet?

Re: Save the code or save the tests? (2020)

#25

Earlier quoted context omitted.

add(a, b) => a + b How would you verify that algorithm?

Something like: assertEquals(0, add(0,0)) assertEquals(1, add(0,1)) assertEquals(2, add(1,1)) assertEquals(8, add(3,5)) assertEquals(-3, add(2,-5)) PS: This kind of proves why a test is a lot less useful than the algorithm. Try figuring out how to implement "add" from the above test.

Personally I see this a lot and I'm not a fan:

  it('should work', () => {
    inputOutputPairs.forEach(([input, output]) => {
      expect(testedFunction(input)).equals(output);
    })
  })
Tests are way more useful when they are written with the intent of acting as documentation. If I had to give this test a grade (from F- to A+), it would get a D+. It catches regressions, yet does an absolutely horrid job of explaining the intent behind testedFunction.

Re: Save the code or save the tests? (2020)

#26
post #20

I see how I am going to be downvoted to hell, but let's give it a shot: I just refuse to write tests. I think it is complete waste of time. Here, I said it. My preferred style of working is writing something and then refactoring it to my satisfaction. I toy with interfaces as I discuss meaning of various things, learning in the process. I am mercilessly cutting stuff that is not necessary, finding ways to make everyt…

If you are trying to put structure to your code, you would like to get instant feedback on how that structure works from the outside. That is how the api works, the best way to do is to make a test which actually uses your code, and asserts some invariants on it.

With tdd the idea is you write a test first to explore how your code should be called, then you make that code, satisfy the test and add more requirements and repeat the whole thing. Once you have a bunch of tests which are defining the use cases in your code, you can refactor/structure your code internally without affecting the tests or if you think the whole api could be simpler, change the whole thing along with tests. The refactoring at this stage is generally of higher quality since you have a concrete idea of all the ways your code needs to work while as if you refactor early there is little guarantee that it's any good except for intuition.

If you are structuring or abstracting without examples of how your code is to be used, then the design often turns out to be inefficient, overly complex, coupled or over abstracted.

Admittedly this TDD approach is a bottom up design method, it takes time and effort and it's not well suited for problems where you already have a good idea of how code should look like/feel/behave.

You don't need to run these tests every time in ci, you can delete them or keep them as documentation or run them whenever you have to work on the code again.

Re: Save the code or save the tests? (2020)

#27
post #10

> More recently, out of curiosity, I pitched the same question as a poll on Twitter. On Twitter, roughly 80% of respondents said that they would save the application code and 20% said they would save the tests. Surprise, common sense. Saving the tests is like needing to know the answer to the Ultimate Question of Life, the Universe, and Everything (which is 42) and discarding the question.

I'm shocked that 20% of devs value tests over working code. This should have been a 99% to 1% thing.

I'll try to justify, imagine you have working code with no tests, it's been running for years but no one can understand it or change any of it. It's providing a very fixed value. Now imagine you have all the tests laying out exactly how the features should work. Sure you don't have anything to show for yet but it's not going to take a lot of time to implement these features. The hardest part in most companies is not building features but what features to build and how should they work. This usually takes domain knowledge and a lot of trial and error. So let me put it this way, if there was a competitor in the business, I would definitely like to see their tests rather than their working code. Since tests actually define how their app should work and not their code which merely tells me how their app is working.

Re: Save the code or save the tests? (2020)

#28
post #27

Earlier quoted context omitted.

I'm shocked that 20% of devs value tests over working code. This should have been a 99% to 1% thing.

I'll try to justify, imagine you have working code with no tests, it's been running for years but no one can understand it or change any of it. It's providing a very fixed value. Now imagine you have all the tests laying out exactly how the features should work. Sure you don't have anything to show for yet but it's not going to take a lot of time to implement these features. The hardest part in most companies is not…

I have worked on exactly that sort of code base. I've also worked on codebases which did super test coverage.

And honestly, the 0 test coverage base was actually easier to deal with.

Tests are good, but like all code they can be implemented poorly.

Re: Save the code or save the tests? (2020)

#29

Earlier quoted context omitted.

Something like: assertEquals(0, add(0,0)) assertEquals(1, add(0,1)) assertEquals(2, add(1,1)) assertEquals(8, add(3,5)) assertEquals(-3, add(2,-5)) PS: This kind of proves why a test is a lot less useful than the algorithm. Try figuring out how to implement "add" from the above test.

Personally I see this a lot and I'm not a fan: it('should work', () => { inputOutputPairs.forEach(([input, output]) => { expect(testedFunction(input)).equals(output); }) }) Tests are way more useful when they are written with the intent of acting as documentation. If I had to give this test a grade (from F- to A+), it would get a D+. It catches regressions, yet does an absolutely horrid job of explaining the intent b…

    cases = [
      { in: 1, arg: 2, expected: 3, message: "base case" },
      { in: 2, arg: -2, expected: 0, message: "negative numbers" },
      ...
    ]
    for ( c in cases ) {
      actual = obj.do( c.in, c.arg )
      assert( actual == c.expected, c.message )
    }
see: http://fit.c2.com/ ... table-based testing, "cases" and "drivers"

Re: Save the code or save the tests? (2020)

#30
post #27

Earlier quoted context omitted.

I'm shocked that 20% of devs value tests over working code. This should have been a 99% to 1% thing.

I'll try to justify, imagine you have working code with no tests, it's been running for years but no one can understand it or change any of it. It's providing a very fixed value. Now imagine you have all the tests laying out exactly how the features should work. Sure you don't have anything to show for yet but it's not going to take a lot of time to implement these features. The hardest part in most companies is not…

> it's not going to take a lot of time to implement these features That's so wrong. Have you ever worked in a complex program?
Post reply on HN