Live data from Hacker News

Most Unit Testing Is Waste (2014) [pdf]

rbcs-us.com

131–140 of 164 posts

Re: Most Unit Testing Is Waste (2014) [pdf]

#131

Earlier quoted context omitted.

Apart from a bit more typeing up front I cant see many costs to typing.

Try programming in Rust. The upfront cost in figuring out how to write the code to minimise unsafe()'s can be very high. This is because Rust's type system is so strong it both does the work of the garbage collector and catches things other languages would miss, like memory leaks. That aside, that "bit" of typing can amount to a lot. Python programs are usually very compact compared to statically typed languages. Yes…

Python is very compact even with heavy use of type annotations. It's not the types that add the bulk of the code, other languages usually are more verbose in other ways also.

Re: Most Unit Testing Is Waste (2014) [pdf]

#132
post #114
post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

> the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do First, in most cases you're going to test your code manually anyway - whether you write unit tests or not. So writing unit tests is just in addition to the time already spent testing manually. But most importantly, unit tests often do very little to prove "your code actually does…

> First, in most cases you're going to test your code manually anyway

Only if you are coding things with UI's. If you are creating a new API endpoint on some service or creating a new service or adding some new parts to a maths library there is no point in manual testing.

I feel a lot of the disagreements people have on development methodologies comes down to people working in different domains. When I do UI work I do mostly manual testing, when I do library work I do only unit testing.

Re: Most Unit Testing Is Waste (2014) [pdf]

#133

This paper is emblematic of a serious problem in the software development field: lack of empirical research. It is unsurprisingly filled with opinion and anecdote, with little mention of research in the area. And in the one actual study cited, "Does Test-Driven Development Really Improve Software Design Quality?", the author mis-characterizes the findings. Not that there is a huge set of research on unit testing to r…

I agree entirely. The more tendency there is to push development as an "engineering" discipline, the more evidence based research needs to be used to solidify or disprove certain development dogmas repeated over and over again, which in many cases, seemingly have no empirical evidence beyond a few anecdotal success cases. This is the norm in the industry, "you're doing it wrong, the best way is this way..."--based on…

Is there a good way to fix it? I mean, if you were building bridges, certainly another engineer could show you data about material properties and successful builds, and you could reasonably extrapolate to your new (unique) design fairly well, but that's because you're building with the same physical materials and operating under the same laws. With software, it is a bit harder to find that common ground, because the 'laws' change if you are on a different architecture or handling different kinds of 'traffic', etc. For a bridge, you can just make a bigger beam because you would never try to build close to the scale which would make the material properties irrelevant, but with a computer, you can do this, and you might start to invalidate prior evidence. A large amount of software development dogma is built around scaling up, but that's also a huge problem in every engineering & scientific discipline.

Though, to counter my own counter-point, it would be nice to have better analytical tools; more informative & accurate ways to understand & visualize how resources are actually being used when a program runs.

Re: Most Unit Testing Is Waste (2014) [pdf]

#134
post #124

Earlier quoted context omitted.

For the same number of lines of test code, an integration test case traverses through much more source code than a unit test case. Not only that, but it also checks that the units are interacting properly and passing each other the right data and returning the right data. Good integration tests can also fee us from static typing because the integration tests implicitly check that the interface contracts between all t…

True, but they're also slower, so you'll have to either test fewer cases or run the tests less often.

They are usually slower but they can still be pretty fast. Also, with integration tests, you can choose the granularity of your test. You don't need to test end-to-end (of course this is the slowest), you can just test a single module with all its dependencies. If you don't mock out dependencies, then it's still technically called an integration test.

Re: Most Unit Testing Is Waste (2014) [pdf]

#135

Earlier quoted context omitted.

What if most type system security is a waste? I don't personally believe this is the case, but I also can't tell you I know it to be false. What I do know is that types have a non-zero cost, and I think sometimes that's ignored, and only their benefits are acknowledged. But the efforts to bring types to historically dynamic languages show that not all common dynamic language idioms are amenable to reasonable type sig…

Apart from a bit more typeing up front I cant see many costs to typing.

There are costs, and they go up significantly the more sophisticated your type system is/the more speicifc you try to be with your types.

The way I see it there are two kinds of costs type systems may impose. Very simplistic type systems, such as the ones in C, Go, and even C++, Java, C#, often force you to write code that is too specific, thus forcing duplication (C and Go especially suffer terribly from not having generics - in Go especially its impossible to write a well typed function that can be passed an array of any kind and return an element of that array).

On the other hand, of your type system is powerful enough, you still get a similar problem when trying to write code that is abstract enough to be reusable, but specific enough to be safe.

For example, if you were writing code for a physical simulation system, you would want to have all quantities carry their measurement unit in their type, to avoid mismatches.

However, you wouldn't want to rewrite maths code for each measurement unit. However, to properly keep track of types in complex maths code gets pretty ugly pretty quickly. For example, a function that wants to compute the scalar product of 2 2-dimensional vectors would only work if the first values of each vector have the same/convertible units as the second pair, in any order (m,s dot s,m should be ok). If you move to matrix multiplication, the types become even more complicated, and this is still very basic algebra. If you want your function to be applicable to arbitrary matrix sizes you're already writing a complex library.

Re: Most Unit Testing Is Waste (2014) [pdf]

#136

Earlier quoted context omitted.

What if most type system security is a waste? I don't personally believe this is the case, but I also can't tell you I know it to be false. What I do know is that types have a non-zero cost, and I think sometimes that's ignored, and only their benefits are acknowledged. But the efforts to bring types to historically dynamic languages show that not all common dynamic language idioms are amenable to reasonable type sig…

Apart from a bit more typeing up front I cant see many costs to typing.

I have two thoughts on this.

First, static typed languages often simply subtract features that would be available in dynamic languages. So, that's a cost that's paid on day zero, and it can be easy to forget about.

Secondly, there are absolutely times when something that is conceptually easy to describe is a real beast to model in a type system. Think DSLs for SQL, for example. A non-trivial amount of time can be lost fighting these battles.

Re: Most Unit Testing Is Waste (2014) [pdf]

#137
post #90

Earlier quoted context omitted.

While I agree that this is hard to implement with a type system, I never want to see this implemented as a unit test. What you describe is a real use case that I personally want to have tested on a real system with all the stuff in place (but test database instead of the real one). It has nothing to do with one unit you can test on its own. And the crucial problem here is that you tie your testing code on the inner w…

With TDD, writing unit tests accompanies writing code - which in itself necessities thinking in terms of testability, which is a good thing making the program's components more separable. Regarding the former example, mocking only a single part where it connects to other parts of the system (for example some interface through which it sends warnings to users) is easy and makes the resulting code better (eg, easy to r…

> With TDD, writing unit tests accompanies writing code - which in itself necessities thinking in terms of testability

I, as a user of your software, only care about the feature, and that the feature works. I don't care about testability, whatever that means. And since I case about features, I think this entails that features have to be tested end to end, and not some unrelated classes ("mocks") that are not even used in the final product.

> is easy and makes the resulting code better (eg, easy to replace the notification manager).

It might or it might not. I have seen too much code which was written way too complex which a lot of unnecessary classes and interfaces, just for the sake of "testability". However, some features did not work every once in a while.

If you have the need for different notification backends, go ahead. Even if you need a notification manager (whatever this is) for this. But don't make code more complicated (sometimes called "test-induced design damage") without need.

Re: Most Unit Testing Is Waste (2014) [pdf]

#138

Earlier quoted context omitted.

I actually don't get where you are going with this breakdown. Can you give me an example.

I mean when you have a class which, instead of importing/including/requiring its dependencies directly, expects instances of various classes to be passed into its constructor for example. So the class cannot do its job unless you pass its dependencies into its constructor... This is an antipattern which unit tests tend to encourage because it allows you to decouple the unit logic from its dependencies and it makes it…

Passing dependencies in the constructor is the only acceptable way of having stateful dependencies (like a database or some external services). I am assuming you are against passing things like Math libraries by creating an instance, which I would agree with.

The only reason dependency injection gets a bad rep is magical frameworks which obscure the actual wiring and end up causing bigger problems.

Re: Most Unit Testing Is Waste (2014) [pdf]

#139
post #89

I did not read the article. I did read all comments Unit tests for the most part aren't about "testing". They are developer tool. To verify rthat modifications (refactoring, additions, bug fixes, etc) doesn't break contracts etc. Oh and showing that your code is a codependent mess of poorly isolated spagehtti, if your unit tests are hard to write, the code under test is a mess. Inittests are more useful in languages…

Surprisingly, the article doesn't engage with this argument! Unless I missed it, the author never mentions that unit tests can make refactoring other code easier. In my experience, after code is checked in, unit tests have two main purposes: 1. Checking that functions aren't completely broken in dynamically typed languages (i.e. a check so basic that a static type checker can do its job) 2. Allowing programmers to re…

I don't think I've ever seen unit tests that didn't need to be changed fundamentally to deal with any sort of refactoring more complex than extracting a function.

Unit tests are supposed to work with the internals of a module and not just its boundaries. Since refactoring normally tries to significantly change the internals without changing the boundary, it usually also results in a rewrite of the unit tests. Integration tests are the ones that usually help me sleep better after a refactor.

Re: Most Unit Testing Is Waste (2014) [pdf]

#140
post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

Code needs to be tested anyway, but integration and functional tests can accomplish that more cost effectively.

Cannot tell you how many times I have seen projects with hundreds of green unit tests at 90% line coverage and yet regress on, miss important and obvious cases, or simply have never had their headline functionality.

Post reply on HN