Most Unit Testing Is Waste (2014) [pdf] • Hacker News Board

Re: Most Unit Testing Is Waste (2014) [pdf]

#151

Post by msclrhd » Sun, Jun 30, 2019, 6:38 AM UTC

When implementing lexers and parsers for language specifications, I have taken the approach of testing each token (keyword, number, etc.) for the lexer for each EBNF symbol. This repeats tests for common keywords and other constructions, but means I can see that the lexer handles all the tokens in each EBNF symbol. In this way, these tests are more BDD (behavioural driven development) in style, but without things like gherkin/cucumber. With this approach, I do full coverage tests for a given symbol/token, and just check the basic symbol/token case elsewhere.

I do a similar thing with the parser tests -- have a test for each valid complete symbol production, and then tests for as many error cases that the parser can handle for that symbol.

With that base, I can add additional tests for bugs and additional error recovery cases I implement.

I never see unit tests as a waste as they provide a set of regression tests that are invaluable for refactoring and making other changes to the code, like implementing new features or better error handling.

Re: Most Unit Testing Is Waste (2014) [pdf]

#152

Post by msclrhd » Sun, Jun 30, 2019, 7:01 AM UTC

winrid wrote: post #130

falsedan wrote: post #74

Earlier quoted context omitted.

Who cares what they’re called as long as they’re fast and don’t crash when running at scale/fill the disk or memory

Because if you're not using mocks the tests get unweildy. Suddenly changing components three layers down the dependency tree breaks "unit" tests at the top. Those are the tests you just end up @ignore'ing.

Mock things like database access, or better yet pass the database connection via an interface and create a test version of the database access so you can create a database-like thing for your tests (e.g. an in-memory database).

If you are depending on another component of the application (a lexer, a JSON class, a maths function, etc.) don't mock that because if that class breaks you want tests to fail instead of silently passing because the broken class/function was mocked.

If you are depending on thirdparty libraries, don't mock those unless you have to (i.e. if you cannot run the tests). This will help avoid unexpected bugs after upgrading libraries or if supporting different versions of a library.

If you are writing code for a complex infrastructure (e.g. an plugin for an IDE), try to use test-specific versions of enough of the infrastructure to get the rest functioning, and use the real versions of as much as you can. This will help pick up issues in your code when new versions of that infrastructure make changes -- you want your tests to fail in this case, as the real code would fail.

Understand why your tests are breaking and address that. Use @Ignore as a last resort. If APIs or behaviour has changed, update the code and tests to reflect that. If you are supporting different versions, create version-specific compatibility layers.

Re: Most Unit Testing Is Waste (2014) [pdf]

#153

Post by floriol » Sun, Jun 30, 2019, 7:11 AM UTC

walshemj wrote: post #87

sulam wrote: post #38

Earlier quoted context omitted.

This is no more or less likely in an integration test vs a unit test. One of the first things you learn in writing unit tests is that you should be selective in picking your inputs. This is even taught in those “trade schools” he subtly derides. You regularly see both public test cases and hidden test cases run against problem sets, at least that’s what my interns tell me.

Interesting so they don't teach testing by deliberately trying to break things, pasting Arabic text into an address field for example. When testing NAPI like to use edge cases like the company with the longest name eg "Donaudampfschiffahrtsgesellschaft" or the Famous Welsh location "Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" I am pretty sure I crashed an A17 clearpath mainframe by doing aggressive te…

That's QA, not the sort of testing we are talking about. In case QA happens to find some bug like that, after the fix, we should include another test case with the formerly breaking one

Re: Most Unit Testing Is Waste (2014) [pdf]

#154

Post by Pistos2 » Sun, Jun 30, 2019, 7:55 AM UTC

ChrisCinelli wrote: post #66

biesnecker wrote: post #4

My experience with testing is like that old adage about advertising, "I know I'm wasting 50% of my money but I don't know which 50%." Most of it is a waste but it's hard to know in advance which test will stop an engineer in a couple of years from altering some fundamental contract and bringing the system down.

I have a 3 part article on testing that I need to post... Regarding wasting time on 50% tests... it depends what your system is. In most of the systems (the no mission critical ones, no general libraries) I like to use the 80%/20% rule. I always ask: what are the 20% of the tests that deliver 80% of the value? You learn with experience but if you do not have it, this is a simple way to learn (as a team). During the s…

> If you also take a look at the tests than never fail, you will also have a sense of the bugs you should not write.

Just checking: Did you mean "the tests you should not write"?

Re: Most Unit Testing Is Waste (2014) [pdf]

#155

Post by cryptica » Sun, Jun 30, 2019, 8:45 AM UTC

tsimionescu wrote: post #138

cryptica wrote: post #129

Earlier quoted context omitted.

I mean when you have a class which, instead of importing/including/requiring its dependencies directly, expects instances of various classes to be passed into its constructor for example. So the class cannot do its job unless you pass its dependencies into its constructor... This is an antipattern which unit tests tend to encourage because it allows you to decouple the unit logic from its dependencies and it makes it…

Passing dependencies in the constructor is the only acceptable way of having stateful dependencies (like a database or some external services). I am assuming you are against passing things like Math libraries by creating an instance, which I would agree with. The only reason dependency injection gets a bad rep is magical frameworks which obscure the actual wiring and end up causing bigger problems.

I also would not pass in a database client instance. I would pass in the config for the database client though.

I think that the class which ineteracts with the database directly via the client should be tightly coupled to the database client. It's not very often that you change database and when you do, you can just swap out that entire class completely. Classes which interact with the database should expose simple interfaces for performing actions against the database and those wrappers should be replaceable.

Re: Most Unit Testing Is Waste (2014) [pdf]

#156

Post by he0001 » Sun, Jun 30, 2019, 10:00 AM UTC

I run a one-man project and I write test first unit tests for all of my code. This has enabled me to advance the code with new functions without breaking current API’s. The tests are not in the way and usually doesn’t need to change because I change the implementation. It has been tremendous for productivity because I don’t need to worry about if stuff breaks. I can trust my tests for changing those and so far they have always done that.

Re: Most Unit Testing Is Waste (2014) [pdf]

#157

Post by derangedHorse » Sun, Jun 30, 2019, 11:40 AM UTC

This article seems to ignore the fact that unit tests are typically made for other programmers down the line as a form of documentation, as an informal description of business rules, and as a way to gauge whether new code will break the product in a very obvious way.

Re: Most Unit Testing Is Waste (2014) [pdf]

#158

Post by [deleted] » Sun, Jun 30, 2019, 2:42 PM UTC

0x445442 wrote: post #120

Frost1x wrote: post #104

Earlier quoted context omitted.

I agree entirely. The more tendency there is to push development as an "engineering" discipline, the more evidence based research needs to be used to solidify or disprove certain development dogmas repeated over and over again, which in many cases, seemingly have no empirical evidence beyond a few anecdotal success cases. This is the norm in the industry, "you're doing it wrong, the best way is this way..."--based on…

Agree with you and parent poster. I remember 15 years ago or more going through CMMI certification and there was a push to get to level 3. Our company hired a consultant who came in every eight weeks or so to track our progress, give direction etc. On one of his visits he started interviewing engineers individually to see how things were going and I asked him what the point of it all was and I could tell he was quite…

[deleted]

Re: Most Unit Testing Is Waste (2014) [pdf]

#159

Post by tsimionescu » Sun, Jun 30, 2019, 4:51 PM UTC

cryptica wrote: post #155

tsimionescu wrote: post #138

Earlier quoted context omitted.

Passing dependencies in the constructor is the only acceptable way of having stateful dependencies (like a database or some external services). I am assuming you are against passing things like Math libraries by creating an instance, which I would agree with. The only reason dependency injection gets a bad rep is magical frameworks which obscure the actual wiring and end up causing bigger problems.

I also would not pass in a database client instance. I would pass in the config for the database client though. I think that the class which ineteracts with the database directly via the client should be tightly coupled to the database client. It's not very often that you change database and when you do, you can just swap out that entire class completely. Classes which interact with the database should expose simple…

Then how would you handle connection pooling, if every class which interacts with the database has its own instance of the database client?

Re: Most Unit Testing Is Waste (2014) [pdf]

#160

Post by anoncake » Sun, Jun 30, 2019, 7:43 PM UTC

Too wrote: post #131

rstuart4133 wrote: post #97

Earlier quoted context omitted.

Try programming in Rust. The upfront cost in figuring out how to write the code to minimise unsafe()'s can be very high. This is because Rust's type system is so strong it both does the work of the garbage collector and catches things other languages would miss, like memory leaks. That aside, that "bit" of typing can amount to a lot. Python programs are usually very compact compared to statically typed languages. Yes…

Python is very compact even with heavy use of type annotations. It's not the types that add the bulk of the code, other languages usually are more verbose in other ways also.

In a statically typed language, you sometimes have to make the code more verbose to satisfy the type checker. In Python, you can just not use it in those cases.