Questions to ask yourself when writing tests • Hacker News Board

Re: Questions to ask yourself when writing tests

#71

Post by spion » Tue, Jan 02, 2018, 1:37 PM UTC

ancarda wrote: post #41

Ace17 wrote: post #39

Earlier quoted context omitted.

> ... micro level unit testing, driven by people like Uncle Bob. "The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest." "The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring.…

I really wish Uncle Bob would provide code examples with his blog posts and videos. I struggle to imagine what my tests would look like if I followed this. Possibly as my tests are so tightly coupled right now that refactoring is actually not possible in some cases. Does anyone know of explations of this with a more hands-on approach, or is this simply a collection of ideas that can’t really be shown?

I think my example captures the spirit of this: https://news.ycombinator.com/item?id=16049441

Re: Questions to ask yourself when writing tests

#72

Post by reledi » Tue, Jan 02, 2018, 1:41 PM UTC

Couple more which haven't been mentioned yet:

- Can I run the tests in random order?

- Are the tests optimised for readability?

- Are the tests unnecessarily testing third-party code?

- Do individual tests contain the whole story? (Or do you have to look in many places to understand each test?)

Re: Questions to ask yourself when writing tests

#73

Post by blowski » Tue, Jan 02, 2018, 1:58 PM UTC

ed_blackburn wrote: post #62

blowski wrote: post #43

Earlier quoted context omitted.

There is no silver bullet. Personally, I let a combination of complexity and importance guide my tests. The more likely it is that a piece of code will break, and the more business damage it will do if it does break, the more tests I wrap around it. For self-contained algorithms that have a lot of branches or complex cases, I use more unit tests. When the complexity is in the interaction with other code, I write more…

100% In critical areas I would suggest parameterised tests are worth the effort, especially in conjunction with generators. Property-based testing in FP for example, or just test data generators that'll generate a good range.

I'm not sure what's involved there. Do you have more detail, like a good blog post, to read about how to do that?

Re: Questions to ask yourself when writing tests

#74

Post by crdoconnor » Tue, Jan 02, 2018, 2:10 PM UTC

andrewflnr wrote: post #34

simonhamp wrote: post #19

Earlier quoted context omitted.

I would be very surprised if an automated test literally took days to write when a manual test is just minutes. Also the time savings still pay off later, as automated tests usually take seconds to run and there’s no training required - once it’s in the test suite and the test suite runs are automated, it will always run and quickly identify a failure - no “oops, we forgot to show Jim the Intern that he had to test t…

Think about something that necessarily involves hardware interaction, or a GUI, or where all the interesting error cases are non-deterministic (concurrency, network error handling). Ok, on the last one you're pretty much hosed anyway. But we're not all writing nice data-in/data-out apps.

Actually, I probably wouldn't count network error handling, because you can probably use something like vaurien - https://github.com/community-libs/vaurien - to deterministically mimic bad network conditions in an integration test in a fairly reasonable amount of time.

Also, integrating that tool would probably have applications beyond a single story so even if the change takes 5 minutes and making the test work with vaurien takes half a day, it's probably still worthwhile.

Assuming that no tool like vaurien exists, though (and there are plenty of scenarios out there where you'd have to build it from scratch), building the test scenario could become prohibitively expensive.

Re: Questions to ask yourself when writing tests

#75

Post by baristaGeek » Tue, Jan 02, 2018, 2:44 PM UTC

reledi wrote: post #72

Couple more which haven't been mentioned yet: - Can I run the tests in random order? - Are the tests optimised for readability? - Are the tests unnecessarily testing third-party code? - Do individual tests contain the whole story? (Or do you have to look in many places to understand each test?)

Being able to run tests in a random order is indeed pretty important. It is not only an indicator that the features were written with high quality, but that the tests are high quality as well. Things such as the mocks mutating happen and the developer writing the tests should avoid those kinds of things happening.

Re: Questions to ask yourself when writing tests

#76

Post by michalc » Tue, Jan 02, 2018, 3:05 PM UTC

V2hLe0ThslzRaV2 wrote: post #2

>> "For every part of my code change, if I break the code, will a test fail?" Since "breaking code" is super subjective, and generally speaking, trying to "cover everything" is a recipe for hell. Anyone able to expand on what the author meant by this?

Perhaps this part is not clear / well defined, but roughly, I meant that code coverage is (about) 100% for the lines added/changed, and some "reasonable" subset of possible breaking changes would be picked up by failing tests.

What I had in mind specifically in the answer, was the case of changing "interfaces" between parts of code. For example, the case of changing a function's arguments, or how it uses them, but omitting to change a call site. Checking that the call site just calls the function would not be enough to produce a failing test, especially without type safety. The test would actually need to assert on what the function does, e.g. its return value for a pure function.

Yes, I think trying to test against every single possible breaking change is not valuable.

Re: Questions to ask yourself when writing tests

#77

Post by u801e » Tue, Jan 02, 2018, 3:06 PM UTC

yeukhon wrote: post #38

u801e wrote: post #27

Earlier quoted context omitted.

> I am so sick of having to code in extremely brittle codebases that are impossible to refactor because of the sheer volume of UTs that are tightly coupled to very specific implementation details I think this comes from "unit tests" where they have assertions such as assertThisOtherSpecificMethodNameWasCalledWithTheseSpecificallyNamedParameters If more methods were written to not call out to disk or some external ser…

> If more methods were written to not call out to disk or some external service (e.g., a database or API) and we had a main method that handled the external communication, then the unit tests would be much less fragile. I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse…

> You can write a seperate function that handles database query, then call this function in your code under test

You can, but that still requires that you mock that call in order to test the method that calls your database query method. But, if you make it such that your method takes the result of the database query as a parameter, then you can test your method without having to use a mock. In other words, you can change this (which requires mocking get_db_result to test):

  def a_method_to_test(param1, param2):
      db_result = get_db_result(param1)
      # do something with db_result

to this:

  def a_method_to_test(param1, param2, db_result)
      # do something with db_result

Now you can test a_method_to_test without having to mock out the call to get_db_result by just passing in whatever you want as the db_result parameter.

Basically, doing this will separate code that manipulates data from code that either writes or reads data. You can unit test methods that manipulate data, but you will need to do functional/integration testing of code that writes or reads data from other sources.

Re: Questions to ask yourself when writing tests

#78

Post by vog » Tue, Jan 02, 2018, 3:18 PM UTC

Ace17 wrote: post #39

discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

> ... micro level unit testing, driven by people like Uncle Bob. "The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest." "The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring.…

I agree that these types of tests are very good and usefuly, but:

Don't they use the wrong term for them?

I believe these tests are more commonly called "integration tests", not "unit tests".

Or, did I misunderstand something?

Re: Questions to ask yourself when writing tests

#79

Post by nexfitter » Tue, Jan 02, 2018, 3:30 PM UTC

reledi wrote: post #70

nexfitter wrote: post #64

> Have I just made something public in order to test it? > > If yes, consider instead writing higher level tests that each each test more of the code. This is one that I struggle with in JS with React.js components. If you have a little helper component in a file that isn't exported but used in the same file by a component that is exported, it is sometimes difficult to test that non-exported component. Because of how…

I run into the same problem with React and it bugs me from a testing standpoint. Extracting code from a big component to helper functions and extracting those functions from the component can lead to cleaner code, and it makes it much easier to test the behaviour of the helpers directly than having to render the component with enzyme. A good example of this is moving state changes to pure functions [1] which makes th…

that is a really cool technique, thanks for the link!

Re: Questions to ask yourself when writing tests

#80

Post by dvlsg » Tue, Jan 02, 2018, 4:41 PM UTC

blowski wrote: post #73

ed_blackburn wrote: post #62

Earlier quoted context omitted.

100% In critical areas I would suggest parameterised tests are worth the effort, especially in conjunction with generators. Property-based testing in FP for example, or just test data generators that'll generate a good range.

I'm not sure what's involved there. Do you have more detail, like a good blog post, to read about how to do that?

Not OP, but I particularly enjoy the fsharpforfunandprofit article about it.

https://fsharpforfunandprofit.com/posts/property-based-testi...