Questions to ask yourself when writing tests • Hacker News Board

Re: Questions to ask yourself when writing tests

#61

Post by ed_blackburn » Tue, Jan 02, 2018, 12:10 PM UTC

ancarda wrote: post #41

Ace17 wrote: post #39

Earlier quoted context omitted.

> ... micro level unit testing, driven by people like Uncle Bob. "The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest." "The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring.…

I really wish Uncle Bob would provide code examples with his blog posts and videos. I struggle to imagine what my tests would look like if I followed this. Possibly as my tests are so tightly coupled right now that refactoring is actually not possible in some cases. Does anyone know of explations of this with a more hands-on approach, or is this simply a collection of ideas that can’t really be shown?

Practical advice? Consider making every type internal. Then open them up to what is strictly required to be public.

Now you are testing the behaviour of your application through its public surface. This reduces the brittleness of your tests because you can change the internal implementation without rewriting your tests. You have higher assurance. It'll also force your hand to enforce invariants and place guard clauses in the right places rather than "everywhere".

If you follow Cockburn's Ports and Adapters approach too, then you can substitute adapters like persistent state, buses, HTTP clients for appropriate in-memory equivalents.

Re: Questions to ask yourself when writing tests

#62

Post by ed_blackburn » Tue, Jan 02, 2018, 12:12 PM UTC

blowski wrote: post #43

seanwilson wrote: post #35

Earlier quoted context omitted.

The argument I get against this is that when a higher level test fails, it's harder to locate the lines of code that broke the test unlike when you have lots of unit tests. I don't find this convincing though. Most of the time it's going to be pretty obvious looking at what lines of code have changed since the last working commit. Lots of projects get by without any tests at all as well. I'm not saying to skip testin…

There is no silver bullet. Personally, I let a combination of complexity and importance guide my tests. The more likely it is that a piece of code will break, and the more business damage it will do if it does break, the more tests I wrap around it. For self-contained algorithms that have a lot of branches or complex cases, I use more unit tests. When the complexity is in the interaction with other code, I write more…

100% In critical areas I would suggest parameterised tests are worth the effort, especially in conjunction with generators. Property-based testing in FP for example, or just test data generators that'll generate a good range.

Re: Questions to ask yourself when writing tests

#63

Post by albertgoeswoof » Tue, Jan 02, 2018, 12:26 PM UTC

falsedan wrote: post #57

simonhamp wrote: post #19

Earlier quoted context omitted.

I would be very surprised if an automated test literally took days to write when a manual test is just minutes. Also the time savings still pay off later, as automated tests usually take seconds to run and there’s no training required - once it’s in the test suite and the test suite runs are automated, it will always run and quickly identify a failure - no “oops, we forgot to show Jim the Intern that he had to test t…

Let me help expand your imagination: what if the code is for sshing to prod and needs a ssh-agent or a password typed in? Would you leave those cress on your CI system? Would Security be happy with that?

Why would a test build ever ssh to production?

If you absolutely need to test connecting to an external system you should mock it or create an environment for testing against.

Your credentials should also be stored as env variables so the test ones are used only for the test environment.

Re: Questions to ask yourself when writing tests

#64

Post by nexfitter » Tue, Jan 02, 2018, 12:48 PM UTC

> Have I just made something public in order to test it? > > If yes, consider instead writing higher level tests that each each test more of the code.

This is one that I struggle with in JS with React.js components. If you have a little helper component in a file that isn't exported but used in the same file by a component that is exported, it is sometimes difficult to test that non-exported component. Because of how enzyme shallow rendering works you don't get the full tree so that component, if sufficiently nested, might not ever be touched. This forces me to export the component just to test it.

example: https://imgur.com/a/gqgcS

Re: Questions to ask yourself when writing tests

#65

Post by crdoconnor » Tue, Jan 02, 2018, 1:01 PM UTC

Ace17 wrote: post #39

discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

> ... micro level unit testing, driven by people like Uncle Bob. "The structure of your tests should not be a mirror of the structure of your code. The fact that you have a class named X should not automatically imply that you have a test class named XTest." "The structure of the tests must not reflect the structure of the production code, because that much coupling makes the system fragile and obstructs refactoring.…

>Then, clearly, if a small change to the production code causes large changes to the tests, there is a design problem. > >Therefore the tests must have their own design. They cannot simply follow along with the structure of the production code.

The jump indicated by the "therefore" confuses me. I essentially couple my test to production behavior, insofar as it is possible. I do this to test program behavior realistically and because I believe tests should, in some sense, form a specification for the program they test. I don't really care about "testing code"; I care about testing behavior.

I don't have the problem of small changes to code necessitating large changes to tests. It makes me wonder what he does that is causing that.

Re: Questions to ask yourself when writing tests

#66

Post by yeukhon » Tue, Jan 02, 2018, 1:03 PM UTC

discreteevent wrote: post #59

yeukhon wrote: post #38

Earlier quoted context omitted.

> If more methods were written to not call out to disk or some external service (e.g., a database or API) and we had a main method that handled the external communication, then the unit tests would be much less fragile. I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse…

Then maybe just consider doing functional testing. Assertion code in a mock is, by definition, code that tests an implementation. In the example given, suppose you wanted to replace the database with a nosql version. Now you have to throw away or modify your tests even though the behaviour of your system has not changed

> In the example given, suppose you wanted to replace the database with a nosql version. Now you have to throw away or modify your tests even though the behaviour of your system has not changed

Well, not really. Because now I have the DB stuff all contained in a separate function (or handler which can be proxy to the correct DB engine/type of DB), so if with careful design I should be okay.

    def create_account(user_manager, user_details):
       user_manager.create_account(user_details)

Here user_manager can abstracts away the exact DB type/engine like Django.

Of course this is ideal, but that was the motive. I do agree on mocking == testing implementation in general, and functional testing is almost always the way to go... since mocking returns dummy data / fixture, might as well return from a real database. The downside is speed :/ (there are various of tricks like setUpClass, run every test or group of tests in docker containers in parallel, but takes much longer than UTs). Ugh trade-offs.

Re: Questions to ask yourself when writing tests

#67

Post by yeukhon » Tue, Jan 02, 2018, 1:07 PM UTC

lmm wrote: post #44

yeukhon wrote: post #38

Earlier quoted context omitted.

> If more methods were written to not call out to disk or some external service (e.g., a database or API) and we had a main method that handled the external communication, then the unit tests would be much less fragile. I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse…

Rather than having 3 different business logic methods that call out to the database, you have each construct a command object and then have a separate service that calls out to the database based on these commands. You can then test the 3 business logic methods by testing that they construct the correct commands based on their parameters.

Thank you. Would you mind to elaborate a little more with perhaps a little pseudo-code snippet?

Re: Questions to ask yourself when writing tests

#68

Post by crdoconnor » Tue, Jan 02, 2018, 1:13 PM UTC

seanwilson wrote: post #35

discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

The argument I get against this is that when a higher level test fails, it's harder to locate the lines of code that broke the test unlike when you have lots of unit tests. I don't find this convincing though. Most of the time it's going to be pretty obvious looking at what lines of code have changed since the last working commit. Lots of projects get by without any tests at all as well. I'm not saying to skip testin…

>The argument I get against this is that when a higher level test fails, it's harder to locate the lines of code that broke the test unlike when you have lots of unit tests. I don't find this convincing though.

This is a valid observation but the problem clears up when you start integrating better debugging tools into your testing infrastructure: having the ability to intercept and inspect API calls made by your app, launch a debugger in in the middle of the test, etc.

It is also minimized by writing the test at the same time (or just before) the code that makes it pass.

Re: Questions to ask yourself when writing tests

#69

Post by mooreds » Tue, Jan 02, 2018, 1:15 PM UTC

KirinDave wrote: post #16

simonhamp wrote: post #6

The most important question to ask when writing tests: Can I do this faster if I don’t write a test? The answer is always no. Even if you are the only person building something, future you will lick your boots clean in gratitude if there are tests. Because even the best developers have to work with their own code sometimes.

I wish people would extend this same logic to type systems and formal methods.

Isn't the wave of FP causing some folks to do this? It takes time for people to change their attitudes. Consider that we're still having this conversation about automated testing, which was promoted by the XP book in the early 2000s.

Re: Questions to ask yourself when writing tests

#70

Post by reledi » Tue, Jan 02, 2018, 1:30 PM UTC

nexfitter wrote: post #64

> Have I just made something public in order to test it? > > If yes, consider instead writing higher level tests that each each test more of the code. This is one that I struggle with in JS with React.js components. If you have a little helper component in a file that isn't exported but used in the same file by a component that is exported, it is sometimes difficult to test that non-exported component. Because of how…

I run into the same problem with React and it bugs me from a testing standpoint.

Extracting code from a big component to helper functions and extracting those functions from the component can lead to cleaner code, and it makes it much easier to test the behaviour of the helpers directly than having to render the component with enzyme.

A good example of this is moving state changes to pure functions [1] which makes them much easier to test, but you'll need to export those functions to test them.

1: https://twitter.com/dan_abramov/status/824310320399319040