Questions to ask yourself when writing tests • Hacker News Board

Re: Questions to ask yourself when writing tests

#91

Post by newfoundglory » Wed, Jan 03, 2018, 8:52 AM UTC

arkh wrote: post #88

newfoundglory wrote: post #85

Earlier quoted context omitted.

I disagree and am horrified by the dystopic image of our entire test suite having two possible results: "everything passed" or "something is wrong".

Not two results. But "when you click on this after that this happen while it shouldn't" is usually enough info to start debugging: you have reproducible steps. Which you can do with a debugger running so you see exactly were and how things break. And it is a lot less brittle than "well we tried to refactor some minor thing and now everything is red; but we don't know if the software behavior changed or just because o…

well, if you're determined to write stupid tests you can do it at any level. And that is not enough info to start debugging unless you had the machine in a known and reproducible state to begin with, which is where we get back to writing focused reproducible tests.

Re: Questions to ask yourself when writing tests

#92

Post by lmm » Wed, Jan 03, 2018, 4:18 PM UTC

yeukhon wrote: post #67

lmm wrote: post #44

Earlier quoted context omitted.

Rather than having 3 different business logic methods that call out to the database, you have each construct a command object and then have a separate service that calls out to the database based on these commands. You can then test the 3 business logic methods by testing that they construct the correct commands based on their parameters.

Thank you. Would you mind to elaborate a little more with perhaps a little pseudo-code snippet?

https://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Mo... is a nice simple example if you don't mind reading Haskell. (Redis rather than a database, but the technique is the same)

Re: Questions to ask yourself when writing tests

#93

Post by michalc » Wed, Jan 03, 2018, 6:31 PM UTC

sethammons wrote: post #31

> Mocking introduces assumptions, which introduce risk. This boils down something I've had on my mind a lot of late. Though, with a different spin. I write a lot of Go. I prefer testing interfaces while some others what to use mock generators. This quote captures part of my reasoning behind avoiding mocks. I plan to write a detailed post at some point full of examples. I think this quote will work its way in there.

I would be interested in reading it... if / when, if you can send me a link, that would be great!

Re: Questions to ask yourself when writing tests

#94

Post by twic » Wed, Jan 03, 2018, 7:26 PM UTC

hdi wrote: post #53

ancarda wrote: post #41

Earlier quoted context omitted.

I really wish Uncle Bob would provide code examples with his blog posts and videos. I struggle to imagine what my tests would look like if I followed this. Possibly as my tests are so tightly coupled right now that refactoring is actually not possible in some cases. Does anyone know of explations of this with a more hands-on approach, or is this simply a collection of ideas that can’t really be shown?

I've been wondering about that for a while. So far, I've not been able to find any examples of test architecture that follows those principles online or in my day job. I've also been too lazy to do a side project and explore a more decoupled and scalable test suite. Maybe I should get off my arse and finally do it.

If you're writing a web application, i've had a reasonably happy time with tests at the controller level, or perhaps right below. You want an interface where values go in and out that correspond to fairly user-level ideas, but which are still tractable for testing.

Re: Questions to ask yourself when writing tests

#95

Post by twic » Wed, Jan 03, 2018, 7:34 PM UTC

ajmurmann wrote: post #5

discreteevent wrote: post #4

"consider instead writing higher level tests that each each test more of the code." I really like the way they keep repeating this as the answer to so many questions. To me it reads as a softly softly approach to weaning people off what appears to be a mania for micro level unit testing, driven by people like Uncle Bob.

I'm not sure I agree with the "instead" in his recommendation. "micro level unit testing" comes with its own benefits, like faster feedback and also design feedback that integration or acceptance style tests won't give you. I like to start out with a high level acceptance tests and then write unit tests for every change to involved classes. You really need a healthy testing pyramid.

> You really need a healthy testing pyramid.

I don't think you do. I think a testing pyramid is a sign of an unhealthy approach and an unhealthy codebase.

I think what you need is a testhenge. For those who aren't familiar with henges, here's a typical (although poorly maintained) one:

http://www.english-heritage.org.uk/remote/www.english-herita...

Rather than being massive at the bottom and then getting thinner as it goes up, there is a continuous band around the top, covering everything, supported by uprights that dive deeper down, just where needed.

That is, you need a layer of high-level tests for everything (for a web application, some mix of browser tests or controller-level integration tests), along with component-level integration tests and isolated unit tests for the bits of the system that are particularly risky (complex, error-prone, critical, new, old, etc).

Re: Questions to ask yourself when writing tests

#96

Post by atticusCr » Thu, Jan 04, 2018, 12:29 AM UTC

viraptor wrote: post #89

atticusCr wrote: post #84

Earlier quoted context omitted.

Kind of, if you have a centralized place to perform input data validation, as it should, then it is just a matter to test that piece of code same if you are using a framework. However, I don't understand why you refer to a db in the first place? Is it because I used the injection attack as an example? if that's the case bare in mind that Injection target other interprets as well not only a db. But getting back to my…

Yeah, my mind substituted parameter with query parameter. Too much database stuff at my $dayjob recently and I get tunnel vision ;-)

lol! thanks for your comments.

Re: Questions to ask yourself when writing tests

#97

Post by michalc » Fri, Jan 05, 2018, 8:53 PM UTC

yawaramin wrote: post #24

Looks like Michal has been immersed in Haskell for at least the past year. I wonder if he will have something to say about balancing testing and coverage with static typing.

Thinking about this, at the moment, I don't think static typing is a replacement for testing (or vice versa!). Although, apparently, with something like LiquidHaskell, you can get more logic "into" the types and checked by the compiler, but I'm unsure how much you can do when it comes to more complex business logic.

Regarding testing "glue": static typing often gives evidence (but not proof) that code is glued together appropriately [refactoring even small projects without tests in Haskell is a joy: the compiler essentially tells you what to change]. However, it doesn't give evidence that the high level behaviour is what it needs to be. So I think higher level tests are still needed.

I think maybe changing the first question from...

> Am I confident the feature I wrote today works because of the tests and not because I loaded up the application?

to...

> Am I confident the feature I wrote today works because of the tests and type checking, and not because I loaded up the application?

will probably help you to answer the question about how much static types allow you to forgo tests. My instinct is that in most cases, high level tests are still worthwhile.

Re: Questions to ask yourself when writing tests

#98

Post by yawaramin » Sat, Jan 06, 2018, 4:20 AM UTC

michalc wrote: post #97

yawaramin wrote: post #24

Looks like Michal has been immersed in Haskell for at least the past year. I wonder if he will have something to say about balancing testing and coverage with static typing.

Thinking about this, at the moment, I don't think static typing is a replacement for testing (or vice versa!). Although, apparently, with something like LiquidHaskell, you can get more logic "into" the types and checked by the compiler, but I'm unsure how much you can do when it comes to more complex business logic. Regarding testing "glue": static typing often gives evidence (but not proof) that code is glued togeth…

I'm thinking much the same, though I do think of tests as looking after the dynamic behaviour and types after the static behaviour. Which seems obvious in retrospect, but once you wrap your head around it you can build neat abstractions like lightweight static capabilities: https://github.com/yawaramin/lightweght-static-capabilities