Superior Testing: Stop Stopping • Hacker News Board

Re: Superior Testing: Stop Stopping

#51

Post by drdeadringer » Thu, Jan 31, 2019, 8:25 PM UTC

bunderbunder wrote: post #32

olau wrote: post #21

Earlier quoted context omitted.

> but for 90% of projects, automated testing is the most efficient long term strategy I don't think this is a good way to think about it. I see plenty of projects that don't go anywhere, and I also see plenty of projects that do go somewhere, but where large parts change so little and are so easy to test manually that there's little point in much automation. Other parts are complex and full of contradictory business…

I've inherited one or two projects like the first one you describe before, and, when they don't have tests, it's awful . The problem is, regardless of how stable the code was, or how easy it was for the original author to test it manually, if the code hasn't been worked on for ages, then the original author is either gone or can't remember all the details of how it was supposed to work anymore. Which leaves you in a…

Documentation.

I swear, if it's not "we don't have time to test" it's "we don't have time to document". Ok, but you've hired all these firefighters instead.

Re: Superior Testing: Stop Stopping

#52

Post by bunderbunder » Thu, Jan 31, 2019, 8:30 PM UTC

drdeadringer wrote: post #51

bunderbunder wrote: post #32

Earlier quoted context omitted.

I've inherited one or two projects like the first one you describe before, and, when they don't have tests, it's awful . The problem is, regardless of how stable the code was, or how easy it was for the original author to test it manually, if the code hasn't been worked on for ages, then the original author is either gone or can't remember all the details of how it was supposed to work anymore. Which leaves you in a…

Documentation. I swear, if it's not "we don't have time to test" it's "we don't have time to document". Ok, but you've hired all these firefighters instead.

http://wiki.c2.com/?SharpenTheSaw

Re: Superior Testing: Stop Stopping

#53

Post by TeMPOraL » Fri, Feb 01, 2019, 8:12 AM UTC

bunderbunder wrote: post #32

olau wrote: post #21

Earlier quoted context omitted.

> but for 90% of projects, automated testing is the most efficient long term strategy I don't think this is a good way to think about it. I see plenty of projects that don't go anywhere, and I also see plenty of projects that do go somewhere, but where large parts change so little and are so easy to test manually that there's little point in much automation. Other parts are complex and full of contradictory business…

I've inherited one or two projects like the first one you describe before, and, when they don't have tests, it's awful . The problem is, regardless of how stable the code was, or how easy it was for the original author to test it manually, if the code hasn't been worked on for ages, then the original author is either gone or can't remember all the details of how it was supposed to work anymore. Which leaves you in a…

Also, if you've inherited code, or didn't work on it for couple weeks, chances are you won't know/remember how to do the manual tests, and will waste time trying to rederive it from first principles. This also applies to any other kind of "ad-hoc" ops, and multiple times more so if the activity involves multiple runtimes - e.g. issuing a shell command. I've learned this the hard way, and these days I note down the precise steps/commands for any manual op that's not very clearly a one-time thing / throwaway. Your future self and whoever inherits your code will both thank you.

Re: Superior Testing: Stop Stopping

#54

Post by dkersten » Fri, Feb 01, 2019, 9:50 AM UTC

lordnacho wrote: post #10

dkersten wrote: post #9

Earlier quoted context omitted.

I like unit tests to test the expected cases (positive or negative, as long as they're expected), to prove to myself that they conform to the spec. Then, if bugs are found (ie unexpected cases are discovered), I like to add tests for those, but I don't need to try and think of them up front. This way the unit tests will prevent regressions (both on spec conformance and bugs) as the code changes over time. Integration…

> Ideally, for catching unknown bugs I forgot to mention this part. We have data recorded from external sources, which we run through our code. Millions and millions of state changes, along with their summary states. This also helps to find cases we didn't think of.

That sounds great when you have the data available! It sounds like your software is thoroughly tested. If only where I currently work had that...

Re: Superior Testing: Stop Stopping

#55

Post by z3t4 » Fri, Feb 01, 2019, 12:04 PM UTC

hibbelig wrote: post #33

z3t4 wrote: post #28

Earlier quoted context omitted.

If it's an accounting service, make a new account/table named "Software test". Book-keeping was invented to spot errors. If you have a non zero balance in your "Software test" account you probably have a bug in the software.

So I write some code and a test, and of course the first time I run the code, the test is going to fail. So now the balance in our "Software test" account is nonzero whereas it should be zero. But audit requires us to record all bookings, so how do we explain these bogus bookings (both from the erroneous code that made the mistake in the first place, and from the manual adjustment later that fixes the mistake for the…

You can have many types of tests. First you have assertions and unit tests with mocks/injections that runs at compile time together with the type checker and other low level tests. Then you have the integration tests that makes sure your code works together with third parties and other API's. Those tests might seem unnecessary when you already have unit tests, but it's very nice to know if some third party have made breaking changes and that your software have stopped working because of it.