Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superior Testing: Stop Stopping

arturdryomov.online

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Superior Testing: Stop Stopping

#21

Post by olau »

kstenerud wrote: post #6

Regardless of all the arguments for and against tests, it's important to remember your purpose as an IT professional: Your job is to solve the customer's problem. That's it. HOW you solve their problems is entirely up to you, and some solutions will work better than others. The only thing about customers guaranteed to be consistent is that they will request changes. Some will be good changes. Some will be bad changes…

> but for 90% of projects, automated testing is the most efficient long term strategy

I don't think this is a good way to think about it.

I see plenty of projects that don't go anywhere, and I also see plenty of projects that do go somewhere, but where large parts change so little and are so easy to test manually that there's little point in much automation. Other parts are complex and full of contradictory business rules so formal tests are necessary.

Of course, YMMV. If you're inside Google, things are probably looking very different. But most people aren't.

Re: Superior Testing: Stop Stopping

#22

Post by purerandomness »

arkh wrote: post #3
chriswarbo wrote: post #2

I heard the "customers don't pay for tests" line recently, from someone boasting about how they migrated hundreds of thousands of lines of PHP from 5.x to 7.x. I didn't have an off-the-cuff reply to this, but thinking about it afterwards the issue became clear to me: customers don't pay for code either! Customers pay for solutions to their problems. I think the key phrase is "how do we know?": maybe those hundreds of…

> customers don't pay for code either! Customers pay for solutions to their problems. And that's why you don't test code. You test that your application comply with your customer's requirements. I'm happy to see things changing about tests with people realizing fine grained unit tests are often an hindrance and you should prioritize end to end testing. Test the interface of what you're selling, not the inner workings…

Are you aware of the Test Pyramid, and why it exists?

Re: Superior Testing: Stop Stopping

#23

Post by arkh »

purerandomness wrote: post #22
arkh wrote: post #3

Earlier quoted context omitted.

> customers don't pay for code either! Customers pay for solutions to their problems. And that's why you don't test code. You test that your application comply with your customer's requirements. I'm happy to see things changing about tests with people realizing fine grained unit tests are often an hindrance and you should prioritize end to end testing. Test the interface of what you're selling, not the inner workings…

Are you aware of the Test Pyramid, and why it exists?

> Are you aware of the Test Pyramid

Yes.

> and why it exists?

Because the xUnit consultant crowd worked a lot, it is easier to make unit tests (which fossilize your code) so there's a lot of tooling around it. And the reliance on external services with no sandbox or ready to use mocks mean E2E is harder to implement. But being harder just mean you have to do your job.

When I buy your software I don't care about how you implemented some pattern. What I care is that when I click on this gizmo in that situation it does what it should in some time-frame using some resources.

Re: Superior Testing: Stop Stopping

#24

Post by moring »

The introduction made me hope for real advice on how to "stop stopping", but it only repeated the old arguments on why we want tests. Why not deal with some of the real problems that actually prevent people from writing automated tests?

Real-world example 1: The system being developed talks to external system X. We don't want tests to litter the production database, especially since it's about accounting and we would get into legal trouble for that. However, there is no possibility to open a test account on the production system, and no budget for a license for a test installation of X. The main troiuble with X is that its public API (web service) changes from version to version and there is no useful documentation about it. How would one write integration tests for that?

Real-world example 2: How would you write tests for a system that has its requirements unspecified, even on a very coarse level, after the deadline where it is forced into production by management?

I'm pretty sure that both examples happen in other places than the ones I've seen them, too.

Re: Superior Testing: Stop Stopping

#25

Post by crdoconnor »

moring wrote: post #24

The introduction made me hope for real advice on how to "stop stopping", but it only repeated the old arguments on why we want tests. Why not deal with some of the real problems that actually prevent people from writing automated tests? Real-world example 1: The system being developed talks to external system X. We don't want tests to litter the production database, especially since it's about accounting and we would…

1. Create a facade abstraction (it could be a microservice, API or even command line app) over the expensive to test thing. Make the abstraction really dumb and easy to test manually in isolation from the rest of the app. Integration test only against the abstraction, not the real thing and do a minimal level of end to end manual testing against the real thing.

2. Don't. If you don't have fixed requirements your tests will have negative ROI.

Re: Superior Testing: Stop Stopping

#26

Post by corebit »

moring wrote: post #24

The introduction made me hope for real advice on how to "stop stopping", but it only repeated the old arguments on why we want tests. Why not deal with some of the real problems that actually prevent people from writing automated tests? Real-world example 1: The system being developed talks to external system X. We don't want tests to litter the production database, especially since it's about accounting and we would…

Nothing prevents people from writing automated tests except their own shortsightedness.

You're going to spend extra time on the the problems you identified. You can either plan for it up front with tests, or be unprofessional and firefight later.

Re: Superior Testing: Stop Stopping

#28

Post by z3t4 »

moring wrote: post #24

The introduction made me hope for real advice on how to "stop stopping", but it only repeated the old arguments on why we want tests. Why not deal with some of the real problems that actually prevent people from writing automated tests? Real-world example 1: The system being developed talks to external system X. We don't want tests to litter the production database, especially since it's about accounting and we would…

If it's an accounting service, make a new account/table named "Software test". Book-keeping was invented to spot errors. If you have a non zero balance in your "Software test" account you probably have a bug in the software.

Re: Superior Testing: Stop Stopping

#29

Post by bunderbunder »

moring wrote: post #24

The introduction made me hope for real advice on how to "stop stopping", but it only repeated the old arguments on why we want tests. Why not deal with some of the real problems that actually prevent people from writing automated tests? Real-world example 1: The system being developed talks to external system X. We don't want tests to litter the production database, especially since it's about accounting and we would…

I don't think a blog post can possibly cover all the cases. A book will get you closer. For a great book on exactly that sort of subject matter, I couldn't agree more with the author of TFA: Working Effectively with Legacy Code is a book that everyone should read at least once.

It's been a while since I personally last read it, but, as I recall, it has an entire chapter devoted to each of your examples.

Re: Superior Testing: Stop Stopping

#30

Post by TarpitCarnivore »

arkh wrote: post #3
chriswarbo wrote: post #2

I heard the "customers don't pay for tests" line recently, from someone boasting about how they migrated hundreds of thousands of lines of PHP from 5.x to 7.x. I didn't have an off-the-cuff reply to this, but thinking about it afterwards the issue became clear to me: customers don't pay for code either! Customers pay for solutions to their problems. I think the key phrase is "how do we know?": maybe those hundreds of…

> customers don't pay for code either! Customers pay for solutions to their problems. And that's why you don't test code. You test that your application comply with your customer's requirements. I'm happy to see things changing about tests with people realizing fine grained unit tests are often an hindrance and you should prioritize end to end testing. Test the interface of what you're selling, not the inner workings…

> you should prioritize end to end testing. Test the interface of what you're selling, not the inner workings.

E2E tests can be slow and extremely fragile though. They have a place and purpose, but they should not be the only form. Why would you skip entirely over "inner workings" tests that could catch bugs sooner?

Post reply on HN