Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are there not more freelance testers out there?

testninjas.com

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Why are there not more freelance testers out there?

#51

Post by scott_w »

Too wrote: post #44
scott_w wrote: post #18

From my personal experience, I wouldn't use a freelance tester because they won't understand the system well enough to provide value for money. The most valuable testing is that which exposes your lack of understanding of what the system should do. These also usually require the most extensive rewrites, so you definitely want them as early as possible. Unfortunately, they also require the most extensive understanding…

There's a big difference between blackbox testing and whitebox testing. For blackbox testing, having someone who doesn't now the inner workings of the system to find workarounds can many times be an advantage. Of course you should do both kinds for security critical projects.

I'm not arguing for black/white box testing, I'm arguing that for testing to have real value, your tester needs to behave in a similar manner to a real user. That way you find the gap between your own understanding and your users' understanding of the system.

Take a concrete example: a piece of medical equipment prints off reports. If your tester has no medical knowledge, he doesn't have a frame of reference to know if the numbers are correct or not. He's also more inclined to use it exactly how you specify.

A real doctor would bring his own experience of similar equipment and try to use it in ways you didn't think of. In addition, he will be able to gauge whether the reports "look right".

Re: Why are there not more freelance testers out there?

#52

Post by nicolasedgwick »

jot wrote: post #10

There's less demand for freelance testers because having a one is a nice to have rather than a must have. Clients rarely understand the value of testers, they appear to simply be an optional additional cost. Freelance developers have little incentive to recommend them since it inevitably leads a smaller slice of client budget for them. When an organisation gets to the stage at which it understands the value of, and d…

Wouldn't be great if freelance developers saw the benefit of a freelance tester that they worked well with and the two of them marketed themselves as a joint enterprise for even better customer value.

Re: Why are there not more freelance testers out there?

#53

Post by nicolasedgwick »

scott_w wrote: post #18

From my personal experience, I wouldn't use a freelance tester because they won't understand the system well enough to provide value for money. The most valuable testing is that which exposes your lack of understanding of what the system should do. These also usually require the most extensive rewrites, so you definitely want them as early as possible. Unfortunately, they also require the most extensive understanding…

The question then is how do you assess value for money before you use the service. I work as a freelance tester and I often see that customers see the value after working with me and then return for repeat business (as someone else has mentioned - you build a reputation).

As an example, an error in an iPad app was dismissed as a known connection issue by the dev team because there were no server side logs so the request wasn't hitting the server. This was the dev team's first iOS app. My specialist knowledge of mobile testing helped identify quickly that the app was connecting to the servers but that this one request wasn't and provide console logs showing where the communication error was occurring.

This customer were to launch in public the following week and this would have killed their reputation due to the critical nature of the function that was failing. This was my first day testing with them after they had put in a year of development effort and dev testing (they brought me in specifically because they felt they were getting "tunnel vision"). This was one bug masking behind another which I spotted due to 3 years experience in testing mobile apps. I think that provides value for money without being employed permanently, however I appreciate that companies/teams don't see that value until it's their reputation that has been protected by the investment.

Re: Why are there not more freelance testers out there?

#54

Post by nicolasedgwick »

scott_w wrote: post #51
Too wrote: post #44

Earlier quoted context omitted.

There's a big difference between blackbox testing and whitebox testing. For blackbox testing, having someone who doesn't now the inner workings of the system to find workarounds can many times be an advantage. Of course you should do both kinds for security critical projects.

I'm not arguing for black/white box testing, I'm arguing that for testing to have real value, your tester needs to behave in a similar manner to a real user. That way you find the gap between your own understanding and your users' understanding of the system. Take a concrete example: a piece of medical equipment prints off reports. If your tester has no medical knowledge, he doesn't have a frame of reference to know…

Very good example! And there is an absolutely huge correlation between good testing and domain experience irrespective of freelance/permanent. I've seen entire teams of permanent testers miss critical bugs because they're busy stressing the system and trying edge cases but don't actually use it like a user because they don't know their users.

Re: Why are there not more freelance testers out there?

#55

Post by nicolasedgwick »

michaelt wrote: post #26
rosiesherry wrote: post #6

Earlier quoted context omitted.

There's a bunch of those types of services which we are deliberately avoiding the approach of. It's really hard to know the quality of the testers behind it all, in addition to that they can raise bugs and give you an idea of what has been tested, but you doesn't give you an understanding of what hasn't been tested.

Wouldn't it be awfully easy for an outsourced tester to say "yep, tested foo, no bugs found" without having tested it thoroughly? You can't outsource auditing that outsourced work was done properly!

Yup - just exactly as easy as it is for a developer to say "no bugs in my code" and leave the job well before a critical ID used in a database reaches the limit of the "INT" definition that was applied instead of "LONGINT". In both situations you want to build a level of trust (and contracts/NDAs/liability insurances) between freelancer and customer ;)

Re: Why are there not more freelance testers out there?

#56

Post by testerfromleic »

TL;DR - we (testers) need to tackle these problems head on ourselves rather than wait for people to "get it".

It seems to me, a lot of folks here wouldn't be adverse to using a freelance tester in general, but are extremely adverse to using bad freelance testing (and have the scars to prove it!).

I'm a contract tester and recently decided to make the transition to freelance. From my experience (and confirmed by the comments here) the main barrier is one of perception - how can testers / add value to my organisation? And how do I know this tester is as good as they claim?

There are numerous ways someone like me can help an organisation:

Help organisations find the right permanent employees; Mentor said permanent employees by sharing knowledge; provide testing experience to a known or unknown business domain; supplement the clients test lab with your own equipment; evangelise testing throughout the company, demonstrate what good testing can do, what it looks like

I had to stop myself there! And pretty much all of those examples are from recent (within 12 months) history. Luckily for me I've been able build a decent enough reputation that there is potential for repeat business.

Two more key problems: first, the kind of testers that people here need may live in a bubble where everyone knows the value of testing and as such may not realise the rest of us exist - my colleagues and I need to start dispensing some red pills :). Second, the testers that are trying to be better appear to be outnumbered by the ones that aren't (snake oil merchants, testing zombies, call them what you will) so again we need to tackle that head on too.

Nice discussion. Thanks for the illuminating comments folks.

Re: Why are there not more freelance testers out there?

#57

Post by eru »

scott_w wrote: post #32
lifeisstillgood wrote: post #22

Earlier quoted context omitted.

That's the same argument for not hiring freelance developers or project managers. And IMHO it leads to lazy coupling of software systems and services, a system that has to be built and tested by different or changing groups has to have much much better scope and decoupling. In short, if you have small (micro) services, they are easy to build and easier to test.

I disagree. With good testing discipline, you can bring in freelance developers. Developers have the luck of not needing 100% understanding, as this is grown through the writing of code. Testing, on the other hand, is a simulation of how your system responds to user input, and how your users respond to the system's outputs. If your tester can't simulate a user, then you can make your services as micro as you like - i…

If new people picking up the app have a problem simulating users, then how do you assume your users will go about picking up the up to actually use it?

Re: Why are there not more freelance testers out there?

#58

Post by scott_w »

eru wrote: post #57
scott_w wrote: post #32

Earlier quoted context omitted.

I disagree. With good testing discipline, you can bring in freelance developers. Developers have the luck of not needing 100% understanding, as this is grown through the writing of code. Testing, on the other hand, is a simulation of how your system responds to user input, and how your users respond to the system's outputs. If your tester can't simulate a user, then you can make your services as micro as you like - i…

If new people picking up the app have a problem simulating users, then how do you assume your users will go about picking up the up to actually use it?

Read my comment below. It's not about users understanding the program per se, it's about testers having the domain knowledge to perform meaningful tests.

Without that domain knowledge, the testers may find implementation bugs, but they won't expose the issues caused by the developer's lack of understanding. The latter is almost always more valuable to find and more difficult to fix.

Re: Why are there not more freelance testers out there?

#59

Post by eru »

scott_w wrote: post #58
eru wrote: post #57

Earlier quoted context omitted.

If new people picking up the app have a problem simulating users, then how do you assume your users will go about picking up the up to actually use it?

Read my comment below. It's not about users understanding the program per se, it's about testers having the domain knowledge to perform meaningful tests. Without that domain knowledge, the testers may find implementation bugs, but they won't expose the issues caused by the developer's lack of understanding. The latter is almost always more valuable to find and more difficult to fix.

Oh, ok. I guess writing automatic tests is one way to nudge developers into understanding the domain better. And getting a person on the team, even freelance, whose job it is to push for tests, might help.

Going off another tangent, getting not a generic tester, but a domain expert who's also a tester, might help.

Post reply on HN