Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Penetration testing and low-cost freelancing

sophron.github.io

Post reply on HN
Print view View original

61–70 of 76 posts

Re: Penetration testing and low-cost freelancing

#61

Post by bostik »

czbond wrote: post #23
an_opabinia wrote: post #5

People are paying pentesters because their payer’s policy asks them to. Of course there will be a huge market for someone with an alleged certification to run automated tools, the value is in the box checking and report generation, not the bespoke broken website fixing.

Obviously this is a bit of a reaching statement, some organizations understand the legal, career, and regulatory risk and proactively do it. In every org I've sat down with the head to explain those risks, I can see how it can be made priority and budget.

Let me demoralise you. There are industries [cough] where (for a good number of jurisdictions) you MUST have the pentests and other audit-like engagements conducted by a specially approved operator. Think of a royal charter.

These operators have no incentive to invest in proper skills. So they don't. They can charge almost anything they like. And sure enough, they do.

In an industry with such massive vendor lock-ins and regulatory capture regimes, organisationally you get to shovel a lot of money for sub-par pentests. The appetite to have yet another one, even from a known-good and reasonably priced provider, is quite hard to come by. After all, these approved providers will REFUSE to even look at a third-party pentest report, let alone let one guide their own testing.

So in aggregate the existence of these approved providers and their level of competence degrades the security posture of entire business domains.

Macchiavelli would have been proud.

Re: Penetration testing and low-cost freelancing

#62

Post by blunte »

robertlagrant wrote: post #47
blunte wrote: post #35

This is a common problem when hiring supposed experts. Unless the one doing the hiring is reasonably skilled in the area being hired for, it is very difficult to judge the skill and quality of the candidate.

This statement ignores the prices. Just look at the prices and you'll see there's no way anyone thinks they're paying for an expert.

There are unfortunately many small business people who greatly underestimate the skill that goes into anything they don't personally do. Those are the foolish people who shop by price and think they are getting a good deal.

These are also the people who keep Walmart alive. They have limited reasoning ability, and even if they might suspect it's too cheap to be any good, they want it to be good enough and are willing to pretend that it is. These happen to also be the people who easily accept absurd and obviously false statements made by certain political leaders.

I know quite a few of these people, and I have tried countless times to advise them on reasonable choices, but it is like talking to a wall (if a wall could nod and pretend to agree).

Re: Penetration testing and low-cost freelancing

#63

Post by kthejoker2 »

As a thought experiment, I think the best course of action for these freelance pen testers - assuming they incur no actual liabilities for being wrong - is to simply declare everything they test "secure" without putting any actual effort into it at all.

If they can manage to do this at any scale whatsoever, there is basically no downside, because there's a huge disconnect between the impact of a false negative (site breached, data stolen, etc.) and the impact to them (a 1-star review - even in your nightmare scenario as a customer, you still give them a score of 20%!)

1) A large number of the people who come to them will have actually produced secure code, so broken clock theory prevails, and there's no downside.

2) Another large subset will never generate enough interest among hackers to exploit their insecure code; a tree falls in the forest and no one hears it.

3) Another subset still will never know they've been breached and have cause to action.

4) If a company does learn they've been breached, they still must connect the dots to the failure of the pen tester.

5) And even then ... 1 star and some bad PR!

And of course if there were actual liabilities, they wouldn't be charging $35 on a freelance site.

Re: Penetration testing and low-cost freelancing

#64

Post by SCHiM »

wayf wrote: post #17

I'm trying to imagine being a customer for this kind of service. When would I be satisfied with a pen testing service? If I'm not familiar with pentesting methodologies, my only metric of satisfaction would be the pen tester's reputation i.e., if a well-reviewed pentester says my site is OK, then maybe my site is safe.

I work as a professional pentester. This question is difficult for both sides.

The business often gets in my way with budgets. I can spend it all on some strange behavior that I detect in the application. I like to spend my time like that. The more experience I get the more time I could spend, even on simple applications. The rabbit hole goes ever deeper.

When am I satisfied that I delivered good work? When I find a number of high impact vulnerabilities. Doesn't mean there aren't still more to be found, like I said you could go on and on, but it does mean that I fixed some bad. Sadly I have a fixed budget, and sometimes I find nothing, those are bad days.

In my experience customers are also satisfied with a report that contains few or no impactful findings. But an empty report is no guarantee that there really isn't anything to be found.

It's difficult to balance the desire to dive deep and to provide broad coverage. I once got bit by that.

I found and clearly documented no less than 3 absolutely critical issues (price adjustment in an e-commerce website). Felt pretty good about that test. Until, in production, someone else found an additional authentication bypass. Oops. I wasn't focused on that due the aforementioned input validation issues and got blinded by trying to find more of those. It's not easy.

Re: Penetration testing and low-cost freelancing

#65

Post by IncludeSecurity »

This sort of thing happens even for high-end pentesting. Here is the same assessment done by four decent consulting companies. They all found risks that the other companies missed.

https://ostif.org/four-audits-of-randomx-for-monero-and-arwe...

This is what I'm giving to clients who have unreasonable expectation that all vulns should be found during an assessment. The usually "time boxed" nature of this sort of work does allows for 1.5 sigma when many companies always expect 3 sigma coverage.

Re: Penetration testing and low-cost freelancing

#66

Post by wrkronmiller »

kthejoker2 wrote: post #63

As a thought experiment, I think the best course of action for these freelance pen testers - assuming they incur no actual liabilities for being wrong - is to simply declare everything they test "secure" without putting any actual effort into it at all. If they can manage to do this at any scale whatsoever, there is basically no downside, because there's a huge disconnect between the impact of a false negative (site…

Counterpoint: if they find _nothing_ wrong, then whoever hired them has potentially failed to justify their job. Better to find minor, easily fixed, bugs I'd think.

Re: Penetration testing and low-cost freelancing

#67

Post by czbond »

bostik wrote: post #61
czbond wrote: post #23

Earlier quoted context omitted.

Obviously this is a bit of a reaching statement, some organizations understand the legal, career, and regulatory risk and proactively do it. In every org I've sat down with the head to explain those risks, I can see how it can be made priority and budget.

Let me demoralise you. There are industries [ cough ] where (for a good number of jurisdictions) you MUST have the pentests and other audit-like engagements conducted by a specially approved operator. Think of a royal charter. These operators have no incentive to invest in proper skills. So they don't. They can charge almost anything they like. And sure enough, they do. In an industry with such massive vendor lock-in…

Absolutely agree.

Re: Penetration testing and low-cost freelancing

#68

Post by czbond »

edw wrote: post #50
czbond wrote: post #23

Earlier quoted context omitted.

Obviously this is a bit of a reaching statement, some organizations understand the legal, career, and regulatory risk and proactively do it. In every org I've sat down with the head to explain those risks, I can see how it can be made priority and budget.

Can we collectively refrain from making comments like this on Hacker News—or in any venue, for that matter? Is it that hard to read an assumed “In my experience, it is often the case that…” in front of everyone’s comment?

I was stating that I have great success showing how said items should be prioritized and budgeted, across numerous organization types, sizes, and industries - and making it happen.

That is my experience, it may not be others.

Re: Penetration testing and low-cost freelancing

#69

Post by jonathanzufi »

airza wrote: post #24

The problem that strikes me here (as a professional pentester) is that these vulnerabilities are so pathological. There is no realistic series of errors that would lead you to really design an app with hardcoded sqli looking credentials or somehow return a cookie with a malformed header, let alone one that somehow automatically authenticated the user. I am extremely onboard with the idea that pentesting as a whole ha…

Given this is your area of expertise, I'm genuinely interested in how/why you believe that these vulnerabilities wouldn't show up in the real world. Generally speaking, isn't the software developer community littered with developers who aren't adequately skilled/copy-paste-from-Stack-Overflow, etc?

Re: Penetration testing and low-cost freelancing

#70

Post by tru3_power »

bostik wrote: post #61
czbond wrote: post #23

Earlier quoted context omitted.

Obviously this is a bit of a reaching statement, some organizations understand the legal, career, and regulatory risk and proactively do it. In every org I've sat down with the head to explain those risks, I can see how it can be made priority and budget.

Let me demoralise you. There are industries [ cough ] where (for a good number of jurisdictions) you MUST have the pentests and other audit-like engagements conducted by a specially approved operator. Think of a royal charter. These operators have no incentive to invest in proper skills. So they don't. They can charge almost anything they like. And sure enough, they do. In an industry with such massive vendor lock-in…

This annoys the hell out of me. The whole security theater /checkbox checking stuff is depressing.
Post reply on HN