Live data from Hacker News

The World's Worst Penetration Test Report by #ScumbagPenTester

it.toolbox.com

11–20 of 26 posts

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#11
post #4

> Microsoft IIS susceptible to CVE-XXXXXXXX. Recommend applying accordingly patch. > Another almost good finding - but according to the appendix, this host is a RHEL 5.x box. Those sysadmins - finding ways to run IIS on linux!! Brilliant! I used to see crap like this all the time coming out of PCI compliance audits. The hosts I dealt with ran Apache on RHEL, too. We'd point this out, and they'd "accept" that "fix."

Some of these sound reasonable. For example there are un-patched minor Denial-of-Service attacks in both Windows and Linux, and I'd expect a pentest to report on those, even if there's nothing we can do about them (for example monitor, inform ops etc of the potential for said attack).

If OS detection is wrong (and its not a precise science), reporting on these is reasonable.

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#13
post #8
post #4

> Microsoft IIS susceptible to CVE-XXXXXXXX. Recommend applying accordingly patch. > Another almost good finding - but according to the appendix, this host is a RHEL 5.x box. Those sysadmins - finding ways to run IIS on linux!! Brilliant! I used to see crap like this all the time coming out of PCI compliance audits. The hosts I dealt with ran Apache on RHEL, too. We'd point this out, and they'd "accept" that "fix."

I've seen similar to this with our pen-test firm (who apparently do pen-tests for the UK NHS) - we're a C#/ASP.NET shop, running on Windows servers and they know this. For some reason when I examine our logs after each test I see attempts at access to /usr etc. It does make me wonder if all so called pen-test companies are just one massive scam.

Usually when we see stuff like that it's because there's a device doing some traffic control (load balancer, proxy, WAF, whatever) for the web server that runs some version of Linux. So you'll get a report that says IIS on Redhat or IIS and Apache both running on port 80.

Since the automated scanners are... stupid, they'll then gleefully go off and check for Linux vulnerabilities too.

The guys doing the analysis should of course catch all that, but if no one is doing any analysis and just parroting what the tools tell them, it ends up in the report.

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#14
post #8
post #4

> Microsoft IIS susceptible to CVE-XXXXXXXX. Recommend applying accordingly patch. > Another almost good finding - but according to the appendix, this host is a RHEL 5.x box. Those sysadmins - finding ways to run IIS on linux!! Brilliant! I used to see crap like this all the time coming out of PCI compliance audits. The hosts I dealt with ran Apache on RHEL, too. We'd point this out, and they'd "accept" that "fix."

I've seen similar to this with our pen-test firm (who apparently do pen-tests for the UK NHS) - we're a C#/ASP.NET shop, running on Windows servers and they know this. For some reason when I examine our logs after each test I see attempts at access to /usr etc. It does make me wonder if all so called pen-test companies are just one massive scam.

They are probably using an automated testing tool of which there are many. These tools are all pretty dumb and the typically end up running all the tests on all the machines w/o taking into account prior knowledge like, "This is a windows server". These tools end up returning a bunch of useless information that a knowledgeable person then needs to sort through to confirm each potential problem.

The issue arrises when someone takes what pops out of the scanner as gospel instead of investigating if it is true or a false positive.

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#15
post #8
post #4

> Microsoft IIS susceptible to CVE-XXXXXXXX. Recommend applying accordingly patch. > Another almost good finding - but according to the appendix, this host is a RHEL 5.x box. Those sysadmins - finding ways to run IIS on linux!! Brilliant! I used to see crap like this all the time coming out of PCI compliance audits. The hosts I dealt with ran Apache on RHEL, too. We'd point this out, and they'd "accept" that "fix."

I've seen similar to this with our pen-test firm (who apparently do pen-tests for the UK NHS) - we're a C#/ASP.NET shop, running on Windows servers and they know this. For some reason when I examine our logs after each test I see attempts at access to /usr etc. It does make me wonder if all so called pen-test companies are just one massive scam.

Those parts are typically automated. Also even if you're a .net shop, you might have your stuff placed in front a linux/bsd based router which responds to certain requests (it does happen).

A pen tester can't know what infrastructure changes you've made, and should work from scratch each time, IMO. It might not have even been your company which made the change, hoster might have made change to the environment etc etc.

If it doesn't cost you any more to have them to try and scan /usr, ../../../../../../etc/passwd and so on, why not?

When I did pentests, we encountered a customer who, via a misconfigured puppet manifest, installed WordPress on a public facing server with a known-vulnerable plugin, which we found by always scanning /wp-admin/ even though the customer was a strictly RoR shop.

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#16

If this is a baseline for "very bad" penetration testing, it makes me think I could start up an "almost competent" low-end penetration-testing business just by blindly following a checklist found on the internet, as long as I used a checklist for the right operating system and proof-read the final report.

The "we're fast, we're cheap, and sometimes we're right" approach to marketing?

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#17
I'm a developer and we outsource some of our projects (we used to anyway) - I've got to tell ya - I've seen worse. Oh, the horrors ... the mess. I remember once when the "consultants" deployed for the first time on our staging env we somehow ended up with 15 databases some containing credit card info and all kinds of transactions (I guess from previous clients), the reason I know is because I had to examine this madness. As far as I could tell their software was dependent on most of the DBs. By the way the software was a recruitment web app, nothing to do with payments what so ever.

Re: The World's Worst Penetration Test Report by #ScumbagPenTester

#20
Just as a note to startups here considering "penetration tests":

"Penetration test" is a term that means wildly different things depending on who you talk to.

The kind of test discussed in this post is the most common kind. People in the field call them "network penetration tests". These are the projects where someone runs nmap and Nessus and Metasploit against your network, dumps the Nessus results into a Word document, and calls it a day.

I'm not wild about these kinds of projects, and even less wild about the firms that specialize in them. They may find things on your network that you need to know. But they generally involve people just running some tools and interpreting the results, and then, if they find something blatant, spending the balance of their time using that finding to pry their way into the rest of your network.

The latter part of the project --- the part where they get to your database, dump your hashes, pivot from machine to machine, &c --- is not a great use of your security dollars. It's generally always going to be the case that if someone finds a way to run code (or SQL) on one of your servers, you're done for. The important finding is the flaw that gets attackers into your network. The findings that come after that look scary, but since there's not a whole lot you're going to be able to do to reliably lock down your internal servers, they aren't very useful to you; the next team that finds some other way onto your servers will embarrass you just as badly even after you "fix" the internal flaws from the first team.

You can get a license to run Nessus pretty cheaply. You can download nmap and Metasploit yourself. If you can build a product, you're more than qualified to run them yourself. If you don't have the bandwidth to do that, don't pay too much to have someone else do it. Also, demand that the team that does the netpen breaks out the findings that actually get them into your network, versus the less valuable findings like "older version of OpenSSL detected that we don't actually know how to exploit" or "customer records recovered after we took control of your database", and make sure the team concentrates on finding new ways into your network, rather than on extending their access into your network once they do find a way.

You'll need to ride netpen people not to waste time extending access, because the Fortune 500 companies that are the bread-and-butter clients for network penetration testers actually do want people to spend time extending access and finding "shock and awe" internal findings --- they're doing these tests for a different reason (to justify security budget), not for the reason you're doing them (to make sure it isn't easy to break into your servers).

Post reply on HN