Security Certifications Are Causing More Harm Than Good • Hacker News Board

Re: Security Certifications Are Causing More Harm Than Good

#191

Post by spydum » Thu, Apr 13, 2017, 2:35 AM UTC

ianai wrote: post #60

w8rbt wrote: post #6

There's 'compliance security' and then there's 'street-smart security'. They are very different things. Most organizations aim for compliance (it's cheap and easy). They base security on contracts, certs and insurance policies. Street-smart security practitioners are appalled by this. And, management doesn't understand why the 'security people' aren't on-board with 'compliance'. It's a lot like the old west with Cowb…

Pretty sure you hit it on the head. Over time, security breeches should alleviate this gap.

IMHO, I dont think it will pan out this way. Companies are being breached, and neither their marketshare nor profits are being significantly affected. Aside from https://www.dailydot.com/layer8/code-spaces-hacked/ -- how many other companies do you know which really paid a significant price for being hacked?

Yes, Anthem/BCBS, Target, HD, Sony, etc, etc have all had losses.. but they really havent been long-term impacted it seems.

I dont know what the answer is, this sucks hard as both a consumer and an infosec person. I tend to view security as a "hidden performance" factor. As long as the security flaws don't inconvenience the paying customers too much, they simply don't care if they exist or not.

Re: Security Certifications Are Causing More Harm Than Good

#192

Post by spydum » Thu, Apr 13, 2017, 2:39 AM UTC

ryan-c wrote: post #70

My pet peeve with CISSP (and all other ISC^2 certifications) is this: https://www.isc2.org/candidate-background.aspx It appears to say that if you ever hung out on IRC and tried to keep your handle private, you're ineligible.

That does sound pretty unrealistic, but doesn't this consider that scenario:

"Omit user identities or screen names with which you were publicly identified."

Re: Security Certifications Are Causing More Harm Than Good

#193

Post by spydum » Thu, Apr 13, 2017, 2:45 AM UTC

unethical_ban wrote: post #95

I find value in the SANS courses I've taken. Not hat the certs mean too much, since the tests are pretty easy... but the concepts, common jargon and best-practices discussions that take place can be useful. It's kind of like college. It's about what you get out of it. From a hiring perspective, though - if a course provides utility, perhaps if one of the interviewers also has taken the cert course, they can probe the…

Nobody is knocking security training -- just the value of the certificate process.

We all know far too many people who have mastered the test, but not the material. This makes the signal from the certification unreliable.

Re: Security Certifications Are Causing More Harm Than Good

#194

Post by tptacek » Thu, Apr 13, 2017, 2:54 AM UTC

amckenna wrote: post #173

tptacek wrote: post #129

Earlier quoted context omitted.

Describe the overflow exploit you wrote. What was the vulnerability, and what did the exploit look like?

Unfortunately I can't get into too much detail because I had to sign an NDA (to prevent cheating). But the process was similar to when I have found them in the wild: identify the app, install it locally, fuzz various parameters (it was a real application, albeit an old one), find the crash, figure out stack space, figure out bad characters, find the right JMP ESP or equivalent instructions in a loaded library, write…

In what way did the exploit they had you write differ from the kind we wrote in 1997?

Re: Security Certifications Are Causing More Harm Than Good

#195

Post by tptacek » Thu, Apr 13, 2017, 2:56 AM UTC

intern4tional wrote: post #167

tptacek wrote: post #153

Earlier quoted context omitted.

Roles I've held: * ISP network security engineering * Network penetration tester * Software developer for network security products * Application security assessor * (Most recently) Security team lead I've had these roles for small companies and for very large ones. What experience am I missing that would lead me to change my mind about the CISSP? I don't think attempting to pigeonhole me as a "crypto expert" is goin…

That's an impressive resume of roles, but security is more than just those areas. I think the grandparent is trying to say that the CISSP is largely for non-technical security roles. People that manage large security organizations are generally believed to be the ones that benefit from the CISSP as they are not interested in the details and more on a 1000 foot strategic view. Without knowing more details about the yo…

In 10+ years consulting for Fortune 100 companies, zero is the number I have seen with 400+ security staff. A 50 person security team is enormous even by the standards of financial services.

Re: Security Certifications Are Causing More Harm Than Good

#196

Post by tptacek » Thu, Apr 13, 2017, 2:59 AM UTC

616c wrote: post #172

tptacek wrote: post #119

Earlier quoted context omitted.

It's unlikely that typical OSCP-holder could write a modern buffer overflow exploit, or even judge exploitability of a memory corruption flaw given the source code and a traceback. Equally importantly: memory corruption exploit development and SQL injection are different skills , and most people who do SQL injection don't need proficiency in "buffer overflows". Why is superficial coverage of "buffer overflows" part o…

This is why when I was in an infosec bootcamp I begged you to talk to my class and give a dose of reality. The leet kids, a minority few and the rest naïve, I would bribe while they whittled away at online CTFs and MicroCorruption and irritate them with mediocre questions until they tuned me out. I did not care for tools; approach and mindset are order of magnitudes harder to explain. I thought you could be a wakeup…

I'm not at NCC!

Re: Security Certifications Are Causing More Harm Than Good

#197

Post by austincheney » Thu, Apr 13, 2017, 3:19 AM UTC

I am wondering how many of these wise commenters have actually taken the CISSP exam, or even know what it is? It costs $600 and only has a 60% pass rate even after it requires certain validations in order to achieve a test appointment.

In discussions like these I really wish people would post up front if they had taken the test (regardless of pass or fail). This lets me know which comments I can ignore as completely ignorant.

Re: Security Certifications Are Causing More Harm Than Good

#198

Post by snowwrestler » Thu, Apr 13, 2017, 3:48 AM UTC

flukus wrote: post #163

raesene6 wrote: post #64

Earlier quoted context omitted.

Certifications are used by many industries to provide a demonstration of a common baseline level of knowledge and experience, so that each individual person doesn't need to be assessed by each hiring organisation. For example Certified accountants, Lawyers etc. Without some common baseline, how do people looking to hire security types who don't have the experience to assess their skills and knowledge avoid getting ba…

How many of those industries does certification actually work in? Does it prevent dodgy lawyers and accountants?

Certification does work in some industries.

When a lawyer is admitted to the bar, that is generally taken as proof that they have some idea of what they are doing. Bar exams are hard.

And the bar also provides a forum for dealing with shady lawyers. If a lawyer treats you badly, you can file a complaint with their bar association and they might get disbarred. This is an area where cyber training and certs is not as good yet, I think: as a forum for resolving disputes.

Re: Security Certifications Are Causing More Harm Than Good

#199

Post by spc476 » Thu, Apr 13, 2017, 4:14 AM UTC

cestith wrote: post #152

jlg23 wrote: post #146

Earlier quoted context omitted.

> The thing with infosec is that no matter if you're a consultant pen tester or an in-house member of a blue team, a high proficiency in technical writing is required. As much as I admire people who show the courtesy to the trash bin that will eat their report, before any human reading it, of not feeding it "bad" reports: The OPs point of "sense of false security" unfortunately already sets in once someone is hired t…

One of the most important aspects of a long report is the prioritization of the contents.

About a decade ago (give or take a year) our small web company had a PCI test against our network and a 500-page PDF ended up in my email. The "report" was obviously the vomit of a program detailing every issue for every site [1]. You would think that after the 50th time our DNS resolvers were "open to the public" (as it were) that the stupid program vomiting the report would realize it's reported the same DNS resolvers each time.

[1] It didn't help my perception of the report when it screamed that "ICMP echo was enabled and nefarious scalawags might be able to do unspeakable acts against our computers, best cut the network cable" type of advice [2] (yes, I know port 443 is open! WE'RE A WEB HOSTING COMPANY SERVING UP COMMERCIAL WEBSITES! ARE YOU PCI AUDITORS STUPID?)

[2] Okay, the ICMP echo thing was reported, but did not need to be disabled to pass the audit. If so, why even bring it up?

Re: Security Certifications Are Causing More Harm Than Good

#200

Post by andrewflnr » Thu, Apr 13, 2017, 5:27 AM UTC

cestith wrote: post #154

lawnchair_larry wrote: post #151

Earlier quoted context omitted.

The burden of proof should be on whoever is suggesting that security has anything at all to do with those professions, but I'll throw out a couple of observations anyway. Those professions have rules, and are backed by either legislation or science. All participants are bound by said rules. For a lawyer, certain things are legal, certain things are not. Security is a game where the whole objective is to either break…

It's difficult to believe that anyone who can claim to really know computers thinks they aren't based on a series of interacting rules. That's basically all they are. Understanding how those rules interact, how to trigger certain interactions others didn't intend, and the best practices to not get bit by those interactions is what security is all about. It's much like law or medicine in that you are looking at unexpe…

Computers follows rules the same way a football player follows physics. Those are not the rules GP is talking about.