Live data from Hacker News

We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

trustcompliance.xyz

11–20 of 83 posts

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#11
> "We may receive compensation from vendors listed below. All recommendations are based on independent research."

this + new HN account? couldn't be more obviously a competitor. not to defend delve, but can’t be pushing this like some noble effort with the goal of transparency

also lol @ the fake realtime "just searched for" toasts on a setInterval in the bottom left.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#13
post #10

Is SOC 2 legit? I have this on my roadmap but now I’m wondering if it’s just security theatre?

All of the audit / certifications are theatre. the only question is if your customer is required to participate in the show.

If you really care about security, you need to separate it from this stuff, it can only hurt you.

Do your own, real, security, and treat this compliance stuff as an opaque customer feature request.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#15
post #10

Is SOC 2 legit? I have this on my roadmap but now I’m wondering if it’s just security theatre?

That's a difficult question to answer. It shouldn't be, but it is. The reality is, SOC2 is a sales-enablement tool. You should:

* Run a SOC2/compliance program that is entirely disjoint from your security practice.

* Defer SOC2 until the work required to sell into customers demanding it (phone calls, questionnaires) exceeds the cost of obtaining SOC2.

* Prepare for SOC2 by making simple best-practices engineering decisions, in particular single-signon for virtually everything and protected branches for all your repositories.

* Do not allow SOC2 to force any engineering decisions that you would not have intuitively made yourself (this is a big risk with the evidence-gathering platforms like Drata, Delve, and Vanta).

* Assume your SOC2 Type I report will suffice as a first attestation (ie: buy you 1 year of time) with all your customers, and understand that you cannot fail to obtain a Type I; your Type I is guaranteed.

Over 5-6 years of discussing SOC2 with other security practitioners pretty intensively, the overwhelming weight of the evidence is that ~practically nobody actually reads SOC2 reports; they just check the box for each vendor and move on. Plan accordingly.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#16
post #10

Is SOC 2 legit? I have this on my roadmap but now I’m wondering if it’s just security theatre?

It’s fine for what it is: some light guardrails that attempt to nudge you towards answering “is this all just a house of cards that will obviously collapse under a light breeze”.

Getting a SOC2 doesn’t mean you’re amazing or secure or stable. If a customer says they’ll write you a fat check but they need you to have a SOC2, tell them you’ll get it within a year if they start paying. Otherwise don’t bother.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#18
post #10

Is SOC 2 legit? I have this on my roadmap but now I’m wondering if it’s just security theatre?

It's complicated. In theory, SOC2 forces you to do some important stuff, like define your threat model and say "I can mitigate against the threats and prove that my mitigations are in place". The problem is always that the companies that care don't need it but are burdened with it while the companies that don't care will just checkbox their way through it. It sort of enforces a very baseline security posture, in theory, but the major win of "We've thought our security through" is more of a choice - SOC2 can't actually force you to care.

A ton of these SOC2 vendors take all of the potential good parts of SOC2 out of the equation, building the threat models for you and then you just hook up your gsuite/ github and they check boxes for you or tell you to flip a policy here or there. Delve took this to the extreme by not even asking you to flip the checkboxes.

That said, it doesn't matter if it's legit. Everyone is SOC2, and part of being SOC2 is that the vendors whose products you purchase are SOC2, so it's not a choice - you have to be SOC2 if you want to sell (industry/ product specific, but at some point it'll be clear if it applies). If your goal is security, well, SOC2 is irrelevant.

Ultimately, you'll end up having a separate compliance team to manage SOC2 and you'll actively try to keep "real security" from it because real security has to change over time. You'll encode the absolute minimum possible into your compliance for that reason so that you can easily pass every year and then, if you care about security, you'll invest in that separately.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#19
post #4
post #2

We analyzed the leaked Delve audit reports and found some wild patterns: - The same auditor license number (PAC-FIRM-LIC-47383) appears in 487 out of 494 reports - Every Type II report has identical page numbers: Section 4 at page 30, tests at page 59, Section 5 at page 82 - 220+ "No exceptions noted" per report, across every single client - The system descriptions were copy-pasted from each company's marketing websi…

Genuinely curious: if you just need an independent audit report to check a box, do you really care how good a job the auditor did?

"You" probably don't, but it's not just "you". There's also the counterparty who's asking to see that report. Maybe they're doing it for paper-pushing purposes of their own, but ultimately, somewhere up the chain, there's someone thinking "I can't personally audit all my suppliers, and I can't be sure they're doing the right thing, so I'm going to ask them to get an independent audit".

Of course, this shows that the entire system is a bit of a charade, but the point is that someone cares and they're gonna be annoyed when they find out that the audit appears to be a sham.

Whether they have a good alternative is a separate question. But here's another way to look at it: if we show blatant disregard for self-regulation, the government is eventually going to show up and come up with more onerous rules.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#20
post #4
post #2

We analyzed the leaked Delve audit reports and found some wild patterns: - The same auditor license number (PAC-FIRM-LIC-47383) appears in 487 out of 494 reports - Every Type II report has identical page numbers: Section 4 at page 30, tests at page 59, Section 5 at page 82 - 220+ "No exceptions noted" per report, across every single client - The system descriptions were copy-pasted from each company's marketing websi…

Genuinely curious: if you just need an independent audit report to check a box, do you really care how good a job the auditor did?

Probably not, in fact your auditors not being terribly thorough might be a selling point. But your clients, who are the ones asking for the box to be checked, might.

In my experience, clients don't dig deeply into the report or the auditor, they just want to see that you 1) have the report 2) it doesn’t have any egregious exceptions. Perhaps if this makes big enough news, that’ll change.

Post reply on HN