Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

trustcompliance.xyz

Post reply on HN
Print view View original

51–60 of 83 posts

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#51

Post by hobofan »

baxtr wrote: post #47
preinheimer wrote: post #23

Looking at our SOC 2 report (we don't use Delve, our auditor isn't on their list) I don't think this is quite the smoking gun it might look like if you're not reading SOC 2 reports for a living. There's a fair amount of boiler plate language in these reports, and a bunch of re-stating the SOC 2 controls. I'd expect two reports (same auditors, same platforms) to be nearly identical. If they're both using AWS, Github,…

But maybe you shouldn’t raise so much money and make a big fuss about it when all you’re selling is a template?

Why not?

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#52

Post by jiveturkey »

fadijob wrote: post #2

We analyzed the leaked Delve audit reports and found some wild patterns: - The same auditor license number (PAC-FIRM-LIC-47383) appears in 487 out of 494 reports - Every Type II report has identical page numbers: Section 4 at page 30, tests at page 59, Section 5 at page 82 - 220+ "No exceptions noted" per report, across every single client - The system descriptions were copy-pasted from each company's marketing websi…

I'd find this more compelling if you looked at a few thousand Vanta or Drata reports grouped by auditor. You're going to find the same commonalities with only trivial language differences.

SOC2 reports are private between you and the auditor (that way if you "fail" you can just find another auditor or have a re-do, and no one is the wiser), and basically always gated behind a sales touchpoint (another hint about what utility they provide). I guess the Delve ones leaked which is why they can all be compared.

220 out of 494 "no exceptions" seems quite high to me. Nobody I've ever dealt with allows an exception to make its way into the report.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#53

Post by chromacity »

throw310822 wrote: post #27
chromacity wrote: post #19

Earlier quoted context omitted.

"You" probably don't, but it's not just "you". There's also the counterparty who's asking to see that report. Maybe they're doing it for paper-pushing purposes of their own, but ultimately, somewhere up the chain, there's someone thinking "I can't personally audit all my suppliers, and I can't be sure they're doing the right thing, so I'm going to ask them to get an independent audit". Of course, this shows that the…

> but the point is that someone cares Is it true, though? Or has everyone just been psyched into asking for that certification out of a vague fear of "consequences" or of being left behind?

It's not either-or. Companies care about security because of the consequences. If you're a big company contracting a small one, you don't want to get owned through that vendor because you know you'll be the one holding the bag (data loss, reputational damage, regulatory scrutiny, lawsuits).

Small vendors will tell you what you want to hear because they're desperate for your business. Independent auditing is, in theory, a way to get closer to the ground truth. Well, in theory.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#55

Post by colechristensen »

tptacek wrote: post #7

The damage this will do to the reputation of the SOC2 Security Attestation is incalculable.

Does SOC2 in general have a particularly high reputation?

The only security compliance frameworks that have any particular reputation with me are the ones associated with the department of defense where the consequences range between a slap on the wrist warning or a small 5 figure fine to execution for espionage (which only ever happened for Julius and Ethel Rosenberg, though one could imagine there may have been more, uh, unofficial consequences that nobody ever heard about). In other words, people actually care about the enforcement of security standards in meaningful ways and there are meaningful consequences.

Everything else... well they're all at least a little better than a participation trophy and the process proving you're trying isn't meaningless. It's just not been my experience with these things that they're particularly good guarantees that the spirit embodying the compliance program is actually being done particularly well.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#56

Post by sgbeal »

jdns wrote: post #11

> "We may receive compensation from vendors listed below. All recommendations are based on independent research." this + new HN account? couldn't be more obviously a competitor. not to defend delve, but can’t be pushing this like some noble effort with the goal of transparency also lol @ the fake realtime "just searched for" toasts on a setInterval in the bottom left.

> lol @ the fake realtime "just searched for" toasts on a setInterval in the bottom left.

There is intermittent XHR traffic, but it's most definitely not returning any such information and it's posting what looks (in FF's dev tools) like garbage binary data.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#58

Post by jgalt212 »

mikert89 wrote: post #5

Just know that alot of startups with all star founders are closer to delve than not. Its mostly marketing, "look at this MIT genius that noticed something about legacy xyz industry that no one else did" Truth is venture funds are allocating a limited pie of what is really societies capital to people that dont deserve it

> Truth is venture funds are allocating a limited pie

This pie does not seem that limited recently.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#59

Post by netsharc »

ecshafer wrote: post #30
preinheimer wrote: post #23

Looking at our SOC 2 report (we don't use Delve, our auditor isn't on their list) I don't think this is quite the smoking gun it might look like if you're not reading SOC 2 reports for a living. There's a fair amount of boiler plate language in these reports, and a bunch of re-stating the SOC 2 controls. I'd expect two reports (same auditors, same platforms) to be nearly identical. If they're both using AWS, Github,…

This mirrors my thoughts. A page of boiler play text with some check boxes, with some checked vs unchecked is going to be 99.8% similar between companies as well. A lot of audits are very much forms with boiler plate and fill in the blank. There is no point rewriting everything from scratch.

And those headlines read like Gemini's "punchy" writing.

Re: We indexed the Delve audit leak: 533 reports, 455 companies, 99.8% identical

#60

Post by preinheimer »

baxtr wrote: post #47
preinheimer wrote: post #23

Looking at our SOC 2 report (we don't use Delve, our auditor isn't on their list) I don't think this is quite the smoking gun it might look like if you're not reading SOC 2 reports for a living. There's a fair amount of boiler plate language in these reports, and a bunch of re-stating the SOC 2 controls. I'd expect two reports (same auditors, same platforms) to be nearly identical. If they're both using AWS, Github,…

But maybe you shouldn’t raise so much money and make a big fuss about it when all you’re selling is a template?

I mean it’s a template, but in theory someone went and checked stuff. Did you actually have a quarterly security team meeting? Was there minutes? Was there an invite?

Did someone actually go and confirm your role based access control matrix is up to date and user accounts have the right access? Were all of those screenshots watermarked with timestamps?

There is work to do, whether or not auditors are doing it is another question.

Post reply on HN