Shopify has paid over $300k in security exploit bounties • Hacker News Board

Re: Shopify has paid over $300k in security exploit bounties

#41

Post by user5994461 » Sat, Dec 17, 2016, 5:19 PM UTC

raesene6 wrote: post #39

user5994461 wrote: post #36

Earlier quoted context omitted.

Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some ove…

Ahh indeed, that was kind of my initial question for the top-commentor. I'm interested in whether Shopify explored getting security consultants in to review this area before going for a bug bounty on it, or went straight to the bug bounty. My personal feeling is that the order of play should be Internal Security Review --> External Security Review --> Bug Bounty as you can use the first two stages to catch all the ba…

I think companies should get standard compliance stuff done first (they're in payments so they certainly have lots of these to have) + a set of standard vulnerability scans (Nesus / FireEye). These things are "cheap" and easy to get, it's standard package 1-week-audit-for-XYZ.

Then get custom pen testing and bug bounty programs later. They're a lot of work to get done and get right. pen testing is a lot of investment and preparation upfront[1], bug bounty is on a longer term.

[1] Don't bring people at $2k a day if you didn't think through what they're gonna do.

Re: Shopify has paid over $300k in security exploit bounties

#42

Post by tptacek » Sat, Dec 17, 2016, 5:22 PM UTC

raesene6 wrote: post #35

user5994461 wrote: post #31

Earlier quoted context omitted.

> you could've engaged a reasonable team for several man-months Doubt that. 1) The daily rate for this kind of work is high. 2) It's hard to find people who can execute. (read: security as in pen testing, not security as in filling a PCI compliance checklist). 3) Multiply 1 and 2 by the number of people you want in the team.

Sorry, but got to disagree with you there. I'm a security tester and have been in the industry for 15+ years either as a buyer or provider of services, for small and large companies, I've been involved in procurement of multi hundred $k tests and involved in the delivery similar sized engagements. You absolutely can get good security testing consultants for $2k/day for example, and probably less depending on the regi…

$2k/day is a reasonable rate, just a shade towards the high side for vanilla web security work.

Re: Shopify has paid over $300k in security exploit bounties

#43

Post by raesene6 » Sat, Dec 17, 2016, 5:50 PM UTC

user5994461 wrote: post #32

andersonmvd wrote: post #27

Earlier quoted context omitted.

It's very delicate to talk about financial burden, given the following references: "Two months after damaging data breach, Target stock has its best day in 5 years" http://blogs.marketwatch.com/behindthestorefront/2014/02/26/... "Sad reality: It's cheaper to get hacked than build strong IT defenses" http://www.theregister.co.uk/2016/09/23/if_your_company_has_... "The Cost of Cyberattacks Is Less than You Might Think"…

So, the best way to monetize a breach is to play with the company's stock while you disclose the breach. Interesting.

Yeah that's already a thing http://www.zdnet.com/article/cybercriminals-turn-talents-to-...

Also monetizing security vulnerabilities by place bets on stock when disclosing has already happened http://www.careersinfosecurity.com/st-jude-medical-files-law... . It does carry the risk of a lawsuit however...

Re: Shopify has paid over $300k in security exploit bounties

#44

Post by raesene6 » Sat, Dec 17, 2016, 5:54 PM UTC

EiNSTeiN_ wrote: post #40

raesene6 wrote: post #22

Earlier quoted context omitted.

So I'd be interested to know your thoughts on bug bounties as against "traditional" security reviews. Have these areas of your application been through external reviews before being opened up to bug bounty or did you decide to start there? I was thinking that for the amount you've paid out in bounties you could've engaged a reasonable team for several man-months, so was interested in what led you more down the bug bo…

You seem to assume we set out to pay this amount to begin with. Indeed for this amount we could have went other ways, but hindsight is 20/20. No one expected to get so many valid sumbmissions in such a short time. We set the payout amounts this high as a way to attract talent at the beginning of the program, which worked quite well to bootstrap it.

Ah no I didn't really assume that, it was part of my question, as to whether it was a deliberate strategy to jump straight to bug bounty with an expectation of a potentially large number of reported issues or an unexpected event, where the number of reported bugs and their severity, was more than you'd expected.

So (and I'm guessing from your comment you work for Shopify) I guess I can take from that, that it was the latter.

Re: Shopify has paid over $300k in security exploit bounties

#45

Post by korzun » Sat, Dec 17, 2016, 6:07 PM UTC

jwilk wrote: post #6

"It looks like your JavaScript is disabled. To use Hacker One, enable JavaScript in your browser and refresh this page." Kinda ironic that a site that is supposedly for hackers wants you to expose yourself to zillion browser vulnerabilities before you can see its content.

You have a point when it comes to irony.

Re: Shopify has paid over $300k in security exploit bounties

#46

Post by tptacek » Sat, Dec 17, 2016, 6:30 PM UTC

user5994461 wrote: post #36

raesene6 wrote: post #35

Earlier quoted context omitted.

Sorry, but got to disagree with you there. I'm a security tester and have been in the industry for 15+ years either as a buyer or provider of services, for small and large companies, I've been involved in procurement of multi hundred $k tests and involved in the delivery similar sized engagements. You absolutely can get good security testing consultants for $2k/day for example, and probably less depending on the regi…

Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some ove…

The penetration test that would have reliably generated the same outcome as this bug bounty program would have cost ~1/6th as much as the bounty did.

Re: Shopify has paid over $300k in security exploit bounties

#47

Post by fjdlwlv » Sat, Dec 17, 2016, 6:30 PM UTC

xal wrote: post #18

This wasn't unexpected outside of the extend of the bounties. What you have to realize is how important Security is to Shopify. We are a trust based business to an extreme extend. We host the livelihoods of hundreds of thousands of other businesses. If we are down or compromised all of them can't make money ( as some of you saw during Black Friday, to the tune of $300k+ a minute at times ). One of the best ways for u…

Paying one engineer's worth of salary to bug bounties isn't overspending, it's freelancing

Re: Shopify has paid over $300k in security exploit bounties

#48

Post by tptacek » Sat, Dec 17, 2016, 6:31 PM UTC

raesene6 wrote: post #39

user5994461 wrote: post #36

Earlier quoted context omitted.

Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some ove…

Ahh indeed, that was kind of my initial question for the top-commentor. I'm interested in whether Shopify explored getting security consultants in to review this area before going for a bug bounty on it, or went straight to the bug bounty. My personal feeling is that the order of play should be Internal Security Review --> External Security Review --> Bug Bounty as you can use the first two stages to catch all the ba…

I think what 'xal is trying to say is that this bounty had more to do with security marketing than with accomplishing a particular tactical security goal. Their comment even concludes with a note that most or all the findings were accounted for with a sandboxing design they'd already planned.

Re: Shopify has paid over $300k in security exploit bounties

#49

Post by tptacek » Sat, Dec 17, 2016, 6:33 PM UTC

antoinefink wrote: post #33

I don't know if it's the right place, but does anyone has feedbacks regarding the Hacker One platform? Especially for small SaaS (between 1-2M ARR)?

I strongly urge you to find security management people at existing startups to talk to before starting a bug bounty program at your own startup. There are things about them that are good, but those things can be counterintuitive.

I haven't had to manage one (yet), but because we'll no doubt be doing that for several startups this year I've been talking to friends about what their bounty programs have been like, and I've learned a lot of stuff. Frankly, bounties are something I might push back on for a lot of startups.

Re: Shopify has paid over $300k in security exploit bounties

#50

Post by fjdlwlv » Sat, Dec 17, 2016, 6:34 PM UTC

tptacek wrote: post #46

user5994461 wrote: post #36

Earlier quoted context omitted.

Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some ove…

The penetration test that would have reliably generated the same outcome as this bug bounty program would have cost ~1/6th as much as the bounty did.

How many times do you need to repeat the pen test to catch new bugs in new releases?