Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security Hole in Sendgrid

chunkhost.com

Post reply on HN
Print view View original

41–50 of 97 posts

Re: Security Hole in Sendgrid

#41

Post by obsession »

hadoukenio wrote: post #7

So what's the answer? Here's two very legitimate scenarios: 1) You sign up, enable two-factor auth, then lock yourself out (lost password and your second-factor). How do you prove to the service provider that you are you? 2) You sign up, enable two-factor auth, then Mallory claims that they locked themselves out. How does the service provider prove that Mallory is not you?

I think these services could use timed release. If you have locked yourself out, the timer starts running. If after 30 days no one has denied the request, access is granted. I'm pretty sure this approach would foil most if not all social engineering.

Re: Security Hole in Sendgrid

#42

Post by danielweber »

gtaylor wrote: post #24
je42 wrote: post #21

Earlier quoted context omitted.

Sending email is hard and for most people not a thing that provide them with a proper ROI, because among other things... - You need to make sure that your email servers IPs are not on black lists. - That you use DKIM properly - your multipart mime encoding is correct - bouncing e-mails are handled... - take care of scaling & operating the servers - and so on.... You can spent ( and waste ) a lot of time on this... es…

> - You need to make sure that your email servers IPs are not on black lists. This alone is a huge, huge chore. Especially if you run a hosted service that allows some user-specified content in outbound email bodies. It's a lot cheaper for us to pay Mandrill to handle all of that for us, provide us excellent metrics and diagnostics, and let us know if one of our users is sending junk mail before it gets out of hand.

Even if you do everything right you can still find yourself on a blacklist. Then you get to pound sand.

Blacklists suck. Not as much as spam sucks, though.

Re: Security Hole in Sendgrid

#43

Post by mantrax3 »

steven2012 wrote: post #38

We use Sendgrid and have hundreds of thousands of customers that might be phished by this social engineering trick. It's absolutely unacceptable that such a crucial piece of infrastructure is vulnerable to such a simple trick. I'm going to bring this up with our team and see if there's another vendor that can more reliably protect our customers.

Your logic is a bit weird.

Sendgrid just experienced this major embarrassment and are currently re-training their staff to avoid it again at all costs.

And you're going to move away from them now?

Re: Security Hole in Sendgrid

#44

Post by derefr »

hadoukenio wrote: post #23
uptown wrote: post #16

Earlier quoted context omitted.

But what happens in those rare cases where that first account gets lost / locked permanently?

Yes, this is part of what I'm trying to get answers to. Do you tell the user on signup to print an in-case-of-emergency-break-glass password which is only ever to be used to get into a locked account and other special circumstances? It may seem over the top but seeing as it's unique across service providers, I think it's a hell of a lot better than the overly abused "what is your mother's maiden name" type questions.…

Presuming you're paying for this service (and thus have a credit card registered to it), how about the "we've made two $0.00 - $0.99 charges on your card; tell us what the cents digits are and we'll refund them and give you a reset link" model? I've only ever seen it used to initially verify a card--but, provided a card has been verified, continued access to it can be used to re-verify a compromised account.

(And if someone has managed to break into both your personal email account and your business's online-banking account, getting your web-host to recognize you will be the least of your problems.)

Re: Security Hole in Sendgrid

#45

Post by danielweber »

sanderjd wrote: post #12
staunch wrote: post #5

Another title for this submission could have been: "Massive Security Hole in ChunkHost. Non-2FA accounts can be owned." Because it turns out anyone with a Sendgrid Support account also effectively had potential access to any account at ChunkHost not using two-factor authentication. Which is also true of thousands of other companies that are relaying their password reset emails through third party SMTP services. SendG…

I don't see how this was specific to ChunkHost, or how anybody else using SendGrid and supporting password resets through email wouldn't be vulnerable to the same attack... Since presumably password reset emails aren't outside SendGrid's target market, it seems reasonable to suggest that this is a problem SendGrid should try to fix.

Generally, an outsider doesn't care that you got owned because someone you relied upon got owned. They consider it part of your job that you make sure all the people you rely upon are reliable.

I'm not saying this is fair.

Re: Security Hole in Sendgrid

#46

Post by erichurkman »

gtaylor wrote: post #28
toomuchtodo wrote: post #22

Earlier quoted context omitted.

If you invoke sending email through Amazon SES, it's free for the first 2K per day, due EC2 blocks being on spam lists. http://aws.amazon.com/ses/pricing/ "You can send 2,000 messages for free each day when you call Amazon SES from an Amazon EC2 instance directly or through AWS Elastic Beanstalk."

We just switched off of SES because of the lack of bounce/rejection diagnostics, and their internal blacklisting policies are really aggressive. If someone's email server goes down for a few hours, they're blacklisted for quite some time, even after it comes back up. After using SES for close to two years, I'd suggest looking elsewhere if you really care about deliverability or stats/metrics. We switched to Mandrill…

Another +1 for Mandrill. We've been quite happy with them as well.

Re: Security Hole in Sendgrid

#47

Post by lstamour »

hadoukenio wrote: post #7

So what's the answer? Here's two very legitimate scenarios: 1) You sign up, enable two-factor auth, then lock yourself out (lost password and your second-factor). How do you prove to the service provider that you are you? 2) You sign up, enable two-factor auth, then Mallory claims that they locked themselves out. How does the service provider prove that Mallory is not you?

Text message or phone verification of details only you should know about the account history, payment methods, etc. As others mentioned, a waiting period during which they try to contact using any means s previously authorised for a response. Compare IP addresses and deny logins from strange countries or origins without further verification, etc. Of course, every measure and countermeasure needs to be justified, since there's an implementation and upkeep cost, but... It's possible to be "more certain," that something is legit or not.

Re: Security Hole in Sendgrid

#48

Post by mantrax3 »

jtchang wrote: post #26

Social engineering will almost always work. I don't really fault Sendgrid for this (though I could see this not working as well if you were using Amazon SES...no support to even talk to!). It sucks that they got caught with their pants down but I bet a good social engineering attempt on ChunkHost might have yielded similar results. The lesson here is to have multiple defenses. 2 factor auth is a great start and it wo…

In a nutshell, social engineering can be countered by proper software & processes engineering.

So if social engineering is possible, blame the software architects.

Maybe in extreme cases changing the email of an account might be needed, but there's no excuse a first level rep was able to do it. Least thing, he/she should've been forced by the system to escalate to someone above her, who has a much lesser chance to screw up.

Re: Security Hole in Sendgrid

#49

Post by jyap »

jtchang wrote: post #26

Social engineering will almost always work. I don't really fault Sendgrid for this (though I could see this not working as well if you were using Amazon SES...no support to even talk to!). It sucks that they got caught with their pants down but I bet a good social engineering attempt on ChunkHost might have yielded similar results. The lesson here is to have multiple defenses. 2 factor auth is a great start and it wo…

You should fault Sendgrid as they specifically have a policy NOT to perform this change of email request (from the article).

Sendgrid can also change their systems so that phone support personnel can NOT perform this change or perform this change with approval from a supervisor.

Sendgrid being in the business they are in should also know that they are susceptible to these types of attacks and what they can lead to (many, many systems which can have password requests sent to email addresses).

Post reply on HN