Live data from Hacker News

Linode Security Advisory

blog.linode.com

51–60 of 119 posts

Re: Linode Security Advisory

#51
post #47

I've been using Linode for years and I haven't had too many problems with the service itself. That being said, this makes me think twice about staying with them. If I wanted to switch, is the only real competitor Digital Ocean or are there any other good choices?

http://oktawave.com and http://vultr.com are said to be good. I haven't used them though, I personally use baremetal servers at Hetzner.

Re: Linode Security Advisory

#52

Hey There, I'm a PagerDuty employee and am the same individual who made this post on the last HN thread: * https://news.ycombinator.com/item?id=10845985 Unfortunately, there are some facts in Linode's post that are not correct. >On July 9 a customer notified us of unauthorized access into their Linode account. The customer learned that an intruder had obtained access to their account after receiving an email notifica…

As a former Linode employee myself, I can verify that I know who this is, and that he knows what he's talking about on both sides of the fence. He's worth listening to.

Re: Linode Security Advisory

#53
post #37

Earlier quoted context omitted.

Pretending to do that is ok? I can't believe you're still employed there. Also, I upvoted the comment because I think it should stay visible. I hope other users do that too instead of just downvoting.

He's not. And I can describe his penis, so. There was also stabbing employees (to the point of requiring an ambulance) while fooling around with a knife, setting the building on fire more than once, and tormenting other employees who he didn't like. All of that was tolerated and dismissed by management, specifically Chris Aker and Tom Asaro, which should tell you what you need to know about ever working there. One of…

8==D

Re: Linode Security Advisory

#54
post #40

Earlier quoted context omitted.

> SHA-2 for password hashes They're moving on from them, but they're going to leave SHA-2s sitting there and wait until everyone logs in to upgrade to bcrypt hashes at rest. Not getting a super competent vibe off of these folks.

Waiting until login until you upgrade to bcrypt is a requirement is compotent password storage. At this point in time all Linode should know is SHA-2(password) and they can't use that to derive bcrypt(password). The way upgrade should work is that the user provides their password, which is verified with SHA-2 and then hashed with bcrypt and stored again. In order to do this without people logging in Linode would have…

  > At this point in time all Linode should know is 
  > SHA-2(password) and they can't use that to derive
  > bcrypt(password).
  > ...
  > In order to do this without people logging in Linode would have
  > to bcrypt hash the SHA-2 hashed passwords and then keep doing
  > that for all password validations.
They just could make the intermediate step bcrypt(SHA-2(password)) via some lockstep code/db backend update. Then on next user login after verifying against bcrypt(SHA-2(password)), update the db to the more straightforward bcrypt(password). At the very least this would increase the difficulty of brute forcing in the meantime.

Re: Linode Security Advisory

#55
post #28

Hey There, I'm a PagerDuty employee and am the same individual who made this post on the last HN thread: * https://news.ycombinator.com/item?id=10845985 Unfortunately, there are some facts in Linode's post that are not correct. >On July 9 a customer notified us of unauthorized access into their Linode account. The customer learned that an intruder had obtained access to their account after receiving an email notifica…

Wow .

No kidding. After reading all this I think people would be nuts to continue using Linode for anything critical (hell, maybe even for anything at all) at this point.

How many times has this kind of thing happened to them now?

Re: Linode Security Advisory

#57

Earlier quoted context omitted.

To add to that, bcrypt is not the best recommendation if choosing a password hash today. In theory they should be adopting Argon2 (or maybe scrypt). In practice, I suspect that either the bindings for Argon2/scrypt don't exist or aren't easily adoptable given their use of ColdFusion. They do exist in Python. Either way, it seems like a sub-optimal decision.

All three are good choices, with their own advantages and disadvantages. Argon2 may be clearly the best choice a few years from now, but both the algorithm and software implementations are immature. It's makes sense to be conservative and go with the more battle-tested options. (Also last I looked Python has no good scrypt bindings.) https://paragonie.com/blog/2016/02/how-safely-store-password... https://news.ycombin…

You're right. I agree. In practice I'd heavily weight the decision based on whatever is best supporting in the Python libraries they are adopting.

Re: Linode Security Advisory

#58
post #47

I've been using Linode for years and I haven't had too many problems with the service itself. That being said, this makes me think twice about staying with them. If I wanted to switch, is the only real competitor Digital Ocean or are there any other good choices?

Why not AWS? They have nano instance types for as little as $5 monthly and infinity more flexibility.

Re: Linode Security Advisory

#59
post #40

Earlier quoted context omitted.

Waiting until login until you upgrade to bcrypt is a requirement is compotent password storage. At this point in time all Linode should know is SHA-2(password) and they can't use that to derive bcrypt(password). The way upgrade should work is that the user provides their password, which is verified with SHA-2 and then hashed with bcrypt and stored again. In order to do this without people logging in Linode would have…

> Waiting until login until you upgrade to bcrypt is a requirement is compotent password storage It's not even remotely competent. This blog makes it clear they're not even sure how their secret key was stolen. These hashes could be walking out their backdoor as I type this. Keeping vulnerable hashes at rest is insane. It would be far more competent to bcrypt the SHA-2s, so that at least when the hashes wander out th…

That will work as a way to strengthen the hashes (a few other people pointed that out as well).

My point was that if you have a system which can go straight from SHA2(password) to bcrypt(password) then the system must be storing the plaintext of the password, which would be very bad.

Re: Linode Security Advisory

#60
post #34
post #32

Earlier quoted context omitted.

My reading was that: 1. Their security partner (whoever that is) didn't see the Lish vulnerability, either because Linode's security team had already fixed it or because they just missed it. 2. Their records weren't sufficient to show the breach itself.

I'm not just trying to be argumentative. Here, let me quote more specifically: no evidence of abuse or misuse of Linode’s infrastructure that would have resulted in the disclosure of customer credentials. I feel like I must be misreading something. Didn't they say earlier that they found secrets from their account credentials database on a customer instance that was used to attack (apparently) PagerDuty? That's not "…

you forgot "that would have resulted in the disclosure of customer credentials".

I'm speculating, but having been a fly on the wall at this kind of meeting before, here's my theory of how this went down:

Manager: "So somebody got the key to generate one-time password tokens for PagerDuty. How did that happen?"

Engineer: "I have no idea."

Manager: "What about that Lish vulnerability? Could it have been that?"

Engineer: "Could have been."

Manager: "We don't know?"

Engineer: "I mean, they could have gotten it that way, but we don't have logging for security vulnerabilities, because, you know, we would just fix them instead of logging that they occurred..."

Manager: "Okay, I want a second opinion. Time to bring in $SECURITY_PARTNER."

$SECURITY_PARTNER: "What Engineer said. And here's a bill for our time."

Manager: "What about if the customer screwed up and lost a device? Could it be that?"

Engineer: "It could be that."

Support: "PagerDuty said something about wiping their phone."

Engineer: "If they did lose a device, it would be indistinguishable from the evidence we have."

Manager: "So, maybe we weren't hacked after all?"

Engineer: "Maybe not."

Manager: "Any other theories?"

Engineer: "I just want to go on record... AGAIN... that I don't trust our ColdFusion infrastructure at all. The people responsible have all left, and none of us understand it. We need to rewrite in a cooler language, like Python. And hire some actual security people."

Manager: "Sigh. We've gotten enough flak over this that I guess I'll put it in the budget. Okay, good meeting everyone. I'll write this up in a blogpost, because we need to show transparency. That way customers are at least as confused as we are."

Post reply on HN