Security incident update • Hacker News Board

Re: Security incident update

#231

Post by [deleted] » Tue, Apr 16, 2013, 5:38 PM UTC

jmillikin wrote: post #228

dangrossman wrote: post #222

Earlier quoted context omitted.

Let's make this more clear: Linode has two servers. One is a web server that runs their website and contains its source code. One is a database server containing customer information and encrypted credit card numbers. The web server is connected to the internet. The database server is not. They communicate over the private network. The web server is compromised. The attacker has the source code, which includes the IP…

> The web server is compromised. The attacker has the > source code, which includes the IP/host and login > credentials for the database server, since the website > code talks to the database to function. IMO, if internet-facing machine A is storing login credentials to machine B, then machine B should be considered internet-facing. There is no reason for a properly-designed system to be storing passwords or credit c…

[deleted]

Re: Security incident update

#232

Post by nenolod » Tue, Apr 16, 2013, 5:51 PM UTC

mekoka wrote: post #142

JshWright wrote: post #139

Earlier quoted context omitted.

>That having been said, I think Linode just learnt a really unpleasant lesson and will become an even more secure provider for it. I believe that moving my sites away from them at this moment would be like selling stocks when they're about to go up. What concerns me is that the would have been perfectly happy to sweep most of this under the rug if Ryan hadn't forced their hand. The security lesson is one thing (hacks…

Don't assume that the most transparent company out there isn't choosy about what it's being transparent about. There are plenty of things we never know that don't hurt us. I choose not to concern myself with such things.

The problem is that Linode is not anywhere near the "most transparent company out there."

Re: Security incident update

#233

Post by nenolod » Tue, Apr 16, 2013, 5:54 PM UTC

corford wrote: post #89

thaumaturgy wrote: post #77

They reset the passwords on Friday. Even assuming that they've been working for the last four days on discovering the extent of the intrusion, there's a pretty big gap between Friday's announcement: "This activity appears to have been a coordinated attempt to access the account of one of our customers. This customer is aware of this activity and we have determined its extent and impact. We have found no evidence that…

This is what Caker (founder of Linode IIRC?) has to say about the strength of the passphrase: "our private key is stored only in encrypted format. The passphrase is not guessable, sufficiently long and complex, not based on dictionary words, and not stored anywhere but in our heads." [1] [1] http://blog.linode.com/2013/04/16/security-incident-update/c...

Except you know, system memory when the key material is unlocked... unless you really believe someone is sitting there typing in the passphrase for each non-batch CC transaction, because I do not.

Re: Security incident update

#234

Post by marshray » Tue, Apr 16, 2013, 5:54 PM UTC

gingerlime wrote: post #27

rainforest wrote: post #22

Earlier quoted context omitted.

From their description it appears encrypted CC numbers were in the database amongst the other customer information. Sure, the data has to live somewhere, but the apparent situation of CC, customer, private key all accessible to the frontend looks sub-optimal.

there's a difference between obtaining the private key and the private key file , which according to Linode was protected with a (hopefully strong) passphrase...

Why would you keep the private key file on a machine on which you would never enter the passphrase?

Re: Security incident update

#235

Post by cmelbye » Tue, Apr 16, 2013, 5:57 PM UTC

gburt wrote: post #209

lawnchair_larry wrote: post #204

Earlier quoted context omitted.

Doesn't make sense. Who here thinks they typed that in manually for every single CC they got? They didn't, encryption/decryption was automated, meaning anyone could have done it if they had access to the process that normally does. Also, the post says salted sha256 passwords are useless. Evidently, they do not have a clue how to safely store passwords, because that is not useless at all. Finally, it's an outright lie…

It's a private key. You need the public key to encrypt, which is not password-encrypted. Although I still wonder about how they run recurring charges.

It could be a manual script that he runs which prompts for the passphrase. I know that earlier in Linode's history, caker made it seem as if it worked that way from the way he acted in their IRC channel when the batch transactions were running.

Re: Security incident update

#236

Post by marshray » Tue, Apr 16, 2013, 5:57 PM UTC

otherdave wrote: post #82

moconnor wrote: post #50

Earlier quoted context omitted.

Bear in mind this can be as simple as a handful of people emailing their LISH password to support while asking if it is correct etc.

Or someone mistakenly logging "Bad LISH Password: 'FOO' for user 'BAR'" somewhere that logs to the DB.

Or users accidentally typing their password in the username field.

Nevertheless, this isn't the first time Linode has needed to spend their benefit-of-the-doubt points.

Re: Security incident update

#237

Post by cheald » Tue, Apr 16, 2013, 5:57 PM UTC

entropy_ wrote: post #73

cheald wrote: post #2

Good to see some discussion and acknowledgment from Linode, but their treatment of the credit card encryption is distressing. They don't address the claim that the private keys were on the servers' filesystems, and instead focus on them being password protected. If the password is brute-forced, then it's game over.

That doesn't make any sense. It's exactly like saying: "Oh sure, the credit card numbers were encrypted but if the encryption key is brute-forced then it's game over". The CC numbers were encrypted using a public key. The corresponding private key was itself encrypted using "[a] complex passphrase [that] is not stored electronically". They could be lying, but if they are not then this is perfectly valid security. Usi…

My concern is that if the password was poorly chosen, the private key won't be very safe if they got the private keyfiles. That many credit card numbers is a huge target so it'd be worth an attacker expending some money on.

If it's a strong, long non-dictionary passphrase, then I'm not nervous. But, we don't really have any idea if it is or not. The fact that their web interface boxes had the private key on them is a huge concern - you'd ideally run all billing stuff on a machine that is highly locked down, perhaps not even accessible outside of a very strict range of IP addresses. The practice of leaving the private keys hanging around on the same boxes that ran the website makes me awfully nervous about the other links in the chain.

Re: Security incident update

#238

Post by cmelbye » Tue, Apr 16, 2013, 6:03 PM UTC

DigitalSea wrote: post #121

Hey Linode how about addressing the claims that the private and public keys were both stored in the same location on the compromised manager server. Is that true? And if so, why wasn't it revealed? Did they get access to any credit card information or was it truly just the last 4 digits used for lookup purposes? In my opinion these are the real questions us Linode customers want to know the answers to, sooner rather…

They did address that.

Re: Security incident update

#239

Post by nenolod » Tue, Apr 16, 2013, 6:05 PM UTC

shadowfox wrote: post #131

InclinedPlane wrote: post #45

Earlier quoted context omitted.

CC data however does NOT have to live on any machine that anyone except your merchant bank owns. If you are writing against billing APIs in the 21st century then you only need to store a token which you use for recurring billing. Not everyone does this, but it's a reasonably good practice, especially if you have had intrusions in the past.

Out of curiosity, is this token generation sufficiently general or is it tied to a specific bank/credit card provider? (Basically is it easy to switch providers without having all your customers enter their cc information again?)

Unfortunately it is tied to the merchant. :(

Re: Security incident update

#240

Post by nenolod » Tue, Apr 16, 2013, 6:06 PM UTC

gingerlime wrote: post #44

markbao wrote: post #38

Earlier quoted context omitted.

I feel like there is a fine line between a "cover up" and a "grey hat disclosure". Reading the IRC logs from #linode [0], HTP hacker ryan* seems to say that they made a deal of "we don't tell if you don't tell" but then Linode broke the deal by reporting them to law enforcement. You could see this as a cover up or you could see this as a disclosure from the crackers to Linode. I see it as a cover up, since Linode sho…

Perhaps nitpicking, but looks like HTP might have gained password hashes and encrypted credit card numbers, as well as a passphrase protected private key file . That's if I'm reading between the lines of both ryan and Linode's statement. This is why you build security in layers. If one layer gets broken, there are other layers to protect you. I'm not underestimating the severity of this incident, but I think it's goo…

Except that LISH passwords were plaintext...