Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

theregister.com

Post reply on HN
Print view View original

31–40 of 50 posts

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#31

Post by throwaway382985 »

I worked at scaleway.

This whole post is basically bullshit, "secure" transporation meant taking a random electric car (renault zoe) at the company headquarters or at DC5 and throwing hardware in the trunk.

No security in place whatsoever, servers laying in datacenter hallways fully loaded with disks, accessible to anyone.

Most of the company had access to the back office where they could just access customers' data without any kind of logging. (Internally this was called sudo mode on the online console, we had basically to click on a button to log as any customer).

The "funniest" was the corporate vpn network being shared by the internal datacenter network which meant any employee had, for instance, full access to all the home made switches management interfaces which had no access control whatsoever, it happened that employees stumbled accross this and wondering what it was (LOL), there was even a button to upload and flash (!) a firmware.

The upper-management was terribly incompetent and would discard any kind of issue that was not affecting sales in the immediate term as not important (security or otherwise).

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#32

Post by emteycz »

throwaway382985 wrote: post #31

I worked at scaleway. This whole post is basically bullshit, "secure" transporation meant taking a random electric car (renault zoe) at the company headquarters or at DC5 and throwing hardware in the trunk. No security in place whatsoever, servers laying in datacenter hallways fully loaded with disks, accessible to anyone. Most of the company had access to the back office where they could just access customers' data…

What about GDPR, did it change anything?

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#33

Post by atonse »

rkeene2 wrote: post #24
jsiepkes wrote: post #14

Earlier quoted context omitted.

> There's a checkbox to encrypt the disks when installing some linux distros. While I'm not going to defend moving around unencrypted disks it is not as simple as that. The difficult part is not the encryption. The difficult part is the key management of the encryption. You need to solve issue like ensuring you don't lose access to the keys (and thereby access to the data), securely providing keys to people and machi…

When I ran a cloud provider, we used a hardware security module on storage systems which had public/private key pair which could be used to decrypt the header on the disk which contained a copy of the symmetric cipher key. Each header had the symmetric cipher key encrypted with multiple different public keys, including a fallback whose private key only existed in on paper in a vault. Each system could reach out to ot…

This is exactly why most of us are more than happy to pay a premium to AWS, GCP, and Azure. They reduce all what you listed to a handful of checkboxes for us plebs. You have to have a few discussions about how you as an org will manage your keys, and you're a couple of terraform files away for having access to this (budget notwithstanding).

This is the level of redundancy/backups/processes the large cloud providers have. It's not just "hey is this encrypted" – it's "what happens, how do you retire a drive, are the modules FIPS 140-2 compliant, how do we physically secure our HSMs?" etc

It's just so much more than "hey Digital Ocean is 50% cheaper"

AWS's egress bandwidth charges still feel criminal though.

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#34

Post by BBC-vs-neolibs »

rasz wrote: post #18
terom wrote: post #6

Earlier quoted context omitted.

Based on links in the other hackernews thread, it was quick-formatted, but a filesystem containing qemu disk images was recovered: [1] The YouTube video is in French without English CC subtitles, but there's screenshots of exploring what I assume would be one of the qcow images: [2] * 03:59 `/root/.ssh` with SSH id_rsa (private) keys, (public) authorized_keys and (hashed) known_hosts * 07:14 server binaries * 08:42 s…

>it was quick-formatted are there OSes that perform quick format without issuing TRIM command to the SSD? TRIM would vanish the data in less than 15 minutes.

Not if you turn the computer off right after deleting the files.

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#35

Post by michaelt »

jtchang wrote: post #27
rkeene2 wrote: post #24

Earlier quoted context omitted.

When I ran a cloud provider, we used a hardware security module on storage systems which had public/private key pair which could be used to decrypt the header on the disk which contained a copy of the symmetric cipher key. Each header had the symmetric cipher key encrypted with multiple different public keys, including a fallback whose private key only existed in on paper in a vault. Each system could reach out to ot…

Is there anything on the consumer level like this?

Well, a lot of laptops these days have TPMs that can store disk encryption keys. BitLocker on Windows, or an exotic dm-crypt setup on Linux.

However for a server, the chance of a thief getting a disk and the HSM at the same time is low; and you really don't want to have to enter a password every boot. So the HSM design delivers some benefits worth having.

Whereas for a laptop, you can easily type in a password; and the chance of a thief snatching the disk and TPM at the same time is basically 100%. So there hasn't been a big push among Linux users to start using the TPM.

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#36

Post by capableweb »

dracodoc wrote: post #21
raverbashing wrote: post #20

The things that are not clear to me are: - Why was the disk being replaced (SMART warnings? HW upgrade? It failed in some way) - Why was the quick format recoverable? Was TRIM issued or the HW didn't support it? - As a follow up, I'm guessing the HW didn't support HW encryption? - Why are they relying on 3rd parties for the destruction of data? Meanwhile, it seems that HDs don't leave Google datacenters in one piece

The disk could be simply transferred to another data center physically so they can move data faster/cheaper than transferring the data by internet.

Reminds me of the following quote:

> Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

- Andrew S. Tanenbaum

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#37

Post by rkeene2 »

atonse wrote: post #33
rkeene2 wrote: post #24

Earlier quoted context omitted.

When I ran a cloud provider, we used a hardware security module on storage systems which had public/private key pair which could be used to decrypt the header on the disk which contained a copy of the symmetric cipher key. Each header had the symmetric cipher key encrypted with multiple different public keys, including a fallback whose private key only existed in on paper in a vault. Each system could reach out to ot…

This is exactly why most of us are more than happy to pay a premium to AWS, GCP, and Azure. They reduce all what you listed to a handful of checkboxes for us plebs. You have to have a few discussions about how you as an org will manage your keys, and you're a couple of terraform files away for having access to this (budget notwithstanding). This is the level of redundancy/backups/processes the large cloud providers h…

We were a relatively small cloud provider, but all disks were encrypted so at least we wouldn't leak someone's data if we had to replace a disk.

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#38

Post by Abishek_Muthian »

throwaway382985 wrote: post #31

I worked at scaleway. This whole post is basically bullshit, "secure" transporation meant taking a random electric car (renault zoe) at the company headquarters or at DC5 and throwing hardware in the trunk. No security in place whatsoever, servers laying in datacenter hallways fully loaded with disks, accessible to anyone. Most of the company had access to the back office where they could just access customers' data…

There was a post in the scaleway user forum(~2020) asking 'Who has access to the data on the disk' to which the founder or CEO replied that the Engineers do have access but the datacenters are very secure as it's thoroughly monitored.

I remember this because I used fscrypt after seeing the post, Scaleway seems to have removed the entire user forum sometime last year.

I stopped using their service after they removed their ARM severs with short notice, But this is more scary.

Unfortunately this is the reality when it comes to AWS[1] or other top-tier cloud providers vs inexpensive ones. The cost saving has to happen somewhere.

[1] https://aws.amazon.com/blogs/security/importance-of-encrypti...

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#39

Post by jcynix »

Nextgrid wrote: post #26
jnwatson wrote: post #13

Earlier quoted context omitted.

It is trivial in the Ubuntu installer.

It's not trivial when it comes to a server. Sure, you can tick that checkbox you'll enable encryption, but when you reboot you realize that you can't actually SSH in it, as it's waiting for a key to be entered on the physical console. Entering (and managing) that key is the hard part.

That's what Dropbear is supposed to do, it can be included in the initramfs of a server: https://matt.ucc.asn.au/dropbear/dropbear.html

Disclaimer: I still didn't try it myself yet, but it's on my (too long) list of things to do.

Re: SSD belonging to Euro-cloud Scaleway was stolen,then turned up on YouTube

#40

Post by gnufx »

rkeene2 wrote: post #24
jsiepkes wrote: post #14

Earlier quoted context omitted.

> There's a checkbox to encrypt the disks when installing some linux distros. While I'm not going to defend moving around unencrypted disks it is not as simple as that. The difficult part is not the encryption. The difficult part is the key management of the encryption. You need to solve issue like ensuring you don't lose access to the keys (and thereby access to the data), securely providing keys to people and machi…

When I ran a cloud provider, we used a hardware security module on storage systems which had public/private key pair which could be used to decrypt the header on the disk which contained a copy of the symmetric cipher key. Each header had the symmetric cipher key encrypted with multiple different public keys, including a fallback whose private key only existed in on paper in a vault. Each system could reach out to ot…

It's perhaps worth mentioning Mandos for unattended booting of servers with encrypted root https://www.recompile.se/mandos
Post reply on HN