Code Spaces data and backups deleted by hackers • Hacker News Board

Re: Code Spaces data and backups deleted by hackers

#81

Post by guiambros » Fri, Jun 20, 2014, 3:49 AM UTC

jacquesm wrote: post #73

guiambros wrote: post #65

Earlier quoted context omitted.

Couldn't agree more. Everything under a single platform, no MFA, no (real) offsite backup, and on top of that they spent 12 hours corresponding with the attacker, instead of immediately calling Amazon to ask their help to shut down everything, while they still had time? I'm sorry, but this is a succession of things not to do in terms of system operations. Probably the team never managed mission critical platforms bef…

How many companies have not yet learned that lesson? There are probably a lot of codespaces on AWS. My reasoning is that if you make it so that a developer can set up a virtual datacenter but does not have the background of actually running such an installation then you're going to have to assume that it is probably quite fragile. Software people tend to make all kinds of assumptions about hardware that do not work o…

That the problem with the recent "DevOps" trend. Lots of people coming from a "Dev" background, but no real "Ops".

And now that spinning up a couple of servers on AWS and creating snapshots on-the-fly are so easy, it gives the false impression that you don't need much to act as a sysadmin.

Re: Code Spaces data and backups deleted by hackers

#82

Post by [deleted] » Fri, Jun 20, 2014, 10:58 AM UTC

gnu8 wrote: post #4

Why is it possible to destroy an entire enterprise by compromising an Amazon account? Where the fuck is their 2FA? What about a cooling off period before committing changes like deleting all of your storage? Amazon's infrastructure seems to be built without essential safeguards.

[deleted]

Re: Code Spaces data and backups deleted by hackers

#83

Post by stevekemp » Fri, Jun 20, 2014, 12:48 PM UTC

gnu8 wrote: post #4

Why is it possible to destroy an entire enterprise by compromising an Amazon account? Where the fuck is their 2FA? What about a cooling off period before committing changes like deleting all of your storage? Amazon's infrastructure seems to be built without essential safeguards.

Amazon does have a remarkably fine-grained control mechanism - but you need to use it.

For example I never publish my Route53 (DNS hosting service) keys, but even if they were leaked the account is only setup on the Amazon side to work from a single source IP.

You can restrict permissions significantly, so again in my case I've got a user configured who can only add/delete DNS records - but cannot create a new zone, or delete other zones. Not ideal, since "remove all records" is almost the same as "delete zone" in practice, but I'm not worried that unrelated zones on that account will be broken if I do lose my keys.

Re: Code Spaces data and backups deleted by hackers

#84

Post by Rapzid » Sun, Jun 22, 2014, 1:46 PM UTC

Arnor wrote: post #16

From the Amazon RDS documentation: When the backup retention changes to a non-zero value, the first backup occurs immediately. Changing the backup retention period to 0 turns off automatic backups for the DB instance, and deletes all existing automated backups for the instance. http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overvi...

Unfortunately one of the major backup related pain points with RDS currently is you have no way to interact with the automated backups aside from restoring an instance to a point in time. They are also deleted when the instance is deleted. Manual snapshots aren't. Manual snapshots can be used to launch and new instance, or they can be moved to a different region. You MUST do your own backups of the databases if you want to get them "offsite" or off account. Like I said, PITA right now and from what I here a common request on RDS is being able to get at the automated backups and snapshots so you can move them offsite.

Re: Code Spaces data and backups deleted by hackers

#85

Post by JohnTHaller » Mon, Jun 23, 2014, 1:26 PM UTC

judk wrote: post #33

maxlybbert wrote: post #28

Earlier quoted context omitted.

Not to be too flippant, but the company's closing shop. So, yeah, the DevOps are fired, along with everybody else. As for the rest of us: AWS is a great one-stop shop. Unfortunately, using just AWS puts you in the "all the eggs in one basket" scenario that we were warned against as children.

Two-factor authentication is a second basket. Sending a copy to Glacier is a second basket. Does Amazon not have 30-day undelete for bulk storage? Seems crazy.

> Two-factor authentication is a second basket.

Two-factor authentication is a second basket is a better carrying strap but it's still on a single basket.