So you are basically paying 125$+ to have S3 cached locally and get an iSCSI interface ?
Which is worth it.
The price of enterprise backup solutions is crazy.
AWS Storage Gateway
11–20 of 34 posts
Re: AWS Storage Gateway
#12I don't have a personal need for this right now, but the fact that it stores the data as EBS volumes is pretty cool. I could imagine having local servers automatically mirrored so that they could be failed over to instances on ec2. Very powerful stuff indeed.
Note that the mirroring is asynchronous -- if your local server fails you can replace it with an EC2 instance, but you lose anything since the last snapshot.
If I read it correctly, the server can fail but because the data is stored via iSCSI, it will not actually be lost (unless it is the appliance itself that fails)
Re: AWS Storage Gateway
#13Earlier quoted context omitted.
Note that the mirroring is asynchronous -- if your local server fails you can replace it with an EC2 instance, but you lose anything since the last snapshot.
If I read it correctly, the server can fail but because the data is stored via iSCSI, it will not actually be lost (unless it is the appliance itself that fails)
Sure. But if we're ignoring the case where the appliance doesn't fail it's no better than a drobo.
Re: AWS Storage Gateway
#14I don't have a personal need for this right now, but the fact that it stores the data as EBS volumes is pretty cool. I could imagine having local servers automatically mirrored so that they could be failed over to instances on ec2. Very powerful stuff indeed.
Note that the mirroring is asynchronous -- if your local server fails you can replace it with an EC2 instance, but you lose anything since the last snapshot.
Not since the last snapshot. It syncs as close to real-time as possible without the latency and network speeds affecting the performance locally.
This solution isn't designed to replace fault-tolerance on local hardware. It is for close to realtime offsite replication and backup.
Data in S3 is stored in at least three geographically separate locations and snapshots are very fast and very efficient on storage space.
The final major advantage you get through a solution such as this is that if you do have your primary site go down (floods, tornados, etc), you can bring up all your existing images via EC2 without having to have a bunch of redundant hardware sitting around waiting for disaster to strike.
And what do you pay for this? $125/month plus a per GB storage cost CHEAPER than enterprise storage generally is.
Re: AWS Storage Gateway
#15This should make EMC sit up and take notice. Amazon is doing great. I would have expected dropbox to do this after icloud was released. They should atleast mimic this now.
Re: AWS Storage Gateway
#16Sounds like a great idea. I can't wait until they release Gateway-Cached Volumes myself as it better suits my use-case.
the volumes are cached... from reading the post, you write synchronously to the iSCSI device, which is Asynchronously sent to S3
Re: AWS Storage Gateway
#17Earlier quoted context omitted.
If I read it correctly, the server can fail but because the data is stored via iSCSI, it will not actually be lost (unless it is the appliance itself that fails)
Sure. But if we're ignoring the case where the appliance doesn't fail it's no better than a drobo.
the storage for the appliance can be outside the appliance (DAS or SAN storage). the appliance can die, but you just plug in a new appliance and point at the storage, and your back... as for the drobo comment, this should be faster than a drobo... and even ignoring the appliance fail, the off site backup is key.
Re: AWS Storage Gateway
#18Can someone give a brief explanation of what this service is good for?
ok, couple of things to start with:
1: iSCSI can be seen as a USB drive over a network. you plug in, your machine sees it as a drive, you write data. you can unplug, and then plug in somewhere else, and as long as the file system is readable on the new machine, you can get your files. 2: the appliance AWS are offering gives you iSCSI volumes, backed by DAS or SAN storage locally, but also backed by S3 storage in AWS.
So, basically, its like having a drive, automagically backed up to S3, but S3 does not need to know anything about what is on the drive. it could be VMs, Videos, your mail server... anything really.
Re: AWS Storage Gateway
#19Can someone give a brief explanation of what this service is good for?
It sounds to me like enterprisey dropbox: It backs up your files from your local file server to the aws cloud, and if your server dies, bung in a new one and all your files will reappear. Great idea, although maybe something you could already do with dropbox?
Except you can use it to power on EC2 machines with those data, so you can use it for dealing with requests spikes, for offsite data elaboration or even Disaster Recovery of your onpremises infrastructure on Amazon.
Re: AWS Storage Gateway
#20Sounds like a great idea. I can't wait until they release Gateway-Cached Volumes myself as it better suits my use-case.
the volumes are cached... from reading the post, you write synchronously to the iSCSI device, which is Asynchronously sent to S3
I'm not sure if that's what kondro meant but what I would like to see is a virtually unlimited size ("elastic") volume where the local disk acts only as the cache.
This would make a wide range of big-storage use-cases ridiculously trivial - those where only ~10% of the data-set is frequently accessed.
I.e. one could lazily scale the expensive local storage with throughput-demand, while the S3 backing store takes care of the long-tail (which can easily be many terabytes long when you're dealing with media files).