16 TB and 20,000 IOPS EBS Volumes
aws.amazon.com
16 TB and 20,000 IOPS EBS Volumes
1–10 of 38 posts
Re: 16 TB and 20,000 IOPS EBS Volumes
#2That's within an order of magnitude of RAM. You're getting 16TB of RAM!
Re: 16 TB and 20,000 IOPS EBS Volumes
#3Still slow on the scale of things these days. I just provisioned a few servers with over 500,000 IOP/s / 2000MB/s read and write each, 100% SSD with 3-10 year warranties and they use bugger all power. Very low running cost and maintenance overhead and cost less than 8k a unit (1u chassis, redundant power, 32GB RAM, 2x 6 core Xeon v3) and I can guarantee the performance is consistent and there when We need it.
I'm all for outsourcing hardware hosting ('cloud') to save costs and to allow for quick provisioning of new instances - but went you need raw power and in cases where it's inefficient to horizontally scale - the latest generation of PCIe NVMe SSDs are really very impressive and in a recent evaluation we performed of our storage - it was actually going to work out significantly cheaper to A) host our high speed storage ourselves and B) buy SSDs and do away with rotational drives.
Re: 16 TB and 20,000 IOPS EBS Volumes
#4Did the IOPS (and better yet, enhanced IOPS) help any big user? DataStax and Eleasticsearch Inc recommend to stripe instance stores (doing raid0), which in the cloud is not the worst idea as long as you have multiple nodes and proper ring designed. That being said, now I lose the ability create snapshot because the volumes are not EBS. From my experience the IOPS isn't very stable.
Re: 16 TB and 20,000 IOPS EBS Volumes
#5Still slow on the scale of things these days. I just provisioned a few servers with over 500,000 IOP/s / 2000MB/s read and write each, 100% SSD with 3-10 year warranties and they use bugger all power. Very low running cost and maintenance overhead and cost less than 8k a unit (1u chassis, redundant power, 32GB RAM, 2x 6 core Xeon v3) and I can guarantee the performance is consistent and there when We need it. I'm all…
Can you completely snapshot those volumes at any time, recreate them and attach them to new servers? Could you take these snapshots and easily copy them around the world?(again assuming you could snapshot). Are those SSD's automatically replicated to 2 different storage devices behind the scenes to give you near-instant failover? When they go boom are you then driving out to the datacenter to replace them (assuming you have replacements and don't need to wait for them to arrive). Can you do all of this without any upfront cost or excess in capacity??
Probably not. At all.
EBS is NOT harddisks inside a server. Comparing them to such is missing out on all the things that makes it a SERVICE and not disks you buy from Newegg/PCmall/. Yes there are disks you can buy to physically put in a server and they are super blazing fast. In fact AWS has those in their i2 instances and they get hundreds of thousands of IOPs as well.
This isn't even comparing apples to oranges, its apples to space monkeys.
Re: 16 TB and 20,000 IOPS EBS Volumes
#6That's within an order of magnitude of RAM. You're getting 16TB of RAM!
> This volume can host a 16 TB and 20,000 IOPS I/O-intensive transactional database requiring single digit millisecond latencies with consistent performance using Provisioned IOPS (SSD) on AWS
No, not even close.
Re: 16 TB and 20,000 IOPS EBS Volumes
#7Did the IOPS (and better yet, enhanced IOPS) help any big user? DataStax and Eleasticsearch Inc recommend to stripe instance stores (doing raid0), which in the cloud is not the worst idea as long as you have multiple nodes and proper ring designed. That being said, now I lose the ability create snapshot because the volumes are not EBS. From my experience the IOPS isn't very stable.
Provisioned IOPS are quite stable, in my experience. That's their entire point.
Re: 16 TB and 20,000 IOPS EBS Volumes
#8Still slow on the scale of things these days. I just provisioned a few servers with over 500,000 IOP/s / 2000MB/s read and write each, 100% SSD with 3-10 year warranties and they use bugger all power. Very low running cost and maintenance overhead and cost less than 8k a unit (1u chassis, redundant power, 32GB RAM, 2x 6 core Xeon v3) and I can guarantee the performance is consistent and there when We need it. I'm all…
Can you completely snapshot those volumes at any time, recreate them and attach them to new servers? Could you take these snapshots and easily copy them around the world?(again assuming you could snapshot). Are those SSD's automatically replicated to 2 different storage devices behind the scenes to give you near-instant failover? When they go boom are you then driving out to the datacenter to replace them (assuming y…
Yes we can and do snapshot them, at several levels actually - I don't think that's a particularly hard thing to do so I'm not sure why that's relevant.
Yes there is replication both to separate disk arrays AND seperate physical servers with live failover and load balancing - again nothing new here?
No we don't send out storage to other countries - in fact that would be illegal, and if we were to do so our clients would suffer as Australia's international peering is pretty woeful.
We also gain on-disk compression and encryption on a LUN by LUN basis as we require it, storage is automatically provisioned to new application instances, all the software is 100% open source and mature, we don't have to phone a large corporate that doesn't really care about us, we pass security audits because we can prove where things are and how they're configured.
By the way, none of this is your 'new egg' gear you referenced, we use Intel DC P3600/P3700 PCIe storage. Oh and as a bonus - there's no licensing or monthly invoices that need attention.
Is shared hosting / hardware outsourcing / cloud computing amazing - yes! Of course it is!
But you must remember it is their intentions to sell their product as the only right answer and to tell you what you should care about. In some cases it applies and in some it doesn't. The danger in jumping on the bandwagon and becoming an Amazon 'fanboy' (I'm really sorry for using that term - I hate it) is that you quickly become silod from external opperuntities and security / high vertical performance solutions.
If I was in a small team of devs working on launching a web app that's going to be targeted at an international audience, my growth is highly unpredictable, our future uncertain and our skill set focused on developing great software - I wouldn't think twice about using AWS/Rackspace etc...
But when you understand your environment well, when you have a limited budget, when you have a predicable customer base with strick security requirements and when you're pushing databases pretty hard - would I use AWS? No, it's not cost effective for us, nor is it legally (and perhaps morally) viable. Do we waste lots of time looking after our hardware? No! It's 2015 - hardware is easy.
Re: 16 TB and 20,000 IOPS EBS Volumes
#9That's within an order of magnitude of RAM. You're getting 16TB of RAM!
I think you severely underestimate RAM bandwidth on modern servers.
Re: 16 TB and 20,000 IOPS EBS Volumes
#10Earlier quoted context omitted.
Can you completely snapshot those volumes at any time, recreate them and attach them to new servers? Could you take these snapshots and easily copy them around the world?(again assuming you could snapshot). Are those SSD's automatically replicated to 2 different storage devices behind the scenes to give you near-instant failover? When they go boom are you then driving out to the datacenter to replace them (assuming y…
Yes we can and do snapshot them, at several levels actually - I don't think that's a particularly hard thing to do so I'm not sure why that's relevant. Yes there is replication both to separate disk arrays AND seperate physical servers with live failover and load balancing - again nothing new here? No we don't send out storage to other countries - in fact that would be illegal, and if we were to do so our clients wou…
You say LUN, are these SAN devices, or is it direct attached storage? Was the replication, load balancing, and snapshotting all something that you set up and manage yourselves?
--edit-- Ahh you've been editing your comments so the thread is a bit out of wack! (no problemo)
Newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167... ;) (yes its not the same as some of the much higher end stuff).
Fair enough, but again, your comment is about hardware that you are managing, that you've built, thats glued together from a lot of different components, both software and hardware, and this post is about a cloud service that doesn't even compare. So your initial post comes off a bit as trolling for the sake of trolling.
I've done my fair share of rack-n-stack, and I've now spent the past few years "in the clouds" as it were. Wouldn't go back for anything, but I dont think this makes me a fanboy. Sure there is kit that you'd only ever be able to build/buy yourself (for now at least), but most ppl will never need more than 100k IOPs, let alone 500k+.
--edit again-- In regards to security, if you think you are a capable of running an infrastructure more secure in a datacenter yourself, than on one of the major 3 cloud provider's infrastructure ( AWS, GOOG, MSFT ) where they have some of the best sec teams in the world, then you are probably not as deeply aware of whats possible in cloud from a security standpoint. Banks, Medical institutions, government agencies, and so forth are all trusting their infrastructure on the cloud, across many countries in the world.