+100 points to the excellent debugging skill demonstrated by the author.
It's great to see someone in top form.
Docker operations slowing down on AWS
11–20 of 176 posts
Re: Docker operations slowing down on AWS
#12The author's takeaways include moving disks to io1. This is a bad bargain in most cases, and particularly bad in the ~500 IOPS range (which is what I'm seeing in the Grafana screenshot there).
gp2 disks get 3 iops per gig "free", bursting up to 3k. (They don't burst after 1 tb, because your baseline performance is higher than the burst rate.) io1 is 25% more expensive per-gb, and you pay by the IOPS on top of that.
A 175gb gp2 disk will give you 525 IOPS baseline, at $17.50 a month. I'm guessing his volume is about 40gb, doing math backwards from his bottlenecked IOPS; a 40gb io1 with 500 IOPS will cost you $83.75 a month! And on top of that, AWS will cap you hard at that 500 IOPS; the gp2 can still burst if needed.
I know of two general cases where io1 disks are worthwhile: if you need more than 10k IOPS, or if you have very high IO requirements but very stable disk space usage (e.g. high-performance RRDs). Crack open Excel and do the math; it's worth the five minutes to check.
(Also, your burst balance is available as a Cloudwatch metric, as I recall! Set alarms on that shit!)
Re: Docker operations slowing down on AWS
#13And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Rent a server: do you mean dedicated hosting or something else?
Re: Docker operations slowing down on AWS
#14How do you flip a volume on the fly? I thought you had to do the "snapshot > make new volume > reattach" route
edit: thanks for the info, znep (hit my comment limit, hence the edit... )
Re: Docker operations slowing down on AWS
#15And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Ah, that nasty AWS vendor lockin. Guess I'll just run my own CDN then, shouldn't be hard (can't use other vendors, because lock-in). L'il ol me running edge nodes all over the five continents where we have clients. I'll have it done by closing time today.
AWS is more than just compute cores, and you're hand-waving away what it offers.
Re: Docker operations slowing down on AWS
#16And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Yes. Look at https://www.packet.net/bare-metal/ for bare metal boxes provisioned in less time than an ec2 instance.
Or if you want to go old school, cheaper, and less sexy/api driven: https://www.delimiter.com/
If you shop, for around 30-40 a month you can get 16 cores, 32gb ram, and a 120GB ssd or 1-2TB spinny disk.
Re: Docker operations slowing down on AWS
#17And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Yes. Look at https://www.packet.net/bare-metal/ for bare metal boxes provisioned in less time than an ec2 instance. Or if you want to go old school, cheaper, and less sexy/api driven: https://www.delimiter.com/ If you shop, for around 30-40 a month you can get 16 cores, 32gb ram, and a 120GB ssd or 1-2TB spinny disk.
That price for that config sounds amazing. Any pointers?
Re: Docker operations slowing down on AWS
#18And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Ah, that nasty AWS vendor lockin. Guess I'll just run my own CDN then, shouldn't be hard (can't use other vendors, because lock-in). L'il ol me running edge nodes all over the five continents where we have clients. I'll have it done by closing time today. AWS is more than just compute cores, and you're hand-waving away what it offers.
To be fair though, even with a setup like this you can still use Cloudfront, or any number of other 3rd party CDNs. Most of these providers including Cloudfront offer 'remote origin' features/configurations.
I agree that AWS offers much more than hosting, but when talking about hosting specifically the numbers are clearly better with 'bare metal' (rented or otherwise) -- it's just substantially more power per dollar. You'll need to do a cost-benefit for your own situation so YMMV of course, just as it would with any other provider or any other service your business may choose to subcontract.
Re: Docker operations slowing down on AWS
#19And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Yes. Look at https://www.packet.net/bare-metal/ for bare metal boxes provisioned in less time than an ec2 instance. Or if you want to go old school, cheaper, and less sexy/api driven: https://www.delimiter.com/ If you shop, for around 30-40 a month you can get 16 cores, 32gb ram, and a 120GB ssd or 1-2TB spinny disk.
Less time than ec2? Last time I test drove packet it was around 10 min
Re: Docker operations slowing down on AWS
#20And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Ah, that nasty AWS vendor lockin. Guess I'll just run my own CDN then, shouldn't be hard (can't use other vendors, because lock-in). L'il ol me running edge nodes all over the five continents where we have clients. I'll have it done by closing time today. AWS is more than just compute cores, and you're hand-waving away what it offers.
I think it’s clear what was intended. Nobody disputes that targeted uses of third party infrastructure services can be appropriate. After all, it’s relatively easy to migrate a service to a new CDN - but it’s a lot harder to migrate from AWS when it’s a core part of your application architecture!