Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker operations slowing down on AWS

jeremyeder.com

Post reply on HN
Print view View original

151–160 of 176 posts

Re: Docker operations slowing down on AWS

#151

Post by eropple »

pizza234 wrote: post #134
hinkley wrote: post #129

Earlier quoted context omitted.

I really struggle with this color of money nonsense that western finance has invented. Tell me again why hiring a developer is cheaper than buying one $8k server that will save us 35 man hours a week? Drives me batty. Rearchitecting may be necessary anyway, but doing this kind of work involves hiring and training better senior devs and retaining them for a couple of years at least. That's not cheap. It's A lot more e…

It's not so black and white. The changes mentioned may be a necessary evil when scaling; in this perspective, moving to the cloud makes a set of solutions (rearchitecting) happen sooner than later. Also, an 8k server is a big unit. Cloud services are much more granular. This is a problem of bare metal - it's easy to overprovision because the base unit is large, and one ends up being happy of having an "overprovisione…

One of the other aspects is that cloud stuff tends to make very apparent stuff that was always true but often ignored (for one reason or another). Like--that rearchitecture would be a good idea regardless of where you are (because stuff like twelve-factor apps are just a fact of life), they just become more obviously necessary when everyone's always hyping up that Anything Can Disappear At Any Time.

Re: Docker operations slowing down on AWS

#152

Post by KaiserPro »

chucky_z wrote: post #148

A 1TB gp2 volume is cheaper than an 1TB, 3000 iops io2 volume, and provides nearly-identical characters. Only use io2 if you have a latency sensitive application or need more than 10,000 iops. Even then you can RAID10 some gp2 volumes together and with enhanced networking get I believe 30k iops out of one instance.

even thenm if you really are worried about latency, AWS isn't the place to be...

Re: Docker operations slowing down on AWS

#153

Post by laumars »

marcosdumay wrote: post #141
laumars wrote: post #71

Earlier quoted context omitted.

Idealistically I agree with you but pragmatically I think more than 1 cloud providers isn't really worth the effort. It's not often that a whole region goes down but even then I can't recall when the whole cloud platform last became inaccessible - usually it's just a region. But once again it comes back to SLIs and client expectations.

But then, "more than 1 cloud providers isn't really worth the effort" and "2 bare metal instances isn't HA. Not even close." are really incoherent. Two machines with two internet connections and a good UPS easily match the availability of AWS.

Not really as HA on physical hardware would be more like 4 machines if you factor in DB plus 2 stacks of switches. But really if you're running HA then you'd probably want 3 web servers rather than 2 so you can perform maintenance and still have redundancy. Which means you'd also need 2 load balancers and some method of code deployment, which will usually mean at least one other box or SAN. If your application is database heavy with lots of reads then you might also want memcache / redis. Or maybe other caching servers like varnish. Bare in mind that if your site is slow and unresponsive then it's as good as unavailable.

This is all in one physical location as well so you'd need to double this spec again.

Then once you've built all of that, you'd probably want to put it behind a CDN as leased lines are expensive.

Only then you're starting to reach feature parity with what I've described in my first post and there will be lots of kit I've not even touched on.

However even if you do just run 2 VMs (web and db) on each of the 2 physical boxes, and don't need redis etc. You still need to double your spec just for the multi-region point I raised earlier.

Re: Docker operations slowing down on AWS

#155

Post by wmf »

nailer wrote: post #80

If you're interested in IO performance, maybe don't run Docker - whose main advantage over VMs is fast IO - on top of VMs unnecessarily? Triton and OpenShift add proper isolation to Docker and hence provide fast IO since you're not adding a layer of Xen.

I love native containers, but none of the major cloud providers support them so that isn't really actionable advice for most people.

Re: Docker operations slowing down on AWS

#156

Post by Deimorz »

neuronexmachina wrote: post #147
geofft wrote: post #107

Earlier quoted context omitted.

How many projects actually get 150,000 hits a second? (And are they not better served by buying and operating their own datacenters?)

How many hits/second does a project have to serve when they're on the front page of HN?

Shouldn't be anything incredibly high. I haven't seen any recent traffic info for HN, but a couple of years ago dang posted some numbers here: https://news.ycombinator.com/item?id=9219581

So at that point it was 2.6M views per day, which means HN itself was getting about 30 views per second. If you look at the 200k uniques instead (which might make more sense since any individual person will probably only click through once), that's about 2 unique visitors per second. So even if HN has grown a ton in the time since that post, I'd be surprised if it sent more than about 5 hits/second to anything.

Re: Docker operations slowing down on AWS

#157

Post by jopsen »

axaxs wrote: post #125
jopsen wrote: post #50

Earlier quoted context omitted.

also scaleway.com (they have API, but no user-data). IMO, docker and container orchestration spells a bright future for bare-metal boxes like these, as you won't need cloudformation, etc.. But I still see few alternatives to S3, many vendors offers block devices, but only the big clouds offer blob storage. Backup and restore from blob storage makes recovery from crash pretty easy.

I rent a dedi arm box from scaleway. It's been up for 700 days, and serves about 30k req/month. I couldn't be happier for the price.

For 30k req/month heroku could a better choice. The cost here is maintenance and backup.

Note: I'm still hopeful that things like k8n will make running stateful containers on random metal easy without huge investments in configuration for backup and reliability.

Re: Docker operations slowing down on AWS

#158

Post by jopsen »

RX14 wrote: post #68
jopsen wrote: post #50

Earlier quoted context omitted.

also scaleway.com (they have API, but no user-data). IMO, docker and container orchestration spells a bright future for bare-metal boxes like these, as you won't need cloudformation, etc.. But I still see few alternatives to S3, many vendors offers block devices, but only the big clouds offer blob storage. Backup and restore from blob storage makes recovery from crash pretty easy.

b2 is a really good alternative to s3.

I assume you mean: backblaze.com/b2

I would be concerned with how good the connection is... Ie. if you use packet or scaleway, how stable/fast is the connection to b2.

S3 being internal network at EC2 is a killer feature too :)

Re: Docker operations slowing down on AWS

#159

Post by flippant »

sockgrant wrote: post #116
sideproject wrote: post #70

Earlier quoted context omitted.

I've been running a web application for the past 6 months and it just crossed 150,000 page views/month mark. Sure, for others it's not that great, but for me, this is the project that is showing the biggest potential. Anyway, the funny thing is, I'm running it on $2.50/month Vultr VPS. I got so worried when it crossed 30,000 that my site will crash. But it didn't. Then when its views got higher, I optimized further.…

150,000 views a month is 150000/30/24/60/60 = 0.0578 qps. If there's no heavy processing you shouldn't need to be doing any optimization for the machine's sake at that rate. Slow queries / frontend code is a different story :)

The traffic likely peaks at certain hours

Re: Docker operations slowing down on AWS

#160

Post by geofft »

neuronexmachina wrote: post #147
geofft wrote: post #107

Earlier quoted context omitted.

How many projects actually get 150,000 hits a second? (And are they not better served by buying and operating their own datacenters?)

How many hits/second does a project have to serve when they're on the front page of HN?

Well, let's also differentiate computationally-intensive hits (e.g., users adding things to a shopping cart, requests to list Pokemon in the area, etc.) from hits to a home page, which should be static or at least cached aggressively. 150,000 index.htmls per second and 150,000 database writes per second are very different.

I used to run a university web hosting platform that currently has, I think, five web server VMs on about as many physical machines, two physical machines for load balancing, and two physical MySQL servers in active/passive replication (i.e., only one gets either reads or writes). We hit the front page of HN fairly frequently—for instance, we host mosh.org—and it hasn't really been a problem. I remember getting paged in ... 2009 or so? ... when a particular website in WordPress got to the front page of Reddit, but we had fewer machines then, and also I think we had not deployed FastCGI for PHP at that point (for complicated shared-hosting reasons), so each WordPress page load was its own PHP process via CGI. If you're optimizing for performance, even if you want to stay on WordPress, step one is to not use plain CGI and step two is to do one of the myriad things you're supposed to do for WordPress caching.

In any case, a handful of physical machines will handle being on the front page of HN just fine. If you're doing something where you have an extremely computationally-intensive process on the first page load and you're worried you might hit HN but you might not, put it on cloud and set up autoscaling, but other than that it probably doesn't make sense. If you know you won't scale too much—and a static site on the front page of HN isn't too much—chances are that your usage is so low that you're paying a premium for the unused ability to scale and you should just pay for two cheap VPSes, and if you know you will scale (e.g., you have a large fixed workload), again you're paying a premium for the unused ability to scale down, and you should just invest in a datacenter and save in the long term.

All that said, if you've got a static site, by all means stick it on a CDN, which I think is a perfectly defensible use of cloud for sites of all sizes.

Post reply on HN