Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Surprising Scalability of Multitenancy

brooker.co.za

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Surprising Scalability of Multitenancy

#21

Post by pclmulqdq »

jmillikin wrote: post #14
pclmulqdq wrote: post #13

Earlier quoted context omitted.

This isn't how things always run in the cloud. I think the conventional wisdom is that the isolation of VMs is good enough unless you are very paranoid. Auth services are regularly run on less than full baremetal machines. AWS serverless, by the way, uses VM isolation.

Both AWS and GCP offer the ability to schedule VMs onto isolated machines: https://aws.amazon.com/ec2/dedicated-hosts/ https://cloud.google.com/compute/docs/nodes/sole-tenant-node... The AWS offering is pretty much turn-key. I've not used the GCP version, but it seems to be similar if you're willing to create a separate "project" for each security domain. Once your company has any PII and/or has regulatory obligation…

Usually you have to buy the whole host when you do that, and there are many ways to buy the whole machine. I personally think baremetal is a better trade - Amazon insiders have a harder time spying on you if you do that, while they can still pause your dedicated VM to take a peek at what's going on. Regardless, I have seen authentication systems and other sensitive things run on multi-tenant machines.

Re: Surprising Scalability of Multitenancy

#22

Post by pclmulqdq »

ec109685 wrote: post #19

It’s ironic that AWS touts the benefit Lambda gets from overcommit, but if you build a lambda that simply turns around and makes an api call, you are paying full price for the cpu usage, even though it’s idle.

It doesn't matter if it's more efficient for Amazon (which serverless very much is) if they don't pass on the savings to you. Lambda is priced as a "value add" not as an efficiency improvement.

Re: Surprising Scalability of Multitenancy

#23

Post by toast0 »

andyp-kw wrote: post #7
Animats wrote: post #5

I'd seen a more useful paper on this subject, on how to organize your game servers for a big MMO. The most economical strategy was to own your servers for the base load, and go out for AWS for peaks. Running 24/7 compute bound work on AWS is at least 2x as expensive as owning your own co-located servers.

I guess the latency between AWS and your data centre would have a negative impact on game performance.

Depends on the details, but pick an AWS location near your DC. And/or pick a DC location near AWS.

Re: Surprising Scalability of Multitenancy

#24

Post by winrid »

jmillikin wrote: post #20
ec109685 wrote: post #17

Earlier quoted context omitted.

Doesn’t it depend on how many cpu cores to some extent and how much the ruby process is idle on io? Amazon doesn’t over commit cpu for normal VM instance types.

I'm not referring to overcommit or actual "cpus pegged at 100%" contention, but to simple loose bin-packing. Imagine you have three Ruby services, where each is allocated 10 cores of CPU time (via pinning with cpuset). If you give them each an 16-core VM, then there'll be 18 cores of "wasted" CPU. If you instead bin-pack them onto a 32-core VM, then they'll have the same number of cores at a lower price point. If eac…

Yeah but running them on the same machine is a pain monitoring wise, unless you're really trying to save dollars.

Re: Surprising Scalability of Multitenancy

#25

Post by ec109685 »

pclmulqdq wrote: post #22
ec109685 wrote: post #19

It’s ironic that AWS touts the benefit Lambda gets from overcommit, but if you build a lambda that simply turns around and makes an api call, you are paying full price for the cpu usage, even though it’s idle.

It doesn't matter if it's more efficient for Amazon (which serverless very much is) if they don't pass on the savings to you. Lambda is priced as a "value add" not as an efficiency improvement.

They should discount based on average cpu used.

Re: Surprising Scalability of Multitenancy

#26

Post by nvartolomei »

Animats wrote: post #5

I'd seen a more useful paper on this subject, on how to organize your game servers for a big MMO. The most economical strategy was to own your servers for the base load, and go out for AWS for peaks. Running 24/7 compute bound work on AWS is at least 2x as expensive as owning your own co-located servers.

Mind linking the said paper?

Re: Surprising Scalability of Multitenancy

#27

Post by gpderetta »

jmillikin wrote: post #20
ec109685 wrote: post #17

Earlier quoted context omitted.

Doesn’t it depend on how many cpu cores to some extent and how much the ruby process is idle on io? Amazon doesn’t over commit cpu for normal VM instance types.

I'm not referring to overcommit or actual "cpus pegged at 100%" contention, but to simple loose bin-packing. Imagine you have three Ruby services, where each is allocated 10 cores of CPU time (via pinning with cpuset). If you give them each an 16-core VM, then there'll be 18 cores of "wasted" CPU. If you instead bin-pack them onto a 32-core VM, then they'll have the same number of cores at a lower price point. If eac…

> If you're reading this and thinking "NUMA node locality", then you're operating two or three levels above where this org was in terms of optimization.

Talking about Ruby services and not hand optimized C kinds of give it away. And even with hand optimized C you would do a cost/benefit analysis of less optimal packing.

Re: Surprising Scalability of Multitenancy

#28

Post by preseinger »

throwawaylinux wrote: post #6
mjb wrote: post #4

Earlier quoted context omitted.

Time slicing significantly predates computers, and was quite well developed even when Erlang was analyzing it a century ago. What's surprising here isn't that time slicing works, it's that the same mechanism drives both the economics of large systems, and their ability to economically support bursty workloads. I can understand that may not be a surprise to you. What's surprising to me is that you took the time to com…

> Time slicing significantly predates computers, and was quite well developed even when Erlang was analyzing it a century ago. I mean its implementation in computer systems. > What's surprising here isn't that time slicing works, it's that the same mechanism drives both the economics of large systems, and their ability to economically support bursty workloads. That's not surprising. > Clearly, I shouldn't have claime…

thank you, good samaritan, for doing the tough but necessary work of disparaging this blog post, and the person who posted it, because you find the conclusions obvious

i'm sure that marc brooker, the author, and one of the most accomplished computer scientists currently living, will think twice before posting such pablum again

Re: Surprising Scalability of Multitenancy

#29

Post by revelio »

The author sounds a bit scared. Maybe the recent wave of "we can save $$$ by leaving AWS" articles have them rattled?

Yes, multi-tenancy and improved hw utilization can save money ... for Amazon. That's of no use if they lack sufficient competition and just capture the savings as profits. Then you're just wasting time on debugging weird contention issues and cloud cost optimization consultants so Bezos can get richer.

The profit margins on AWS are so huge that even though you they can binpack better it often doesn't matter, you're going to still save money by going to either a cheaper cloud or using your own HW (or renting your own dedicated HW). The savings from multi-tenancy are drowned by the added costs.

One intriguing model that might be worth exploring is micro-clouds. In that model there's a kind of clearing market, and users with strong diurnal cycles and not many batch jobs can re-sell their CPU capacity at night to other users. They just implement some Lambda-ish API and configure the kernels/hypervisors to always prioritize their own jobs over guests. The guests don't care because they're getting the resources cheap, for the company the additional income offsets the cost of their own machines and the market takes a cut. The difference vs today's cloud models is it's more decentralized and the "cloud provider" is really just a match maker, so it's easy to set up competitors and margins would be low.

Re: Surprising Scalability of Multitenancy

#30

Post by sokoloff »

ec109685 wrote: post #25
pclmulqdq wrote: post #22

Earlier quoted context omitted.

It doesn't matter if it's more efficient for Amazon (which serverless very much is) if they don't pass on the savings to you. Lambda is priced as a "value add" not as an efficiency improvement.

They should discount based on average cpu used.

You’re still consuming the RAM for the duration. In our on-prem VMWare environment, we didn’t charge, but we thought of the limited allocation being RAM far ahead of CPU ahead of disk.
Post reply on HN