Live data from Hacker News

AWS Adds support for nested virtualization

github.com

21–30 of 124 posts

Re: AWS Adds support for nested virtualization

#21
post #13
post #11

This is a big deal because you can now run Firecracker/other microVMs in an AWS VM instead of expensive AWS bare-metal instances. GCP has had nested virtualization for a while.

whats the ~ perf hit of something like this?

As a practical matter, anywhere from 5-15%.

Re: AWS Adds support for nested virtualization

#26
post #9

What's the performance impact for nested virtualization in general? I'd think this would be adding multiple layers of MMU overhead.

depends on the workload and how they've done it

pure CPU should be essentially unaffected, if they're not emulating the MMU/page tables in software

the difference in IO ranges from barely measurable to absolutely horrible, depending on their implementation

traps/vmexits have another layer to pass through (and back)

Re: AWS Adds support for nested virtualization

#27
post #12

Earlier quoted context omitted.

Yep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.

I was an Amazon EC2 Specialist SA in a prior role, so I know a little about this. If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. On bare metal, you've always been free to run your own VMs, and we had some customers who rolled their own nested VM implementations on it. But EC2 is not like your home server. There are some nontrivial considerat…

I always enjoy the color you add to these conversations. Thanks!

Re: AWS Adds support for nested virtualization

#28
post #12

Earlier quoted context omitted.

Yep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.

I was an Amazon EC2 Specialist SA in a prior role, so I know a little about this. If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. On bare metal, you've always been free to run your own VMs, and we had some customers who rolled their own nested VM implementations on it. But EC2 is not like your home server. There are some nontrivial considerat…

Nitro is very interesting stuff

Re: AWS Adds support for nested virtualization

#29
I feel vindicated :). We put in a lot of effort with great customers to get nested virtualization running well on GCE years ago, and I'm glad to hear AWS is coming around.

You can tell people to just do something else, there's probably a separate natural solution, etc. but sometimes you're willing to sacrifice some peak performance just have that uniformity of operations and control.

Re: AWS Adds support for nested virtualization

#30
post #12

Earlier quoted context omitted.

Yep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.

I was an Amazon EC2 Specialist SA in a prior role, so I know a little about this. If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. On bare metal, you've always been free to run your own VMs, and we had some customers who rolled their own nested VM implementations on it. But EC2 is not like your home server. There are some nontrivial considerat…

Seriously curious, don’t Firecracker VMs already run on EC2 instances under the hood when they host Lambda and Fargate?
Post reply on HN