Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Adds support for nested virtualization

github.com

Post reply on HN
Print view View original

111–120 of 124 posts

Re: AWS Adds support for nested virtualization

#111

Post by laurencerowe »

otterley wrote: post #107
laurencerowe wrote: post #106

Earlier quoted context omitted.

Nested virtualisation with KVM works on the Linux GitHub Actions runners which I believe run on Azure.

GitHub says: > While nested virtualization is technically possible while using runners, it is not officially supported. Any use of nested VMs is experimental and done at your own risk, we offer no guarantees regarding stability, performance, or compatibility. https://docs.github.com/en/actions/concepts/runners/github-h...

It seems to work for my https://github.com/libriscv/kvmserver tests at least.

Re: AWS Adds support for nested virtualization

#112

Post by PunchyHamster »

blibble wrote: post #103
PunchyHamster wrote: post #70

Earlier quoted context omitted.

All that sounds like it would better be a contribution to KVM from the get go rather than invent stuff that eventually showed up in KVM anyway

it's been in kvm since the mid 10s and in Xen (which they used to run) for at least as long

I meant in general, as the linked presentation talked about many features, not just nested virt

Re: AWS Adds support for nested virtualization

#113

Post by PunchyHamster »

pezgrande wrote: post #87
PunchyHamster wrote: post #68

Earlier quoted context omitted.

It is expensive. But the point where it stops being expensive is far above most companies use case. If you're paying less than a developers salary for hosting you most likely won't see all that many benefits from moving. Renting a server from cheaper hosting providers can be massive savings but you now need to re-invent all of the AWS APIs you use or might use and it's big CAPEX time investment. And any new feature y…

System admins are probably cheaper that Cloud experts devops.

Not at scale to run your own bunch of servers competently.

Re: AWS Adds support for nested virtualization

#114

Post by PunchyHamster »

krab wrote: post #67
PunchyHamster wrote: post #66

Earlier quoted context omitted.

> Bare-metal on Hetzner now has a setup fee of up to 350 EUR.. you can find some stuff with 0 setup fee, but it's usually quite old kit. I don't understand what you are paying for here, nested virtualization doesn't need any extra setup for hardware compared to normal one ... or you are saying Hetzner wants 350 EUR for turning on normal virtualization option in BIOS ?

Hetzner charges a fee for setting up your bare-metal machine. Often zero for their smaller machines and for those in auction. Probably they don't want someone to order a large fleet large of machines for one month and then cancel. They might not get another customer for those machines soon.

...but servers come with virtualization on by default for like... at least a decade if not more

So they literally want money to fix what they fucked up the first time

Re: AWS Adds support for nested virtualization

#115

Post by PunchyHamster »

alexellisuk wrote: post #83
PunchyHamster wrote: post #66

Earlier quoted context omitted.

> Bare-metal on Hetzner now has a setup fee of up to 350 EUR.. you can find some stuff with 0 setup fee, but it's usually quite old kit. I don't understand what you are paying for here, nested virtualization doesn't need any extra setup for hardware compared to normal one ... or you are saying Hetzner wants 350 EUR for turning on normal virtualization option in BIOS ?

They used to charge a fair admin fee like 30-70 EUR for most bare-metal hosts.. now it's 99 EUR for the most basic/cheapest option.. up to 350 EUR for something modest like a 16 Core Ryzen.. monthly fees haven't changed much. https://www.hetzner.com/dedicated-rootserver/matrix-ex/ https://www.hetzner.com/dedicated-rootserver/matrix-ax/

Feels weird to roll it in into setup fee vs monthly price

Re: AWS Adds support for nested virtualization

#116

Post by BobbyTables2 »

matheus-rr wrote: post #65
BobbyTables2 wrote: post #40

Is nested VMX virtualization in the Linux kernel really that stable? The technical details are a lot more complex than most realize. Single level VMX virtualization is relatively straightforward even if there are a lot of details to juggle with VMCS setup and handing exits. Nested virtualization is a whole another animal as one now also has to handle not just the levels but many things the hardware normally does, plu…

Fair concern, but this has been quietly production-stable on GCP and Azure since 2017 — that's 8+ years at cloud scale. The LKML debates you're referencing are mostly about edge cases in exotic VMX features (nested APIC virtualization, SGX passthrough), not the core nesting path that workloads like Firecracker and Kata actually exercise. The more interesting signal is that AWS is restricting this to 8th-gen Intel ins…

Curiously 8th Intel is also about the minimum for Windows 11… (which can virtualize most of the kernel)

Re: AWS Adds support for nested virtualization

#117

Post by zxspectrum1982 »

ohthehugemanate wrote: post #34

I wonder if this is connected to Azure launching OpenShift Virtualization on "Boost" SKUs? There are a lot of VMWare customers going to OpenShift Virt, and apparently the CPU/memory overhead on Azure maxes out around 10% under full load... but then hyper V has been doing a lot of work on it. No idea if nitro includes any of the KVM-on-KVM passthrough of full KVM, to give it an edge here.

OpenShift Virtualization on AWS, even as a managed service ("ROSA Virtualization"), has been available for a while on bare metal. Theoretically this enables ROSA Virtualization on EC2, in case you had valid reasons for such a thing.

Re: AWS Adds support for nested virtualization

#118

Post by drizmans »

sidewndr46 wrote: post #110
lelandbatey wrote: post #100

Earlier quoted context omitted.

Hetzner offers uniquely cheap dedicated hosting, even beating OVH. Per their statement about the fees, they're having to do this because without the setup fees, recent hardware prices increases would otherwise raise the price of acquiring new hardware so high that they would essentially never make a profit on the hardware they would have to buy for new orders. They're also saying that their overall prices are going t…

That seems counter to a "pay as you go" or "pay what you use for" model. I'd rather have sky high monthly fees, so that I don't have a sunk cost.

Bare metal has never been a pay as you go model, its so much cheaper you usually over provision by a factor of 10-100, and still spend less than you would on the cloud if you have moderate needs. You are trading ops tax for money tax.

Re: AWS Adds support for nested virtualization

#119

Post by drizmans »

stoneforger wrote: post #78
re-thc wrote: post #71

Earlier quoted context omitted.

> but you now need to re-invent all of the AWS APIs you use or might use and it's big CAPEX time investment Or maybe you just never needed most of these in the first place. People got into this "AWS" mentality like it is the only way to do things. Everything had to be in a queue, event driven etc. I'd argue not using AWS means simplifying things and it'll be less expensive not just in server cost but developer time.

You don't get how this works. You buy in AWS because everyone else is , so it's expected. It diffuses risk to your stock options. This also begets a whole generation of people who can only use cloud services so now you are more hard pressed to find people with experience to run things without the cloud. You also create a bigger expenses sheet so it shows you're investing and growing, attracting more investors. "We pa…

[dead]
Post reply on HN