Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Kubernetes Engine adds support for Arm nodes

cloud.google.com

Post reply on HN
Print view View original

41–50 of 51 posts

Re: Google Kubernetes Engine adds support for Arm nodes

#41

Post by sigmaml »

runlevel1 wrote: post #34
jonatron wrote: post #16

Earlier quoted context omitted.

I always found the CPU pricing of cloud to be relatively reasonable. It's everything else that's expensive, and egress bandwidth just ridiculous.

I've done some cost analyses between our AWS and DC infrastructure. To come up with our on-prem compute costs, we baked in the cost of power, real estate, staff, taxes, network infrastructure, servers (both in-use and in reserve), etc. On the AWS side, we used 3 year RIs and Savings Plan. After all that, there was around a 30% cost advantage on-prem. That's non-trivial, but not as big as one might think. Outbound net…

A quick question.

Have you also included:

  - storage costs (equivalent of EBS, S3 and Glacier) and
  - cost of analytics pipelines (equivalent of EMR, Athena, SageMaker, ...)
in the above price comparison?

Would you have some insights there? Thanks.

Re: Google Kubernetes Engine adds support for Arm nodes

#42

Post by dilyevsky »

bogomipz wrote: post #30
vocram wrote: post #25

Earlier quoted context omitted.

The instance exposed by GCE is virtualized. If you want to run any hw virtualized workload inside it, you need nested virtualization.

I'd be curious to hear more about your Kubernetes workloads. What virtualized hardware do your pods require?

Any untrusted workloads (say CI runners running your clients arbitrary code) better be run inside kata containers so you can’t use t2a vms for that

Re: Google Kubernetes Engine adds support for Arm nodes

#43

Post by ksec »

BillinghamJ wrote: post #28
milesward wrote: post #27

Earlier quoted context omitted.

Where did you want it?

In every zone, obviously. AWS has had widespread ARM support for many years.

I mean first gen Graviton was announced in late 2018 during reInvent. And it wasn’t even generally available until mid 2019. It was underwhelming, but layering some required software foundation. Graviton 2 was in late 2019 and wasn’t GA and available in many region until 2021.

Hardly widespread support for many years.

Re: Google Kubernetes Engine adds support for Arm nodes

#44

Post by cagataygurturk »

jrockway wrote: post #32
cagataygurturk wrote: post #21

Earlier quoted context omitted.

You can already self-host GH Runners on ARM.

What kind of ARM machine should I buy if I want to self-host an ARM server? My not-that-recent look at the market shows everyone kind of doing their own thing; Amazon makes Amazon's ARM servers, Apple makes Apple's ARM chips, etc. As some random guy who wants to test on ARM, it's annoying. Maybe things have improved recently, though? (I'm guessing self-host realistically means "get a VM on AWS", which is probably fin…

The ARM nodes which are exactly the topic of this submission could be a place where you can self-host your GH runners :)

https://github.com/actions-runner-controller/actions-runner-... works quite well with its autoscaling. You could create a zonal GKE cluster, which is in free tier, and create a small spot vm node pool with ARM nodes. It wouldn't be entirely free but it would cost quite low amount of money.

Re: Google Kubernetes Engine adds support for Arm nodes

#45

Post by Klasiaster »

FooBarWidget wrote: post #22
cagataygurturk wrote: post #21

Earlier quoted context omitted.

You can already self-host GH Runners on ARM.

The point is to not have to self-host. It costs money, is a pain because maintenance is required, and there are security issues with running CI for public pull requests.

With garm (https://github.com/cloudbase/garm) you can spawn ephemeral runner VMs, be it as Azure/GCE ARM instances or as lxc VMs (or lxc containers).

Re: Google Kubernetes Engine adds support for Arm nodes

#46

Post by bogomipz »

dilyevsky wrote: post #42
bogomipz wrote: post #30

Earlier quoted context omitted.

I'd be curious to hear more about your Kubernetes workloads. What virtualized hardware do your pods require?

Any untrusted workloads (say CI runners running your clients arbitrary code) better be run inside kata containers so you can’t use t2a vms for that

In GKE you can just enable GKE Sandbox/gVisor on a node pool to run your untrusted workloads. gVisor serves the same purpose as Kata containers.

Re: Google Kubernetes Engine adds support for Arm nodes

#47

Post by dilyevsky »

bogomipz wrote: post #46
dilyevsky wrote: post #42

Earlier quoted context omitted.

Any untrusted workloads (say CI runners running your clients arbitrary code) better be run inside kata containers so you can’t use t2a vms for that

In GKE you can just enable GKE Sandbox/gVisor on a node pool to run your untrusted workloads. gVisor serves the same purpose as Kata containers.

Yes except slow io

Re: Google Kubernetes Engine adds support for Arm nodes

#48

Post by bogomipz »

dilyevsky wrote: post #47
bogomipz wrote: post #46

Earlier quoted context omitted.

In GKE you can just enable GKE Sandbox/gVisor on a node pool to run your untrusted workloads. gVisor serves the same purpose as Kata containers.

Yes except slow io

Can you elaborate? What type of I/O, network, disk? What is the issue exactly?

Re: Google Kubernetes Engine adds support for Arm nodes

#49

Post by dilyevsky »

bogomipz wrote: post #48
dilyevsky wrote: post #47

Earlier quoted context omitted.

Yes except slow io

Can you elaborate? What type of I/O, network, disk? What is the issue exactly?

You can refer to gvisor performance docs - https://gvisor.dev/docs/architecture_guide/performance/#file... throughput is really terrible, same deal with networking and also if your userland issues a lot of syscalls

Re: Google Kubernetes Engine adds support for Arm nodes

#50

Post by bogomipz »

dilyevsky wrote: post #49
bogomipz wrote: post #48

Earlier quoted context omitted.

Can you elaborate? What type of I/O, network, disk? What is the issue exactly?

You can refer to gvisor performance docs - https://gvisor.dev/docs/architecture_guide/performance/#file... throughput is really terrible, same deal with networking and also if your userland issues a lot of syscalls

Thanks for the link. I'm curious, how is the I/O performance with Kata? Does it use VirtIO?
Post reply on HN