Live data from Hacker News

GKE On-Prem Alpha

cloud.google.com

11–20 of 89 posts

Re: GKE On-Prem Alpha

#11
post #8

Will masters be on-prem or will the on-prem services communicate to masters at GKE over the master-authorized network? Excited to check this out, completely came out of left field.

GKE On-Prem eng here - the entire cluster will live on prem, including the masters.

Re: GKE On-Prem Alpha

#12
post #8

Will masters be on-prem or will the on-prem services communicate to masters at GKE over the master-authorized network? Excited to check this out, completely came out of left field.

Masters will run on-prem. We have connection agent that let's us securely talk to the Kube API Server from GCP. We wanted to ensure that the cluster is fully functional even if the connection goes down.

Re: GKE On-Prem Alpha

#14
post #4
post #3

> With GKE On-Prem, you get the Google Kubernetes Engine (GKE) experience directly in your data center. A quick and simple install and upgrade experience that’s validated and tested by Google. GKE On-Prem also registers your cluster with Google Cloud Console in order to have a single-pane-of-glass view for managing all your clusters. That latter is an interesting way to mentally merge your local DC and Google Cloud.

It will get even more interesting if you get decent federation support between your local GKE clusters and GCP GKE cluster allowing you to scale out to the cloud if you run out of capacity in your local cluster.

Exactly. This is really the beginning of multi-cluster scenarios that work well across different environments. Failover from on-prem -> GKE is something we're working on.

Re: GKE On-Prem Alpha

#16
post #4
post #3

> With GKE On-Prem, you get the Google Kubernetes Engine (GKE) experience directly in your data center. A quick and simple install and upgrade experience that’s validated and tested by Google. GKE On-Prem also registers your cluster with Google Cloud Console in order to have a single-pane-of-glass view for managing all your clusters. That latter is an interesting way to mentally merge your local DC and Google Cloud.

It will get even more interesting if you get decent federation support between your local GKE clusters and GCP GKE cluster allowing you to scale out to the cloud if you run out of capacity in your local cluster.

That would be great!

Depends on the latency between your DC and Google's, and the cost of traffic (which could be significant).

Re: GKE On-Prem Alpha

#17
post #2

Hey GKE team, can you give any details about how this works? What does it assume/require from the underlying infrastructure?

We're going to give a breakout session that goes into more depth on Wednesday @ 4:35pm. IO244.

Some quick details: It's a bit of a split between what GKE runs and what the customer runs. Alpha runs on vSphere 6.5 and we're packing up a Google-hardened OS in much the same way we package GKE for GCP. A lot of the integrations for things like networking and storage will be coming from partners. We'll also have remote mgmt capabilities so we can manage the cluster's control plane in much the same way our SREs do for GKE.

Re: GKE On-Prem Alpha

#18
post #9
post #6

Earlier quoted context omitted.

Exactly. As described, it sounds a bit like magic. Of course, it's not. Thus, one wonders about what happens in the lower layers of the stack, namely who gets to run them (GKE or the HW owners): * DNS (not the Kubernetes in-cluster one like kube-dns) * DHCP * LDAP or equivalent * SSH and its keys * on-prem security of the cloud identities (does it require TPM? SGX?) * bootloaders * base OS image * drivers for attache…

Not to mention probably the hardest bit, which is how does it do persistent storage? Running k8s on the various cloud providers tends to use storage engines for those providers (ebs, etc)... Does it ship with ceph out of the box? Some in-house block store? What happens when it breaks? Persistent volumes are IMO the very hardest thing to get right, and for me it's the big reason why I'd rather put my trust in a hosted…

I assumed they would punt persistent storage for the first release, kinda like they launched Cloud Filestore with only NFSv3 support and without snapshots. As you say, it's a tough problem to tackle. If they to ship with something, I'd expect them to go for Ceph. As far as I can tell, if you squint hard enough, its lowest layers are the ones that resemble the most those at Google (Colossus/D).

Re: GKE On-Prem Alpha

#20
An admission that cloud is not necessarily the best answer for all use cases.

It is smart of Google to recognize that and yet still have a product for it. It will tend to commoditize data centers as well.

Post reply on HN