Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GKE On-Prem Alpha

cloud.google.com

Post reply on HN
Print view View original

21–30 of 89 posts

Re: GKE On-Prem Alpha

#21

Post by merb »

ninkendo wrote: post #9
puzzle wrote: post #6

Earlier quoted context omitted.

Exactly. As described, it sounds a bit like magic. Of course, it's not. Thus, one wonders about what happens in the lower layers of the stack, namely who gets to run them (GKE or the HW owners): * DNS (not the Kubernetes in-cluster one like kube-dns) * DHCP * LDAP or equivalent * SSH and its keys * on-prem security of the cloud identities (does it require TPM? SGX?) * bootloaders * base OS image * drivers for attache…

Not to mention probably the hardest bit, which is how does it do persistent storage? Running k8s on the various cloud providers tends to use storage engines for those providers (ebs, etc)... Does it ship with ceph out of the box? Some in-house block store? What happens when it breaks? Persistent volumes are IMO the very hardest thing to get right, and for me it's the big reason why I'd rather put my trust in a hosted…

> Not to mention probably the hardest bit, which is how does it do persistent storage?

on bare-metal this is solved with rook.io. load balancing (not the api servers) is also solved with metallb.

Re: GKE On-Prem Alpha

#22

Post by joshwget »

westonh wrote: post #10

Hey Weston here from the GKE On-Prem team. Wifi is spotty at the conference but I'll try to answer questions as they come in.

Will Container-Optimized OS be used as the operating system for on-prem? If so, any plans to spin this off as a more general purpose OS now that it needs to support on-prem use cases?

Re: GKE On-Prem Alpha

#23

Post by sofaofthedamned »

This is awesome! One of my problems with showing clients the ability of GKE is having to pay a lot of money to demonstrate.

How complete is this? Can I do the usual ingress/LB annotations for GKE and apply them to an on-prem instance?

Re: GKE On-Prem Alpha

#24

Post by ninkendo »

merb wrote: post #21
ninkendo wrote: post #9

Earlier quoted context omitted.

Not to mention probably the hardest bit, which is how does it do persistent storage? Running k8s on the various cloud providers tends to use storage engines for those providers (ebs, etc)... Does it ship with ceph out of the box? Some in-house block store? What happens when it breaks? Persistent volumes are IMO the very hardest thing to get right, and for me it's the big reason why I'd rather put my trust in a hosted…

> Not to mention probably the hardest bit, which is how does it do persistent storage? on bare-metal this is solved with rook.io. load balancing (not the api servers) is also solved with metallb.

> on bare-metal this is solved with rook.io

Do you have a source for this? Is there documentation anywhere that says GKE On-Prem is using rook? Or are you just saying "people who use kubernetes on-prem often use rook.io"?

Re: GKE On-Prem Alpha

#26

Post by ferrantim »

ninkendo wrote: post #9
puzzle wrote: post #6

Earlier quoted context omitted.

Exactly. As described, it sounds a bit like magic. Of course, it's not. Thus, one wonders about what happens in the lower layers of the stack, namely who gets to run them (GKE or the HW owners): * DNS (not the Kubernetes in-cluster one like kube-dns) * DHCP * LDAP or equivalent * SSH and its keys * on-prem security of the cloud identities (does it require TPM? SGX?) * bootloaders * base OS image * drivers for attache…

Not to mention probably the hardest bit, which is how does it do persistent storage? Running k8s on the various cloud providers tends to use storage engines for those providers (ebs, etc)... Does it ship with ceph out of the box? Some in-house block store? What happens when it breaks? Persistent volumes are IMO the very hardest thing to get right, and for me it's the big reason why I'd rather put my trust in a hosted…

I can attest that persistent storage is the hard part! Full disclosure, I work for a company[1] who makes a persistent storage solution for containers/Kubernetes. We are absolutely seeing that our large customers (folks like GE, Verizon, Dreamworks, Comcast, etc) are running "cloud native" applications on-prem as well as in the public cloud so this is a really smart move for Google.

[1] https://portworx.com

Re: GKE On-Prem Alpha

#27

Post by wmf »

sofaofthedamned wrote: post #23

This is awesome! One of my problems with showing clients the ability of GKE is having to pay a lot of money to demonstrate. How complete is this? Can I do the usual ingress/LB annotations for GKE and apply them to an on-prem instance?

Since this runs on vSphere presumably it will cost far far more than public GKE.

Re: GKE On-Prem Alpha

#28

Post by chumboslice »

nine_k wrote: post #16
eicnix wrote: post #4

Earlier quoted context omitted.

It will get even more interesting if you get decent federation support between your local GKE clusters and GCP GKE cluster allowing you to scale out to the cloud if you run out of capacity in your local cluster.

That would be great! Depends on the latency between your DC and Google's, and the cost of traffic (which could be significant).

Which is where Google's network comes into play.
Post reply on HN