Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes at GitHub

githubengineering.com

Post reply on HN
Print view View original

131–140 of 142 posts

Re: Kubernetes at GitHub

#131

Post by smarterclayton »

yebyen wrote: post #130
smarterclayton wrote: post #127

Earlier quoted context omitted.

> Maybe the chances of that happening are low, but there are enough differences that from my understanding, I should not ever expect Kubernetes projects to be directly portable to OpenShift without modification (or vice-versa.) It's true that openshift goes a lot further to disabling things that are dangerous or not ready. Ie preventing root containers, or not enabling third party resources until it went to beta. But…

The fact is, most Kubernetes projects I know are installed by Helm, and (it might have been you, personally who) explained to me that Helm is incompatible with a multitenant environment. I think they've made some strides since RBAC has gotten a little more polished, ... but please correct me if I'm wrong, OpenShift permissions model and RBAC are more compatible than I think. The last I heard, you just can't really us…

Helm isn't incompatible, it's just not currently set up for dealing with different tenants. You can use Helm in a single tenant fashion on OpenShift just like you can use it in a single tenant fashion on Kube today.

Starting with OpenShift 3.6 (on Kube 1.6) all RBAC roles between Kube and OpenShift are treated equivalently, and from OpenShift 3.7 onwards the OpenShift RBAC rules are just a compatible API shim on top of Kube RBAC. The out of the box rules on OpenShift are more restrictive simply to ensure that full multi-tenancy is possible, but they can always be lifted.

Re: Kubernetes at GitHub

#132

Post by zeeZ »

robszumski wrote: post #102
tachion wrote: post #95

Earlier quoted context omitted.

I'm currently deploying Tectonic flavoured Kubernetes at a large organisation and I can vouch for how great you guys are at supporting users (who are not yet customers) at any stage of the process - love that, and can't recommend you guys for that more. However, as the comment above says, Tectonic (and Quay for that matter) documentation is... just horrible and if not the engineers support, I'd be pretty much stuck o…

Happy to report that the Tectonic docs are open source and we would love to review your PRs: https://github.com/coreos/tectonic-installer/tree/master/Doc... Any topics that stick out as needing the most attention? Glad you're enjoying your interaction with our engineers :) (Product manager for Tectonic)

The bare-metal scaling documentation needs to be scrapped and rewritten from scratch. With very limited knowledge about terraform, it's faster to start over with nothing than trying to get terraform (apparently you need the installer bundled one) to work with your assets.zip (which is not even mentioned in the installation documentation).

Re: Kubernetes at GitHub

#133

Post by sandGorgon »

thockingoog wrote: post #119
sandGorgon wrote: post #110

Earlier quoted context omitted.

I think I hashlinked directly. I wanted to post the main page. Ingress also use nodeport, the difference is not in nodeport..But in the ingress setup itself. Docker Swarm's inbuilt ingress is now trying to build in proxy protocol and ipip mode for default usage. Fwiw, you can use Swarm's inbuilt ingress with an external load balancer as well.

The link you posted highlights my exact concern with Swarm. If I use port 8080, then nobody else can use port 8080. That is a different tradeoff than Kubernetes is willing to make. Functionally, we have this in NodePort, but because it is exclusively managed, you're very unlikely to have a meaningful conflict. BUT it depends on traffic ingress being managed. If you just want to map port 8080 on every node into your k…

swarm is not limited to working with nodeport. I think i was not able to express what i was saying correctly.

Swarm has an ingress mode that is "built-in". It works like the kuberenetes ingress (with currently the same limitation of source-ip mapping). Let me re-emphasize built-in.

Swarm also has a load balanced nodeport mapping (called host mode). That is also built-in.

In fact, what I'm trying to say is the opposite of what you are perceiving - swarm is not superior. It is actually similar to kuberenetes.

However, swarm has saner built-in defaults - both ingress and overlay networks (no more flannel vs calico vs weave incompatibility issues). Kubernetes has an escape hatch for defaults - kubeadm+kompose. I keep praying that kubeadm+kompose becomes the keras for kubernetes tensorflow. Lots of defaults that lets you get started quickly.

Re: Kubernetes at GitHub

#134

Post by yebyen »

smarterclayton wrote: post #131
yebyen wrote: post #130

Earlier quoted context omitted.

The fact is, most Kubernetes projects I know are installed by Helm, and (it might have been you, personally who) explained to me that Helm is incompatible with a multitenant environment. I think they've made some strides since RBAC has gotten a little more polished, ... but please correct me if I'm wrong, OpenShift permissions model and RBAC are more compatible than I think. The last I heard, you just can't really us…

Helm isn't incompatible, it's just not currently set up for dealing with different tenants. You can use Helm in a single tenant fashion on OpenShift just like you can use it in a single tenant fashion on Kube today. Starting with OpenShift 3.6 (on Kube 1.6) all RBAC roles between Kube and OpenShift are treated equivalently, and from OpenShift 3.7 onwards the OpenShift RBAC rules are just a compatible API shim on top…

Awesome. This makes me feel more optimistic about OpenShift, especially given that I probably can't realistically take Deis Workflow to production now.

(I don't know how much you've looked at Deis, but I couldn't think of anything better to compare it to than OpenShift. I could probably switch from Deis to OpenShift without too much hassle. Now I'm going to have to go ahead and try Deis _ON_ OpenShift, though :)

Re: Kubernetes at GitHub

#135

Post by bmicklea »

yebyen wrote: post #37
awill wrote: post #21

Earlier quoted context omitted.

You had a big announcement about openshift.io. Everyone on HN signed up, but it's been months, but I'm still 'awaiting approval' Have you let anyone in? What's the value in all the marketing hype if you then don't let people in.

Seconded. The last e-mail I got from OpenShift.io indicated they haven't let anyone in. I've already practically lost interest. It was evidently only "coming soon" but that announcement really looked like "coming tomorrow." The original OpenShift Developer Preview made you sign up, but you would be allowed into the platform within hours or days.

OpenShift.io is pre-beta but we have begun onboarding in order to get early feedback. So far ~400 people and in and we are slowly adding more as we stabilize and build out the product and underlying SaaS infrastructure.

We will be communicating our progress more openly and consistently with those on the waitlist. We really do appreciate your patience and are working hard to get people onboard.

Disclosure: I'm a PM in the Red Hat DevTools BU.

Re: Kubernetes at GitHub

#136

Post by thockingoog »

sandGorgon wrote: post #133
thockingoog wrote: post #119

Earlier quoted context omitted.

The link you posted highlights my exact concern with Swarm. If I use port 8080, then nobody else can use port 8080. That is a different tradeoff than Kubernetes is willing to make. Functionally, we have this in NodePort, but because it is exclusively managed, you're very unlikely to have a meaningful conflict. BUT it depends on traffic ingress being managed. If you just want to map port 8080 on every node into your k…

swarm is not limited to working with nodeport. I think i was not able to express what i was saying correctly. Swarm has an ingress mode that is "built-in". It works like the kuberenetes ingress (with currently the same limitation of source-ip mapping). Let me re-emphasize built-in . Swarm also has a load balanced nodeport mapping (called host mode). That is also built-in. In fact, what I'm trying to say is the opposi…

Are people happy with a built-in L7LB? I find most people are very particular about which LB they use.

Re: Kubernetes at GitHub

#137

Post by yebyen »

bmicklea wrote: post #135
yebyen wrote: post #37

Earlier quoted context omitted.

Seconded. The last e-mail I got from OpenShift.io indicated they haven't let anyone in. I've already practically lost interest. It was evidently only "coming soon" but that announcement really looked like "coming tomorrow." The original OpenShift Developer Preview made you sign up, but you would be allowed into the platform within hours or days.

OpenShift.io is pre-beta but we have begun onboarding in order to get early feedback. So far ~400 people and in and we are slowly adding more as we stabilize and build out the product and underlying SaaS infrastructure. We will be communicating our progress more openly and consistently with those on the waitlist. We really do appreciate your patience and are working hard to get people onboard. Disclosure: I'm a PM in…

I don't mean to spout off in a public forum, don't think I don't understand how much work is involved in putting out something that we're not going to be disappointed with!

But I have been interested in OpenShift, and it is a good sign for me that so many people who obviously are involved in OpenShift are representing on this thread.

I will definitely give Openshift another look soon. We're building a group of experienced devs to help onboarding our new dev employees as they come on, and we're going to have some standard-setting capacity when it comes to showing off the tools we use.

(We are currently well behind in the K8S space imho, but getting better and I think making a more serious push for it is going to come soon. Our institution is large and like all large institutions Byzantine bureaucracy results in some extreme levels of inertia; well we haven't really dove into containers near the level I had at my last position, which was still only about token level – most core infrastructure still not containerized or even touching any containers.)

So, blank canvas!

I'm very much hoping to have an array of tools that I can share with new devs, so they can organically decide what works best for their own selves. I had kind of discounted oc based on not having used it much myself, but I do remember having some joy at reading the documentation that it was more centrally located and easier to "grok" as well as feeling more complete.

Anyway both things that will be super helpful to anyone who is new at Kubernetes or Dev stuff in general. There is just so much to learn and the ecosystem is ofc constantly evolving!

I will keep openshift origin in the toolbox and take another look at pro offering.

Thanks for introducing yourself to me!

Re: Kubernetes at GitHub

#138

Post by spmurrayzzz »

ninkendo wrote: post #64
spmurrayzzz wrote: post #57

Earlier quoted context omitted.

They could easily still use standalone puppet to handle the config management for individual container images. I currently do this with salt-minion. It reduces the burden on the Dockerfile itself, and lets you embrace a declarative configuration state at build time.

It definitely seems like the wrong approach to me to have puppet manage your base images. They're not VM's, they shouldn't have multiple services, they shouldn't require any complex configuration management, they should just be the minimum requirements to support your application's local runtime dependencies, and that's it. From previous experience migrating from a puppet setup to one that used containers, puppet's v…

1) I think you missed my point entirely here, I probably didn't do a good job of explaining it. I was trying to say that you run Puppet once at build time to bootstrap the configuration for the image, that's it. You could even uninstall it at the last build step if you want to reduce final image size. The primary distinction here is declarative vs imperative configuration management.

2) The one process-per-container dogma isn't necessarily the only way to run a successful docker stack. For example, I don't see anything wrong with using supervisor to manage whatever process you're running in your container.

Re: Kubernetes at GitHub

#139

Post by spmurrayzzz »

dkh99 wrote: post #98
ninkendo wrote: post #64

Earlier quoted context omitted.

It definitely seems like the wrong approach to me to have puppet manage your base images. They're not VM's, they shouldn't have multiple services, they shouldn't require any complex configuration management, they should just be the minimum requirements to support your application's local runtime dependencies, and that's it. From previous experience migrating from a puppet setup to one that used containers, puppet's v…

There's nothing inherently about Puppet that means it has to manage multi-service "whole OS"-like installations. It can just as easily be put to the task of a Dockerfile: install dependencies and deployables for a single application. Its robust ability to manage things like user accounts, packages, scheduled jobs (e.g. for alerting, though you would have to install at least a second service for this: _crond) and the…

The point about layer complexity is a great one I didn't even consider. Your "config" step is no longer a mish mash of dozens COPY/RUN/etc directives (resulting in N new intermediate image layers), it just results in a single atomic layer where you run the Puppet bootstrap.

Obviously you could accomplish this with shell scripts as well to constrain your config step into one docker RUN directive, but I prefer the declarative state approach to the imperative one in this case.

Re: Kubernetes at GitHub

#140

Post by yebyen »

yebyen wrote: post #97
xkarga00 wrote: post #92

Earlier quoted context omitted.

> Maybe OpenShift, but it is not "really K8S" Can you elaborate more? What makes you think that this is true?

It has incompatible resource types. They built OpenShift to handle authorization and permissions before Kubernetes RBAC was fully baked. So there are OpenShift solutions that don't exist on K8S, and vice-versa. The BuildConfig and ImageStream for starters. It's not a substantive difference that makes OpenShift much harder to learn, but it is a difference that means "if RedHat decides to 'Deis' OpenShift," we're stuck…

Hmm. It looks like[1] I'm wrong about one more thing, that being where you can run OpenShift. The current installation docs describe a single all-in-one binary that you can use to run OpenShift origin on any current Linux kernel, or Mac:

[1]: https://docs.openshift.org/latest/getting_started/administra...

I don't know if you need to have Docker for Mac installed, but I would guess you don't (it would be crazy to try to interface with arbitrary versions of Docker, it probably runs its own docker inside of a virtualized layer.)

I'm going to have to look at this again in some more depth. Looks better than when I saw it last time! (That's to be expected, I guess, but again it is encouraging!)

Post reply on HN