Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes SidecarContainers feature is merged

github.com

Post reply on HN
Print view View original

21–30 of 65 posts

Re: Kubernetes SidecarContainers feature is merged

#21

Post by smarterclayton »

CSDude wrote: post #6

It's a shame it took so long. If the main container shutdown (i.e connection drain, processing inflight queue items) takes a while, and your service mesh dies (nice go binary) and main container cannot communicate with internet anymore. But I'm not sure about initContainers being used. init keyword implies it'd run and die in order for others to continue. Using restartPolicy with init instead of a dedicated sideCars…

We did that to leave open more complex ordering of both init containers and sidecars (regular containers do not have a restart order). For instance, you might have a service mesh that needs a vault secret - those both might be sidecars, and you may need to ensure the vault sidecar starts first if both go down. Eventually we may want to add parallelism to that start order, and a separate field would prevent simple ordering from working now.

Also, these are mostly init containers that run longer, and you want a sidecar not starting to be able to block regular pods, and adding a new container type (like ephemeral containers) is extremely disruptive to other parts of the system (security, observability, and UI), so we looked to minimize that disruption.

Re: Kubernetes SidecarContainers feature is merged

#23

Post by jeremy_k »

fnord77 wrote: post #18

> Pod is terminated even if sidecar is still running this is great for things like Jobs and Istio eliminates the scheme where the main container had to signal to the sidecar it was exiting otherwise the pod would hang

Yep, I was looking into running Jobs with Sidecars awhile back and came across this issue. I was actually surprised this morning to see a link on HN be in the "already read" state. Nice to see this feature merged, however our Cluster is on 1.25 I think? Probably a ways away from being able to use this.

Re: Kubernetes SidecarContainers feature is merged

#24

Post by jauntywundrkind »

On the one hand, great.

The other hand, one of the main criticisms of Kubernetes is that it has no composition or orchestration capabilities. It's great about defining pieces of state, but managing blocks of state & multiple things at once is left almost entirely to external tools.

The ability to compose &sequence multiple containers feels like a very specific example of a much broader general capability. There's bedevilling infinite complexity to trying to figure out a fully expressive state of state management system - I get why refining a couple specialized existing capabilities is the way - but it does make me a little sad to see a lack of appetite for the broader crosscutting system problem at the root here.

Re: Kubernetes SidecarContainers feature is merged

#25

Post by penciltwirler »

jauntywundrkind wrote: post #24

On the one hand, great. The other hand, one of the main criticisms of Kubernetes is that it has no composition or orchestration capabilities. It's great about defining pieces of state, but managing blocks of state & multiple things at once is left almost entirely to external tools. The ability to compose &sequence multiple containers feels like a very specific example of a much broader general capability. There's bed…

There's lots of tools built on top of K8s to accomplish this tho. For example, Argo, Tekton, Flyte etc.

Re: Kubernetes SidecarContainers feature is merged

#27

Post by jauntywundrkind »

penciltwirler wrote: post #25
jauntywundrkind wrote: post #24

On the one hand, great. The other hand, one of the main criticisms of Kubernetes is that it has no composition or orchestration capabilities. It's great about defining pieces of state, but managing blocks of state & multiple things at once is left almost entirely to external tools. The ability to compose &sequence multiple containers feels like a very specific example of a much broader general capability. There's bed…

There's lots of tools built on top of K8s to accomplish this tho. For example, Argo, Tekton, Flyte etc.

Absolutely, no shortage of things atop. Helm is probably the most well used composition tool.

It seems unideal to me to forever bunt on this topic, leaving it out of core forever. Especially when we are slowly adding im very specialized composition orchestration tools in core.

Re: Kubernetes SidecarContainers feature is merged

#28

Post by numbsafari »

jauntywundrkind wrote: post #27
penciltwirler wrote: post #25

Earlier quoted context omitted.

There's lots of tools built on top of K8s to accomplish this tho. For example, Argo, Tekton, Flyte etc.

Absolutely, no shortage of things atop. Helm is probably the most well used composition tool. It seems unideal to me to forever bunt on this topic, leaving it out of core forever. Especially when we are slowly adding im very specialized composition orchestration tools in core.

Helm really solves a different use case than this.

This is about describing the desired coordination among running containers. Helm is about how you template or generate your declarative state. You could certainly add this description to your templates with Helm, but you couldn't actually implement this feature with Helm itself.

Re: Kubernetes SidecarContainers feature is merged

#29

Post by numbsafari »

yla92 wrote: post #12

A very welcome change. It's gonna be helpful for the case where the database proxy (CloudSQL) and the main container got terminated out of order. https://cloud.google.com/sql/docs/postgres/connect-kubernete...

The fact that this is needed for so many different things Google pushes, and it has been so slow to make it in, has been very frustrating and telling.

Re: Kubernetes SidecarContainers feature is merged

#30

Post by NathanKP »

jauntywundrkind wrote: post #24

On the one hand, great. The other hand, one of the main criticisms of Kubernetes is that it has no composition or orchestration capabilities. It's great about defining pieces of state, but managing blocks of state & multiple things at once is left almost entirely to external tools. The ability to compose &sequence multiple containers feels like a very specific example of a much broader general capability. There's bed…

Yeah I work on the team that builds Amazon Elastic Container Service so I can't help but compare this implementation with how we solved this same problem in ECS.

Inside of an ECS task you can add multiple containers and on each container you can specify two fields: `dependsOn` and `essential`. ECS automatically manages container startup order to respect the dependencies you have specified, and on shutdown it tears things down in reverse order. Instead of having multiple container types with different hardcoded behaviors there is one container type with flexible, configurable behavior. If you want to chain together 4 or 5 containers to start up one by one in a series you can do that. If you want to run two things in parallel and then once both of them have become healthy start a third you can do that. If you want a container to run to completion and then start a second container only if the first container had a zero exit code you can do that. The dependency tree can be as complex or as simple as you want it to be: "init containers" and "sidecar containers" are just nodes on the tree like any other container.

In some places I love the Kubernetes design philosophy of more resource types, but in other aspects I prefer having fewer resource types that are just more configurable on a resource by resource basis.

Post reply on HN