61–70 of 135 posts
Earlier quoted context omitted.
It seems to me, not to be combative, that if Fargate can't afford the "noschedule: node is old" overhead and customers of Fargate can't handle their containers restarting on a regular basis, there's something wrong with your management engine or with their design and implementation. Much of the point of containerization is that you can roll containers often and run enough of them that you never have a single point of…
There are any number of reasons to avoid restarting things. Some customers are running code that has a cold start and needs some time to warm up its cache if it restarts. Some customers are running jobs (video rendering, machine learning training, etc) that might take literally days to complete. Interrupting these jobs and causing them to restart wastes the customer time and causes them to lose progress. Other contai…
I'm confused about how the documentation recommends using a Kubernetes operator to manage OS updates. That seems weird and backwards to me. I would rather see an immutable OS AMI in an auto-scaled group, and just replace the node instance whenever there is an update. I can see a place for managing OS updates on an instance, but that seems more like "pets" than "cattle"... and I've always treated Kubernetes nodes like…
This is how OpenShift 4 does things. I too thought it was strange at first but now with some experience it's quite pleasant. Can be a beast to debug though if you haven't done it before.
I once worked on a competing product (before the OS update operator was available) and the update-in-place model was always a disaster. Various problems like dns, service discovery, timeouts, breaking changes to dependency pkgs, etc make for a problematic process. Combine that with the frantic pace of k8s develeopment, short node compatibility window (2-3 minor k8s releases) and various CVEs - you end up debugging a lot of machines in unknown states that fail to rejoin clusters after reboots.
I'm confused about how the documentation recommends using a Kubernetes operator to manage OS updates. That seems weird and backwards to me. I would rather see an immutable OS AMI in an auto-scaled group, and just replace the node instance whenever there is an update. I can see a place for managing OS updates on an instance, but that seems more like "pets" than "cattle"... and I've always treated Kubernetes nodes like…
Earlier quoted context omitted.
This is how OpenShift 4 does things. I too thought it was strange at first but now with some experience it's quite pleasant. Can be a beast to debug though if you haven't done it before.
I can assure you that OpenShift doesn't take this path because it is "better". It does so because bare-metal is a significant part of their market and there isn't a better option to automate the process currently. I once worked on a competing product (before the OS update operator was available) and the update-in-place model was always a disaster. Various problems like dns, service discovery, timeouts, breaking chang…
So far, aside from a few small flakey issues, having the cluster nodes _and_ the OpenShift cluster update in lock step has been dramatically simpler to manage.
Firecracker, Bottlerocket, starting to see a trend here https://aws.amazon.com/blogs/aws/firecracker-lightweight-vir...
Earlier quoted context omitted.
There are any number of reasons to avoid restarting things. Some customers are running code that has a cold start and needs some time to warm up its cache if it restarts. Some customers are running jobs (video rendering, machine learning training, etc) that might take literally days to complete. Interrupting these jobs and causing them to restart wastes the customer time and causes them to lose progress. Other contai…
This stuff is probably waaaay over my head, but isn't that why SIGTERM was made for ? To notify a running process that the host needs to be shutdown/restarted and to let the running process finish it's current task (current frame encoding / current multiplayer game / current request / ...) and that the state / cache / progress / ... needs to be saved. The process on aws side would then be : send SIGTERM to all worklo…
What is the better option here? Implement a SIGTERM based process that allows the user to block the patch for a critical, possibly zero-day CVE for xx hours, remaining in a vulnerable state the entire time? Or implement a system that just patches the underlying host without interrupting the workloads on the box?
You aren't wrong, what you described is a possibility, but it is not the best possibility.
I'm confused about how the documentation recommends using a Kubernetes operator to manage OS updates. That seems weird and backwards to me. I would rather see an immutable OS AMI in an auto-scaled group, and just replace the node instance whenever there is an update. I can see a place for managing OS updates on an instance, but that seems more like "pets" than "cattle"... and I've always treated Kubernetes nodes like…
I'm confused about how the documentation recommends using a Kubernetes operator to manage OS updates. That seems weird and backwards to me. I would rather see an immutable OS AMI in an auto-scaled group, and just replace the node instance whenever there is an update. I can see a place for managing OS updates on an instance, but that seems more like "pets" than "cattle"... and I've always treated Kubernetes nodes like…
It sounds like you're trying hard to reinvent Kubernetes while doing your best to avoid mentioning Kubernetes features like Kubernetes operators.