Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Controllers for Kubernetes

github.com

Post reply on HN
Print view View original

31–40 of 54 posts

Re: AWS Controllers for Kubernetes

#31

Post by scarface74 »

takeda wrote: post #26
sytse wrote: post #3

Kelsey Hightower had a good take in https://twitter.com/kelseyhightower/status/12963119515307048... "AWS Controllers for Kubernetes is pretty dope. You can leverage Kubernetes to manage AWS resources such as API gateways and S3 buckets. Think Terraform but backed by Kubernetes style APIs and "realtime" control loops." An in the thread he mentions Crossplane as the cross-cloud way to do this https://twitter.com/kelsey…

I thought whole point of running K8S in AWS was to have infrastructure not tied to their offerings. If you use something like that, why use K8S and not just use AWS services natively?

The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale. I have a feeling that anyone who thinks it is easy or even usually worth it has never been part of planning a large scale migration.

At the same time, you usually end up spending more money and having worse results when you don’t go all in.

As far as why use EKS vs ECS - the “native service”? They seem to have feature parity, ECS is easier to use for the unitiated. But, there are so many people who know k8s and your knowledge is portable.

Which brings up my second point. Most software engineers don’t care about cloud mobility as much as they claim. They care about career mobility. There is a much better chance that you will leave a company and move to a company on a different provider than your company will. I’m not saying it’s a bad thing to focus on technologies that give you as an individual the most optionality.

Re: AWS Controllers for Kubernetes

#32

Post by scarface74 »

cactus2093 wrote: post #20
mfer wrote: post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

I think you have mostly the same problems in Terraform, Pulumi, or Cloudformation though right? Is there anything that makes it easier to accidentally do in k8s? One layer of defense in all of these cases is keeping the IAM credentials that the configuration management tool uses from having any deletion permissions.

That’s an excellent idea. I never thought about that. In hindsight, it makes perfect sense.

Also, set the DeletionPolicy in CF to false.

Re: AWS Controllers for Kubernetes

#33

Post by robszumski »

dilyevsky wrote: post #29
jrockway wrote: post #19

Earlier quoted context omitted.

Deletion is typically specified by an immutable finalizer on the resource. For example, you can create a StorageClass that saves your PersistentVolume when you delete the PersistentVolumeClaim, and you can't change the class of a claim, so even if your templating goes haywire and deletes your volume claim, the actual data is safe and can be recovered. I have to imagine that any API that lets you create things like RD…

Finalizers dont do much for safety. They are simply there to ensure controller (in this case ACK) won’t miss the deletion event and leave the resource dangling. To actually prevent object from being deleted you need a validation webhook

An interesting part of the Operator Lifecycle Manager (OLM) is the capability to scaffold webhooks for Operators, rotate their secrets, etc to make this type protection easy for everyone to provide. All you need to do is bring your validation logic specific to the app.

https://olm.operatorframework.io/docs/advanced-tasks/adding-...

Re: AWS Controllers for Kubernetes

#35

Post by harpratap »

takeda wrote: post #26
sytse wrote: post #3

Kelsey Hightower had a good take in https://twitter.com/kelseyhightower/status/12963119515307048... "AWS Controllers for Kubernetes is pretty dope. You can leverage Kubernetes to manage AWS resources such as API gateways and S3 buckets. Think Terraform but backed by Kubernetes style APIs and "realtime" control loops." An in the thread he mentions Crossplane as the cross-cloud way to do this https://twitter.com/kelsey…

I thought whole point of running K8S in AWS was to have infrastructure not tied to their offerings. If you use something like that, why use K8S and not just use AWS services natively?

This is just one small step in that direction. Once everyone is on the same page about Infrastucture-as-Data and behaves in a consistent and expected manner, we can move forward with wrapping these interface. Crossplane is a one such project.

Re: AWS Controllers for Kubernetes

#36

Post by harpratap »

MuffinFlavored wrote: post #23

How is this different than Terraform?

Traditional Infrastructure-as-Code solutions were based on edge-triggers. Like create ingress, delete ingress. But what about when ingress misbehaves or is in an unrecoverable state?

Kubernetes introduced edge-triggered level-driven with resync reconciliation based "controllers". User defines a state and the controller does it's best to keep the infra in this desired state all the times. (Although Terraform has also moved to this same design in recent times)

This establishes a consistent experience. Everyone knows you just need to do kubectl get my-resource to check your desired state. All the issues will be logged in status and controller. You can combine multiple controllers to achieve your desired application design. For example, Knative has their own kind called "Service" which has some custom components, some inherited from istio and things like replicasets from default kubernetes controllers.

Re: AWS Controllers for Kubernetes

#37

Post by dilyevsky »

scarface74 wrote: post #31
takeda wrote: post #26

Earlier quoted context omitted.

I thought whole point of running K8S in AWS was to have infrastructure not tied to their offerings. If you use something like that, why use K8S and not just use AWS services natively?

The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale. I have a feeling that anyone who thinks it is easy or even usually worth it has never been part of planning a large scale migration. At the same time, you usually end up spending more money and having worse results when you don’t go all in. As far as why use EKS vs ECS - the “native service”? T…

> The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale

Pretty categorical statement and I don’t find this to be true at all if you avoid using anything managed except block devices and vms themselves. That is unless “any type or scale” is tens of thousands of VMs and petabytes of storage in which case it is indeed a moot endeavor since someone who is still on public cloud at that point apparently likes to give all their money to cloud vendors anyway.

Re: AWS Controllers for Kubernetes

#39

Post by scarface74 »

dilyevsky wrote: post #37
scarface74 wrote: post #31

Earlier quoted context omitted.

The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale. I have a feeling that anyone who thinks it is easy or even usually worth it has never been part of planning a large scale migration. At the same time, you usually end up spending more money and having worse results when you don’t go all in. As far as why use EKS vs ECS - the “native service”? T…

> The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale Pretty categorical statement and I don’t find this to be true at all if you avoid using anything managed except block devices and vms themselves. That is unless “any type or scale” is tens of thousands of VMs and petabytes of storage in which case it is indeed a moot endeavor since someone who…

If all you’re doing is using a cloud provider to host a bunch of VMs, you’ve already lost the plot. Once you use any cloud provider as a glorified colo without changing any of your processes you’re already spending more than a colo and not getting any of the benefits of managed services.

Have you ever budgeted a project plan involving a large migration.

Re: AWS Controllers for Kubernetes

#40

Post by dilyevsky »

scarface74 wrote: post #39
dilyevsky wrote: post #37

Earlier quoted context omitted.

> The entire idea of your infrastructure not being tied to your provider is completely lost when you are at any type of scale Pretty categorical statement and I don’t find this to be true at all if you avoid using anything managed except block devices and vms themselves. That is unless “any type or scale” is tens of thousands of VMs and petabytes of storage in which case it is indeed a moot endeavor since someone who…

If all you’re doing is using a cloud provider to host a bunch of VMs, you’ve already lost the plot. Once you use any cloud provider as a glorified colo without changing any of your processes you’re already spending more than a colo and not getting any of the benefits of managed services. Have you ever budgeted a project plan involving a large migration.

> Once you use any cloud provider as a glorified colo without changing any of your processes you’re already spending more than a colo and not getting any of the benefits of managed services.

So you’re saying running multi regional, burstable (metered by minutes or even seconds) VMs with redundant power and networking and without having to hire a bunch of maintenance staff is not one of the benefits? Imho it’s perfectly economical at small to medium scale.

> Have you ever budgeted a project plan involving a large migration

I’m doing one right about now. If you design your stack to be multi-regional from the get go and dont get yourself into aforementioned cloudlock it’s a simple matter of bringing up new kubernetes clusters in a new cloud region and possibly setting up some peering.

Post reply on HN