Live data from Hacker News

AWS Controllers for Kubernetes

github.com

11–20 of 54 posts

Re: AWS Controllers for Kubernetes

#11
I started building something along this line a few years ago. the ability to control AWS VM and treat them as pods (i.e. can backend a service, access other services) to have a hybrid (VM / Container) infrastucture that is all managed in the kubernetes way. Future work would have been to try and manage other resources similarly.

Sadly startup interest changed and then went under (but the freedom I was given to explore there was the best experience I have ever had)

https://github.com/apporbit/infranetes

Re: AWS Controllers for Kubernetes

#12
post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

Hi I'm a developer advocate in the AWS Container organization. This question of how to handle resource destruction is an active issue on the project, and we'd welcome your comments (or those of anyone else) on this Github issue: https://github.com/aws/aws-controllers-k8s/issues/82

Our goal is to make this project have "no surprises" and therefore no unexpected destruction of resources. The specifics of how we mark resource as safe to delete instead of retaining by default are under discussion on that Github issue.

Re: AWS Controllers for Kubernetes

#14
post #5
post #3

Kelsey Hightower had a good take in https://twitter.com/kelseyhightower/status/12963119515307048... "AWS Controllers for Kubernetes is pretty dope. You can leverage Kubernetes to manage AWS resources such as API gateways and S3 buckets. Think Terraform but backed by Kubernetes style APIs and "realtime" control loops." An in the thread he mentions Crossplane as the cross-cloud way to do this https://twitter.com/kelsey…

Every Kubernetes shop I've seen uses Helm to deploy their workloads, & now you can set up all your S3 Buckets, SQS, SNS stuff in that same Helm chart as your app. This is hella ideal. I also just generally love the thought of being able to manage any cloud resources at all via standard open-protocols & systems. Compounding the investment, rather than having to invest in a bunch of specific not-interconnected areas is…

It’s all fun and games until a deployment is acting funny and somebody destroys the chart and re-creates it and takes the bucket out with it.

Does it have functionality like Kube DB that makes a “dormant” version of the state store?

Re: AWS Controllers for Kubernetes

#15
post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

Hi I'm a developer advocate in the AWS Container organization. This question of how to handle resource destruction is an active issue on the project, and we'd welcome your comments (or those of anyone else) on this Github issue: https://github.com/aws/aws-controllers-k8s/issues/82 Our goal is to make this project have "no surprises" and therefore no unexpected destruction of resources. The specifics of how we mark re…

I sincerely hope this works well. CloudFormation has already many issues with unintended changes, visibility of the changes to be done, auto update/deletion of resources, etc. Having another path to provision infrastructure that is safe, consistent, and provides visibility at all phases would be great.

Re: AWS Controllers for Kubernetes

#16
post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

Also, not all AWS follow the same deletion semantics. Example: S3 buckets. The report as being deleted somewhat quickly, but their name may not be available again for hour or so.

In this case the delete will appear to succeed, but the recreation, if done with the same name, may fail.

Re: AWS Controllers for Kubernetes

#18
post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

Exactly. Planning accordingly is the right answer. Assume anyone can destroy your infrastructure at any time (by mistake or otherwise). This could be done with cloudformation, terraform, API calls, and essentially any automation(with different levels of safeguards). Be prepared for that. Be careful with your data. Not so careful with individual servers - they should be cattle, not pets. EDIT: If this is a production…

"cattle, not pets" is a nice slogan, but I think most ranchers would be angry if a junior ranch hand accidentally poisoned the entire herd with a typo.

Re: AWS Controllers for Kubernetes

#19
post #14
post #5

Earlier quoted context omitted.

Every Kubernetes shop I've seen uses Helm to deploy their workloads, & now you can set up all your S3 Buckets, SQS, SNS stuff in that same Helm chart as your app. This is hella ideal. I also just generally love the thought of being able to manage any cloud resources at all via standard open-protocols & systems. Compounding the investment, rather than having to invest in a bunch of specific not-interconnected areas is…

It’s all fun and games until a deployment is acting funny and somebody destroys the chart and re-creates it and takes the bucket out with it. Does it have functionality like Kube DB that makes a “dormant” version of the state store?

Deletion is typically specified by an immutable finalizer on the resource. For example, you can create a StorageClass that saves your PersistentVolume when you delete the PersistentVolumeClaim, and you can't change the class of a claim, so even if your templating goes haywire and deletes your volume claim, the actual data is safe and can be recovered. I have to imagine that any API that lets you create things like RDS instances would have similar protection. (You might think that the k8s world is very mutable, but there are some immutable things hanging around. For cases exactly like this.)

I don't use AWS so I didn't look into this thoroughly, but the word "finalizer" does show up in the code frequently:

    // MarkManaged places the supplied resource under the management of ACK.
    // What this typically means is that the resource manager will decorate the
    // underlying custom resource (CR) with a finalizer that indicates ACK is
    // managing the resource and the underlying CR may not be deleted until ACK
    // is finished cleaning up any backend AWS service resources associated
    // with the CR.
I dunno how good it is, but clearly they've thought about it. Test it before you invest billions of dollars into it, though.

Re: AWS Controllers for Kubernetes

#20
post #8

One consequence is the accidental deletion of AWS things... If a CRD is deleted the CRs described it are also deleted. So, deleting a CRD (even accidentally) could end up deleting resources in AWS (e.g., backups). So, be careful. Some things being managed by Kubernetes would be really cool. Other things being managed by k8s could break things if something goes wrong. I would plan accordingly.

I think you have mostly the same problems in Terraform, Pulumi, or Cloudformation though right? Is there anything that makes it easier to accidentally do in k8s?

One layer of defense in all of these cases is keeping the IAM credentials that the configuration management tool uses from having any deletion permissions.

Post reply on HN