Live data from Hacker News

Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

getkloak.io

41–50 of 55 posts

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#41
Awesome project! We need more eBPF projects, and congrats on launching.

Assuming I hijack a production pod, can I not just make an http call to myself with the `kloak:...` secret and get back the real secret? Is there a way to validate destination?

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#42

how does this compare to something like google secrets manager?

It probably doesn't. Google Secrets Manager is a cloud API. This runs in Kubernetes and sits between a Kubernetes Secret object and the application calling for it.

Generally speaking, if you're running Kubernetes in GCP (likely via GKE), and you control how your applications retrieve their secrets, you're likely better off with a combination of Workload Identity Federation, tight IAM to Secrets Manager, and a smart secrets retrieval strategy which likely involves lazy loading secrets and attempting a reload in case of a permission denied so it can deal with secrets rotation.

For applications where that's not an option, the state-of-the-art has been ensuring etcd is actually encrypted (as opposed to the default Base64), and relying on Kubernetes Secrets, usually either mounted in the filesystem or passed to environment variables.

Both these approaches have weaknesses since they're immediately available to all processes in the container.

OP seems to solve that by never exposing the secrets to the application, by sitting between the application and the service and replacing the secret on the wire, outside of the application's reach.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#43
post #16

You should split your controller - it is running in both the control and data planes. Idea is good though, wish you luck.

Thank you! Not really, the controller is not doing dataplane per-say, it only pushes eBPF programs to the kernel for the relevant apps/cgroups so that could be considered control-plane. The full data-plane run in eBPF.

1. Code that manages the Kubernetes manifests. This doesn’t need to live alongside your actual app.

2. Code that does injection in eBPF and needs to live along your app.

From my understanding from the README and helm chart, these are both in the daemonset.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#44
post #41

Awesome project! We need more eBPF projects, and congrats on launching. Assuming I hijack a production pod, can I not just make an http call to myself with the `kloak:...` secret and get back the real secret? Is there a way to validate destination?

Yes, we have host and ip filtering in place that can be used to ensure the secret is sent only for the destination we expect.

It's not perfect though, see Host Filtering | https://getkloak.io/docs/guides/host-filtering.html

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#45

how does this compare to something like google secrets manager?

It probably doesn't. Google Secrets Manager is a cloud API. This runs in Kubernetes and sits between a Kubernetes Secret object and the application calling for it. Generally speaking, if you're running Kubernetes in GCP (likely via GKE), and you control how your applications retrieve their secrets, you're likely better off with a combination of Workload Identity Federation, tight IAM to Secrets Manager, and a smart s…

That's right, OP is the main maintainer and the idea he has is that nothing should change in the application. The application believe it has the secret, but the secret is injected on the wire AND only for the intended destination.

Please have a look at the demo if you can ; there is a webhook that abstract changing the secret resource name for you. You just "annotate" the secret resource and kloak admission controller will rewrite secrets of your deployment resource for you after that. This means the app never actually see the secret (accidental or not).

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#46

Earlier quoted context omitted.

It probably doesn't. Google Secrets Manager is a cloud API. This runs in Kubernetes and sits between a Kubernetes Secret object and the application calling for it. Generally speaking, if you're running Kubernetes in GCP (likely via GKE), and you control how your applications retrieve their secrets, you're likely better off with a combination of Workload Identity Federation, tight IAM to Secrets Manager, and a smart s…

That's right, OP is the main maintainer and the idea he has is that nothing should change in the application. The application believe it has the secret, but the secret is injected on the wire AND only for the intended destination. Please have a look at the demo if you can ; there is a webhook that abstract changing the secret resource name for you. You just "annotate" the secret resource and kloak admission controlle…

Interesting, thanks! I've also forwarded this to our security principal so we can evaluate it.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#47
post #36

Earlier quoted context omitted.

For egress proxy the app need to: - send traffic to the proxy (either in a non transparent way or using routes or even ebpf to redirect traffic to the proxy transparently) - trust the proxy certs or use plain http/TCP to the proxy With kloak, the app don't need any modification and you avoid a single point of failure (aka egress proxy). Each app has an independent ebpf program attached to it that can survive the cont…

cool, but the single point of failure (it could be HA-proxy) is the point, it's a choke point. I get both architectures have pros and cons, with the proxy approach you remove secrets from the application environment entirely. Plain HTTP shouldn't be an issue, neither should internal certs whose only point is to allow applications that refuse to work with plain-http to function. I would prefer the best of both worlds,…

Thank you! I agree, each architecture have its pro and cons. If an egress gateway is available and can handle secrets it's definitely a viable solution.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#48

Was just talking about this the other day - although more in-line with a custom controller to replace _all_ secrets / env variables used at runtime automatically (LD_PRELOAD get_env ?). Recognize this serves a different use case - I was trying to only decrypt KMS encrypted secrets in-memory / in-flight so that an attacker would have a harder time reading secrets in-cluster or in pod shell. Such a sick idea, and incre…

Thank you!

We are planning to integrate with external secret operators, like AWS secret manager or Openboa/Vault so users can benefit from an end to end secrets protection. secret encryption/sealing at rest (through secrets managers) and protecting secrets from in-memory exfiltration attacks with kloak.

The idea is to let the ESO handle the secret at rest and delivering it to Kloak that then would continue to do the kloaked secret rewrite so the secret will only be available in a non encrypted form in Kloak. We can even push the concept further and do KMS decryption just in time to reduce the window where the secret is available.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#49
post #16

Earlier quoted context omitted.

Thank you! Not really, the controller is not doing dataplane per-say, it only pushes eBPF programs to the kernel for the relevant apps/cgroups so that could be considered control-plane. The full data-plane run in eBPF.

1. Code that manages the Kubernetes manifests. This doesn’t need to live alongside your actual app. 2. Code that does injection in eBPF and needs to live along your app. From my understanding from the README and helm chart, these are both in the daemonset.

Actually we have 2 applications along those boundaries you described. a webhook app that manage kubernetes manifest and another to inject the ebpf code and manage the ebpf maps.

Thank you for the feedback though! I think we need to clarify the doc to make that separation clear. I will open an issue for that and we will work on it.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#50
I used to run a similar company in this space. gearsec.io My 2 cents are that no one we spoke was comfortable with us having mitm plus access to their secrets no matter how much we told we'll host it in your cloud.

The few who agreed were rigorously testing our product and asked for code SBOMs before even a pilot.

Infiscals agent vault might be the best middle ground for this kind of setups I feel sometimes

Post reply on HN