Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

getkloak.io

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#31

Post by neo2006 »

subhobroto wrote: post #27
neo2006 wrote: post #19

Earlier quoted context omitted.

Thank you! We will reachout and see what can be done

Please let me know how I can help. Can I write or review the initial forum post for you or anything that can help both of us? - What's the best way to discuss this specific topic with you? As an https://github.com/spinningfactory/kloak/issues or something else? - My specific usecase is to not need Conjur Secretless Broker ( https://github.com/cyberark/secretless-broker ) - my understanding of eBPF is entirely superfi…

yes please open an issue on https://github.com/spinningfactory/kloak/issues and we can discuss this. I'm not familiar with secretless-broker but we can definitely see if that use case fit with kloak and get into more specifics on how you can help.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#32

Post by neo2006 »

subhobroto wrote: post #28
neo2006 wrote: post #10

Earlier quoted context omitted.

Thank you! We appreciate your enthusiasm! :-) From technology perspective nothing prevent kloak to do rewrite on any workload scheduler or even without a scheduler (native Linux). The main challenge is to find a flow to signal to kloak what to rewrite and how to inject kloaked secrets to the workload. TBH supporting other technologies is not something we thought about but we can definitely consider if there is an ask…

> The main challenge is to find a flow to signal to kloak what to rewrite and how to inject kloaked secrets to the workload Would it be realistic or reasonable to detect a header like `X-kloak-ENABLED` or specific endpoints in the case of HTTP? Similar for wire protocols like PostgreSQL or gRPC? Our would a usermode proxy be easier but not preferred due to overhead?

The way we thought about it is from the lense of 2 personas: - a persona that control the control plain side, what secret to distribute to which user and what hosts they are allowed to send that secret to (probably platform team or secops team) - a persona that represent the user that need to reach host X with secret Y (probably the dev team)

based on this secret rewrite signal need to be out of band and not part of the request it self or the whole model will fall apart.

We already have the intention to support rewrites for specific headers but those headers are defined by the first persona out of band too.

btw, we support rewrite for postgres protocol for db password.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#33

Post by neo2006 »

sandeepkd wrote: post #30

There is also a model where the applications sign the payload using the secret, AWS is a big user of that across all its API's.

This is not something we support currently. We will need to do some research on ways to support it.

The main hurdle is that we can't rewrite secrets in any of the user buffers as this will defy our threat model and signing is usually done in user space.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#34

Post by __turbobrew__ »

neo2006 wrote: post #23
giladd wrote: post #22

This is pretty cool, nice project. Can you expand on what threat model this combats? Also, does the replace op happen only for specific fields in HTTP, or for every matching string in the request? I can imagine the latter if you want to support non-standard authentications methods, though there's always the edge case where the secret string placeholder is not used as a secret and should not be replaced.

The main threat model is application leaking secrets: - Internet facing app that could potentially be hacked and bad actor exfiltrating secrets - AI agent that can exfiltrate secrets through prompt injection for example or context poisoning - The general use case where a secret can be for example inject by mistake in logs for instance

How does this compare with TPUs? Can you not have secrets in the TPU which cannot be accessed directly by apps, solving this threat vector? I get that you want compatibility with popular libraries, but I wonder if the actual solution is to use hardware support to enforce the secret boundaries.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#35

Post by notepad0x90 »

I've heard one way to acheive this is by using api proxies/gateways. you can store secrets in a vault if you wish, but with a proxy, your app makes requests as usual without using secrets, its requests are then intercepted by the proxy to add authentication information transparently.

The added benefit is that you can also manage things like api rate limits, and implement all sorts of cool monitoring and api-specific threat detection centrally. I don't know of a way to do this outside of cloud provider services though.

Architecturally speaking, you have an environment that is at the same level of trust with respect to the data it processes, anything in there is unsecured, but all interactions outside of the system passes through a gateway proxy that manages all of what i mentioned earlier, including secret management.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#36

Post by neo2006 »

notepad0x90 wrote: post #35

I've heard one way to acheive this is by using api proxies/gateways. you can store secrets in a vault if you wish, but with a proxy, your app makes requests as usual without using secrets, its requests are then intercepted by the proxy to add authentication information transparently. The added benefit is that you can also manage things like api rate limits, and implement all sorts of cool monitoring and api-specific…

For egress proxy the app need to:

- send traffic to the proxy (either in a non transparent way or using routes or even ebpf to redirect traffic to the proxy transparently)

- trust the proxy certs or use plain http/TCP to the proxy

With kloak, the app don't need any modification and you avoid a single point of failure (aka egress proxy). Each app has an independent ebpf program attached to it that can survive the control plane going down and don't need to trust any special certs or change the endpoint it sends traffic to.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#37

Post by neo2006 »

__turbobrew__ wrote: post #34
neo2006 wrote: post #23

Earlier quoted context omitted.

The main threat model is application leaking secrets: - Internet facing app that could potentially be hacked and bad actor exfiltrating secrets - AI agent that can exfiltrate secrets through prompt injection for example or context poisoning - The general use case where a secret can be for example inject by mistake in logs for instance

How does this compare with TPUs? Can you not have secrets in the TPU which cannot be accessed directly by apps, solving this threat vector? I get that you want compatibility with popular libraries, but I wonder if the actual solution is to use hardware support to enforce the secret boundaries.

I'm not super familiar with TPUs and Trusted execution environments but my understanding is that it serve a different threat model.

TEE aim to protect a certain workload from the host to avoid another workload on the same host from steeling secrets. Kloak aim is to protect the secret from the workload itself not the host.

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#38

Post by jackdecker »

Was just talking about this the other day - although more in-line with a custom controller to replace _all_ secrets / env variables used at runtime automatically (LD_PRELOAD get_env ?). Recognize this serves a different use case - I was trying to only decrypt KMS encrypted secrets in-memory / in-flight so that an attacker would have a harder time reading secrets in-cluster or in pod shell.

Such a sick idea, and incredibly useful. Would be nice if it integrated directly with secrets managers RE: ESO

Re: Show HN: Kloak, A secret manager that keeps K8s workload away from secrets

#39

Post by notepad0x90 »

neo2006 wrote: post #36
notepad0x90 wrote: post #35

I've heard one way to acheive this is by using api proxies/gateways. you can store secrets in a vault if you wish, but with a proxy, your app makes requests as usual without using secrets, its requests are then intercepted by the proxy to add authentication information transparently. The added benefit is that you can also manage things like api rate limits, and implement all sorts of cool monitoring and api-specific…

For egress proxy the app need to: - send traffic to the proxy (either in a non transparent way or using routes or even ebpf to redirect traffic to the proxy transparently) - trust the proxy certs or use plain http/TCP to the proxy With kloak, the app don't need any modification and you avoid a single point of failure (aka egress proxy). Each app has an independent ebpf program attached to it that can survive the cont…

cool, but the single point of failure (it could be HA-proxy) is the point, it's a choke point. I get both architectures have pros and cons, with the proxy approach you remove secrets from the application environment entirely. Plain HTTP shouldn't be an issue, neither should internal certs whose only point is to allow applications that refuse to work with plain-http to function. I would prefer the best of both worlds, where the proxies are per-node personally.

But not everyone wants to, or can afford to run a proxy for credential management. I started looking into this mostly to regulate API usage, especially burning through tokens when calling LLM apis, the credential benefit only occurred to me afterwards. Great work with it, no idea how the eBPF magic is making it work, I'll have to find out.

Post reply on HN