Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

End-to-End Encryption through Kafka

github.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: End-to-End Encryption through Kafka

#11

Post by mattgreg »

bdamm wrote: post #6
gopalv wrote: post #4

Looking at the setup code here, this looks a lot like SSL reinvented over Kafka (instead of over TCP), which is not a bad thing, just the mechanism feels a lot like you've automated a tcpdump into the protocol by making the handshake packets persisted in the stream + committed to disks. This somewhat makes sense if you're going to buy Kafka from a cloud provider without the ability to trust them, but the trust here i…

Perfect Forward Secrecy is pretty standard in TLS handshakes these days, so the persistence of messages should not be an issue. This pattern of burying TLS inside of application payloads is popping up all over the place. People realize that securing the transport isn't the same as securing the application, which often hops over multiple transports.

"People realize that securing the transport isn't the same as securing the application, which often hops over multiple transports."

Exactly right. Application layer trust, with the transport and security layers abstracted away. This is important, since app developers typically can't control the transports from end to end, or at scale.

Re: End-to-End Encryption through Kafka

#12

Post by mwadhwa »

vatkhrt wrote: post #3

Does the threat model include malicious system software on the endpoints? Or slightly differently, have you looked at any mechanisms to provide encryption when in use.

We have a little bit and are watching the development of techniques like computing on Homomorphically Encrypted data, Multi-party Computation and Zero Knowledge Proofs with great interest.

For example, the credentials functionality in Ockam is using BBS+ signatures that enable selective disclosure of attested attributes. This has both privacy and bandwidth benefits when proving possession of a credential.

But we feel end-to-end encrypted communication is a prerequisite to building systems that compute on encrypted data. So we started with a focus on making end-to-end easy.

Re: End-to-End Encryption through Kafka

#13

Post by mwadhwa »

mwadhwa wrote: post #12
vatkhrt wrote: post #3

Does the threat model include malicious system software on the endpoints? Or slightly differently, have you looked at any mechanisms to provide encryption when in use.

We have a little bit and are watching the development of techniques like computing on Homomorphically Encrypted data, Multi-party Computation and Zero Knowledge Proofs with great interest. For example, the credentials functionality in Ockam is using BBS+ signatures that enable selective disclosure of attested attributes. This has both privacy and bandwidth benefits when proving possession of a credential. But we feel…

Also, about malicious software at the endpoint ... Ockam Vaults can be backed by Hardware and Vault implementations are pluggable. So Key Storage and Cryptographic Operations can be delegated off the primary the processor.

So far we've built vault implementations for the Microchip ATEEC hardware which is popular in IoT circles. We plan to have Vaults for Cloud HSMs, Cloud Secret Stores, ARM TrustZone etc.

That said, if an attacker in the primary processor of an endpoint. We could protect agains key theft (with vault) ... but its very hard (maybe impossible) to protect against the attacker using the key until they are kicked out somehow.

Re: End-to-End Encryption through Kafka

#14

Post by mwadhwa »

gopalv wrote: post #4

Looking at the setup code here, this looks a lot like SSL reinvented over Kafka (instead of over TCP), which is not a bad thing, just the mechanism feels a lot like you've automated a tcpdump into the protocol by making the handshake packets persisted in the stream + committed to disks. This somewhat makes sense if you're going to buy Kafka from a cloud provider without the ability to trust them, but the trust here i…

Great observations!

Data removal from Kafka for GDPR compliance is trivial in this setting. You just purge your keys. There are also key rotation and revocation protocols that are part of Ockam and we're making them simple to use from our libraries.

The underlying cryptography in Ockam SecureChannels comes from the Noise Protocol Framework (created by one of the authors of Signal Messenger's protocols). The Noise Framework protocols have many published security proofs and is gaining adoption in multiple secure systems. We are proposing an alternative to SSL/TLS but this alternative is well proven/understood in terms of it security guarantees.

Re: End-to-End Encryption through Kafka

#16

Post by mwadhwa »

jack2 wrote: post #15

Instead of bidirectional messages using two topics, could you use messages in a single topic encrypted with something like PGP, S/MIME or JWE?

Great question!

We're thinking about Ockam Routing, Transports, Streams and SecureChannels as composable building blocks for end-to-end encryption. This allows us to provide end-to-end encryption along a route that may look like this:

Alice Bluetooth hop TCP hop Kafka TCP hop Bluetooth hop Bob

Data between Alice and Bob is end to end encrypted. This guide has some detail about how that works, if you're interested https://github.com/ockam-network/ockam/tree/develop/document...

Currently One Ockam Stream is backed by one Kafka topic. This is simpler, more composable and easier to think about so we started with this approach. In theory there could be a Bidirectional Stream that is backed by one topic, but we haven't focused on that yet.

Re: "PGP, S/MIME or JWE etc."

A secure channel protocols like Ockam SecureChannel or TLS are more comprehensive and have more security guarantees than simple encryption primitives like PGP or JWE. They provide guarantees like forward secrecy and resistance to Key Compromise Impersonation among other things

Re: End-to-End Encryption through Kafka

#17

Post by staticassertion »

Ockam looks super cool. I've been thinking about trust in our networks and I think that there's a lot left on the table to be leveraged from these sorts of frameworks.

The old school method is "if it's in the network it's trusted". Obviously (in retrospect) this is terrible.

The new thing is Zero Trust, where you have point to point authentication. Ockam gives you this at an application layer, which is critical if your application serves APIs across trust boundaries (ie: "this API can be called by user group A, this one from B).

Further, Ockam also enables a sort of tunneled trust where you can have applications A, B, and C, and B can be untrusted, and A can communicate with C across B. This unlocks a bunch of things like encrypted delegation (C can know that B has been delegated permissions from A).

Really cool stuff. Excited to see it evolve.

Post reply on HN