Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Theseus DHT Protocol (2018)

wootfish.github.io

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Theseus DHT Protocol (2018)

#31

Post by sohle »

boramalper wrote: post #23
eykanal wrote: post #16

This seems to be ~4 years old. Have any cryptographers reviewed / audited this protocol in the interim? A quick google search comes up empty [1], and google scholar doesn't show anything [2]. [1]: https://www.google.com/search?&q=Theseus+dht+protocol+%22aud... [2]: https://scholar.google.com/scholar?q=theseus+dht+protocol

IANAC† but I think the use of Noise Protocol is completely bogus. Regardless of your protocol of choice, you cannot encrypt things end-to-end if you cannot authenticate who is at the other end that you're encrypting for—hence the use of "safety numbers"[0] in Signal, "certificate authorities"[1] in TLS, "web of trust"[2] in PGP, and the list goes on and on... This is at best an obfuscation attempt to make fingerprint…

Hi, I wrote this post. Let me use your comment as a chance to talk about what I was going for with the use of Noise here.

You're correct that part of the assumption inherent in end-to-end encryption is that you've got the other end right; otherwise, MitM attacks become possible. To prevent this, Social apps like Signal (also WhatsApp and many others) give you, roughly speaking, a way of confirming that both endpoints have negotiated the same secure channel. Less-social web apps use PKI. In both these cases, the design makes sense because you care who the other endpoint is. Their identity is meaningful to you outside of the context of the protocol.

In the peer-to-peer context, though, this identity information is mostly meaningless. I think, as long as there are no "network admins" or similarly privileged identities worth claiming, it's less important to certify identities up-front (what would even be a reasonable standard for accepting them?) and it's more important to be able to persist them. You want to make sure you're still talking to the same person you were talking to last time, whoever they are. This is what the PSKs are meant to accomplish (though they are not the only possible solution here).

This is technically still MitM'able, but only if you catch two parties on their very first connection; after that, they have a shared secret, and you should be locked out. Technically this falls slightly short of the ultimate goal of end-to-end encryption, but I still think it's a good fit for this context.

I went into this in (much) more detail in this old blog post: https://eli.sohl.com/2017/06/09/message-encryption-in-theseu...

Re: Theseus DHT Protocol (2018)

#32

Post by sohle »

anacrolix wrote: post #28

I like the ideas in this, and that it's got changes that allow it to work over stream based protocols, which means webrtc and tor. However DHTs (Kademlia at least) aren't efficient on a large scale if they're not built on packet oriented protocols. I don't know what the solution is, maybe allowing TCP is just a compromise you have to make sometimes.

Hi, I wrote this post, and this is something I was wondering about the whole time. It seemed like TCP was a necessary compromise, but I wasn't sure how painful the overhead would be. If there are any notable examples you know of where this went wrong (or right) I'd be interested.

Re: Theseus DHT Protocol (2018)

#33

Post by sohle »

remram wrote: post #2

Does it allow someone with knowledge of a key to find the IP addresses of other people interested in that key? Because if yes, the other security guarantees are not all that interesting. The fact that Tor is mentioned would suggest that you have to add that to be secure.

Hi, I wrote this post, and yes, you're right that IPs would be disclosed, because it is necessary to learn a remote peer's IP before you can make a TCP connection to them. Whether or not this is "secure" depends on who you're trying to be secure against. If you're sharing Linux ISOs, you really don't care who sees you do it; if you're sharing pirated movies, you just care that Disney doesn't find out it was you, so you can use a VPN; if you're sharing government secrets, you might have a more rigorous threat model.

For users in extreme circumstances, Tor gets them a lot, but it also adds a lot of overhead; you'd want to avoid it if you don't need it, not just for your own sake but also for the sake of Tor's finite bandwidth. There is a lot to be said for having a feature like Tor be supported-but-optional, opt-in, and interoperable with users who are not on Tor.

I did consider the idea of allowing Tor hidden service addresses as an alternative identifier to IPs, though that idea comes with its own compromises: you'd only be able to talk to other Tor peers, for instance, or, alternately, every peer would have to be on Tor.

The latter case is obviously a non-option. The former case would kind of work, but the Tor-hidden-service peers' effective network size would be much smaller (since they can only talk to Tor peers) and thus easier to compromise. This is obviously bad - users in need of greater security would end up in a less secure version of the network - and if this subsection gets owned, that could have a second-order impact on the network's overall resilience.

Re: Theseus DHT Protocol (2018)

#35

Post by sohle »

orthecreedence wrote: post #19

Cool project, which lead me to this post: https://eli.sohl.com/2020/06/10/sybil-defense.html I'm in this space right now (Sybil-resistant DHTs) for an identity project and finding things like this is absolute gold for me. Is there a good place to find more articles on modern sybil protection mechanisms (other than the proof-of-whatever blockchain bs)?

Hi, I wrote this post, and I just wanted to say thanks for the kind words - this research was a lot of fun to do and I'm glad it has an audience.

The "best" resources I know of are the ones I cited in that post, though do bear in mind it's a few years old. That said, I did want to make sure you didn't miss this companion piece: https://eli.sohl.com/2020/06/05/dht-size-estimation.html which is applicable to detecting (and - conjecturally - responding to) Sybil attacks.

Re: Theseus DHT Protocol (2018)

#36

Post by sohle »

rainsford wrote: post #25
psychphysic wrote: post #22

Earlier quoted context omitted.

Can you explain this, how can it be indistiguishable from random noise? What does that even mean? And why is it even important/useful? Surely even if it looks like noise anyone can see that you are communicating with the DHT network?

The idea is that from the point of view of an observer able to see every byte of data your protocol puts on the network, there is no way for them to tell whether you are actually speaking the protocol or just exchanging random bytes. Basically this is an extension of the idea that encrypted data should look completely random to someone who doesn't know the encryption key, just applied to an entire network protocol. A…

This is a great explanation, and the point about TLS is well taken as well. If you want to go for that level of misdirection, then depending on your threat model you might consider e.g. using a remote proxy and wrapping your session to it in TLS.

The tricky thing about baking TLS in at the protocol level is that it brings its baggage with it. This is not necessarily a bad thing, but it makes the design more complex to reason about. In particular, it is arguably overkill when you're not planning on using certificates, as is the case here. Just compare the number of steps here: https://tls13.xargs.org/ to any of the patterns here: http://www.noiseprotocol.org/noise.html#interactive-handshak...

Another thing worth mentioning regarding indistinguishability from randomness is the impact of metadata. Even if the bytes you send on the wire look meaningless, there's still the size of the message, the spacing between messages, the time of day, etc. Any of these channels can carry signal, and it is very hard, if not impossible, to get rid of those signals completely.

That said, I still think the original goal of indistinguishability is worthwhile, because if you can force the passive adversary to move from perfectly accurate methods (e.g. fingerprinting message contents) to imperfect ones (e.g. guessing the protocol from message timing), that seems like a win to me.

Re: Theseus DHT Protocol (2018)

#37

Post by wh33zle »

kang wrote: post #8
lenova wrote: post #6

Oooh interesting! I've been reading up on DHT's for the first time recently (I'm very late to the party). It looks like the original inspiration was an anti-censorship network for sharing/providing scientific papers: https://eli.sohl.com/2017/02/17/theseus-robust-system-for-pr... But as another poster noted, the project seems to be defunct do it being a solo effort: https://eli.sohl.com/2017/02/17/theseus-robust-syst…

libp2p: https://curriculum.pl-launchpad.io/curriculum/libp2p/dht/

At the bottom is the link to the more technical specification: https://github.com/libp2p/specs/blob/master/kad-dht/README.m...

Re: Theseus DHT Protocol (2018)

#38

Post by anacrolix »

orthecreedence wrote: post #19

Cool project, which lead me to this post: https://eli.sohl.com/2020/06/10/sybil-defense.html I'm in this space right now (Sybil-resistant DHTs) for an identity project and finding things like this is absolute gold for me. Is there a good place to find more articles on modern sybil protection mechanisms (other than the proof-of-whatever blockchain bs)?

http://bittorrent.org/beps/bep_0042.html

Re: Theseus DHT Protocol (2018)

#39

Post by KirillPanov »

sohle wrote: post #32
anacrolix wrote: post #28

I like the ideas in this, and that it's got changes that allow it to work over stream based protocols, which means webrtc and tor. However DHTs (Kademlia at least) aren't efficient on a large scale if they're not built on packet oriented protocols. I don't know what the solution is, maybe allowing TCP is just a compromise you have to make sometimes.

Hi, I wrote this post, and this is something I was wondering about the whole time. It seemed like TCP was a necessary compromise, but I wasn't sure how painful the overhead would be. If there are any notable examples you know of where this went wrong (or right) I'd be interested.

> If there are any notable examples you know of where this went wrong

IPFS

Stop using sessionful protocols for DHTs. It. does. not. work.

Re: Theseus DHT Protocol (2018)

#40

Post by anacrolix »

KirillPanov wrote: post #39
sohle wrote: post #32

Earlier quoted context omitted.

Hi, I wrote this post, and this is something I was wondering about the whole time. It seemed like TCP was a necessary compromise, but I wasn't sure how painful the overhead would be. If there are any notable examples you know of where this went wrong (or right) I'd be interested.

> If there are any notable examples you know of where this went wrong IPFS Stop using sessionful protocols for DHTs. It. does. not. work.

This is true. Last time I looked at the DHT in IPFS it was full of kludges and workarounds for performance issues due to the fact that it uses TCP connections underneath.
Post reply on HN