Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scuttlebot: Peer-to-peer database, identity provider, and messaging system

scuttlebot.io

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#81

Post by cel »

Vendan wrote: post #61
TimJRobinson wrote: post #56

Earlier quoted context omitted.

Once someone has written a library to do this correctly in Go/Rust/Whatever isn't this problem solved? Everyone building scuttlebutt apps with that language can use that library. It didn't seem like this protocol is changing.

I tried, and tried, and tried to make that library for Go. I failed, cause of serialization issues too involved for it to be worth it to me (got to the point where I'd have to write my own json implementation) Just giving people the heads-up that JS seemingly is the only blessed language for scuttlebutt

The work was picked up and completed.

Rust:

https://github.com/sunrise-choir/ssb-legacy-msg/blob/master/...

https://github.com/sunrise-choir/ssb-publish/blob/master/src...

https://sunrise.social/

Go:

https://github.com/cryptoscope/ssb/blob/master/message/legac...

https://github.com/cryptoscope/ssb/blob/master/message/legac...

https://planetary.social

Spec:

https://spec.scuttlebutt.nz/feed/messages.html#json-encoding

https://spec.scuttlebutt.nz/feed/datamodel.html#signing-enco...

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#82

Post by fabianhjr »

crispyporkbites wrote: post #63
camdenlock wrote: post #26

Earlier quoted context omitted.

Yeah, this is where I lost interest too.

I don’t understand why npm install is so bad here? It’s just a package manager. It’s not hard to write your implementation without npm if you prefer.

it is, I tried with haskell and gave up when I noticed I couldn't easily get json serialization to be in the same order as the javascript implementation and therefor couldn't verify nor sign messages with interoperability with the current clients.

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#83

Post by viraptor »

SyrupThinker wrote: post #47
Vendan wrote: post #31

Scuttlebutt is a neat concept, burdened by a bad protocol. Signing a message involves serializing a json object, signing it, adding the signature as a field on that json object, and then serializing it again. To verify, you deserialize the message into an object, remove the signature field, and then reserialize it, and verify the signature against that new serialization. This means that all the clients have to have a…

After reading about the protocol I came to a similar conclusion as you. Although it needs to be noted that the JSON serialization is defined as JSON.stringify as defined in ECMA-262 6th Ed. plus some more. To me it's worse that key order must be preserved, which this standard does not specify the way I understand it. Source: https://ssbc.github.io/scuttlebutt-protocol-guide/#message-f...

This is terrible. It's basically not JSON anymore. It's some custom text protocol with similar value escaping...

If they wanted both the descriptions to be visible and the order to be preserved, they could use:

    [ ["previous", "..."], ["author", "..."], ...

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#84

Post by zenhack »

elviejo wrote: post #78
flir wrote: post #55

Earlier quoted context omitted.

Binary protocols. Or out-of-band signing.

Do you mind going deeper here? Which protocols? Can you point to examples? Because I was about to design a signing Json solution but based on the comments here it is a bad idea.

The signing schemes I've seen used in binary protocols fall into two categories:

1. Canonicalize and sign: the format has a defined canonical form. Convert to that before doing cryptographic operations. If the format is well designed around it, this is doable, whereas JSON doesn't really have this and with many libraries it's hard to control the output to the degree that you'd need. 2. Serialize and sign: serialize the data as flat bytes, and then your signed message just has a "bytes" field that is the signed object. This is conceptually not far off from the base64 solution above, except that there's not extra overhead, since with a binary protocol you'll have a length prefix instead of having to escape stuff.

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#85

Post by bmm6o »

elviejo wrote: post #78
flir wrote: post #55

Earlier quoted context omitted.

Binary protocols. Or out-of-band signing.

Do you mind going deeper here? Which protocols? Can you point to examples? Because I was about to design a signing Json solution but based on the comments here it is a bad idea.

Being able to separate the object and signature saves tons of trouble https://latacora.micro.blog/2019/07/24/how-not-to.html

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#86

Post by tptacek »

viraptor wrote: post #83
SyrupThinker wrote: post #47

Earlier quoted context omitted.

After reading about the protocol I came to a similar conclusion as you. Although it needs to be noted that the JSON serialization is defined as JSON.stringify as defined in ECMA-262 6th Ed. plus some more. To me it's worse that key order must be preserved, which this standard does not specify the way I understand it. Source: https://ssbc.github.io/scuttlebutt-protocol-guide/#message-f...

This is terrible. It's basically not JSON anymore. It's some custom text protocol with similar value escaping... If they wanted both the descriptions to be visible and the order to be preserved, they could use: [ ["previous", "..."], ["author", "..."], ...

lvh at Latacora wrote a blog post about this problem.

https://latacora.micro.blog/2019/07/24/how-not-to.html

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#87

Post by rakoo »

Vendan wrote: post #31

Scuttlebutt is a neat concept, burdened by a bad protocol. Signing a message involves serializing a json object, signing it, adding the signature as a field on that json object, and then serializing it again. To verify, you deserialize the message into an object, remove the signature field, and then reserialize it, and verify the signature against that new serialization. This means that all the clients have to have a…

Indeed, json maps are not supposed to be ordered so doing anything that depends on the order is bound to fail.

This is the exact reason bencode (https://en.wikipedia.org/wiki/Bencode) was invented, and I still believe we can replace all uses of json by bencode and be better off it, because it solves all too common issues:

- bencoding maps are in lexicographical order of the keys, so no confusion possible for hashing/signing (a torrent id is the hash of a bencoding map)

- bencoding is binary friendly, in fact it must be because it stores the pieces hashes of the torrent

Why don't we use bencoding everywhere ?

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#88

Post by rakoo »

sneak wrote: post #43
Vendan wrote: post #31

Scuttlebutt is a neat concept, burdened by a bad protocol. Signing a message involves serializing a json object, signing it, adding the signature as a field on that json object, and then serializing it again. To verify, you deserialize the message into an object, remove the signature field, and then reserialize it, and verify the signature against that new serialization. This means that all the clients have to have a…

When I was faced with this (signing a structure), I serialized the json into base64, then put that base64 string as a value (along with the MAC) into a new json document. It of course increases deserialization overhead (json, verify, unbase64, inner json) but sidesteps this issue. I thought about sorting keys and other things like that, and the dozen edge cases and potential malleability issues dissuaded me for the c…

Use Bencoding, like bittorrent does: https://en.wikipedia.org/wiki/Bencode

As I put in another comment, a torrent id is a hash of a map, where one of the keys contains binary data. bencoding solved that decades ago already.

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#89

Post by rakoo »

jekrb wrote: post #27
z0mbie42 wrote: post #10

Earlier quoted context omitted.

Thank you! The lack of multi-device support wasn't too much cumbersome?

The lack of multi-device support was a constraint that gave me a different perspective of interactions on the web. My ssb keypair was on a work laptop, so when I changed jobs and had to give my laptop back, I lost my keypair. Now, I could have exported the keypair and continued to use my "account" on my new laptop. The network would have synced on my new device, and I'd get all my posts and pictures back. But decided…

Creating a new account from scratch also means rotating your keys, which is a good practice. As the last few discussions on PGP have shown, the model of having a long-term identity key is more dangerous than it seems, because a single mistake (by you or by the application developer) means so much content can now be leaked. It's probably easier to let the natural connection between people be the vector of long-term trust, which ironically SSB emphasizes on

Re: Scuttlebot: Peer-to-peer database, identity provider, and messaging system

#90

Post by codygman »

fabianhjr wrote: post #82
crispyporkbites wrote: post #63

Earlier quoted context omitted.

I don’t understand why npm install is so bad here? It’s just a package manager. It’s not hard to write your implementation without npm if you prefer.

it is, I tried with haskell and gave up when I noticed I couldn't easily get json serialization to be in the same order as the javascript implementation and therefor couldn't verify nor sign messages with interoperability with the current clients.

> I tried with haskell and gave up when I noticed I couldn't easily get json serialization to be in the same order as the javascript implementation and therefor couldn't verify nor sign messages with interoperability with the current clients.

This gets you most of the way there:

> With the Aeson library, the order of keys in objects is undefined due to objects being implemented as HashMaps. To allow user-specified key orders in the pretty-printed JSON, encodePretty' can be configured with a comparison function.

- https://hackage.haskell.org/package/aeson-pretty-0.8.7/docs/...

I'm not sure that messes with spacing or not though or you require specific space preserving requirements to interoperate with current clients.

Even if not you can probably look at the source of encodePretty' and work out how to not mess with spacing or keep it all on one line.

I've wanted a version of Aeson for a while that preserves order and all formatting for things like linters that are very unobtrusive. It seems your use case would have benefitted from that as well.

Post reply on HN