Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PeerPad – A realtime P2P collaborative editing tool powered by IPFS

peerpad.net

Post reply on HN
Print view View original

11–20 of 25 posts

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#11

Post by daviddias »

Hi everyone!

Excited that you found PeerPad so fast. We've released its alfa just a week ago at Mozfest (https://github.com/MozillaFoundation/mozfest-program-2017/is...).

I see that there are some questions about PeerPad's architecture. If you would like to know how it works, I invite you to glance through the Architecture.md at https://github.com/ipfs-shipyard/peerpad/blob/master/docs/AR... If there are things that are not clear to you please open an issue so that we can make it better.

Also, you can expect to see soon a video tutorial of how to build a p2p flipchart using the same underlying kv store that PeerPad uses, y.js + IPFS. If you can't wait you can always take a look at https://github.com/ipfs-shipyard/p2p-flipchart and see how it works :)

We will continue working on PeerPad and add the remaining features necessary to make it usable for meeting notes, publishing and more. You can track the development and also help us built it through the Github Repo https://github.com/ipfs-shipyard/peerpad

Thank you \o/

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#12

Post by pgte »

espadrine wrote: post #9
pgte wrote: post #8

Earlier quoted context omitted.

If you don't mind me asking, what browser and version did you use? It uses WebRTC for peer-to-peer communication, which is yet unsupported / untested in some browsers. To answer your question: The collaborative data is saved locally by every participating peer. There are plans to add remote tracking and pinning, increasing the persistence and availability guarantees: https://github.com/ipfs-shipyard/peerpad/issues/90

How would remote pinning work? If I understand correctly, each keystroke in the pad generates a new snapshot, which has a distinct IPFS ID. Should the remote pinning service publish a list of all IPFS IDs on the p2p network, and store each individual document (each one keystroke away from the previous one) in the history graph? Should it keep the complete list of CRDT operations, and map IPFS IDs to CRDT operations I…

Remote pinning would work as any other node that you give the permissions to read the feed. This node would follow the CRDT changes, persisting them locally.

Each keystroke produces a change in the CRDT, which is then eventually propagated to all participating nodes.

Each CRDT message is signed and encrypted before being sent over the IPFS pubsub network.

This protocol is not IPFS-specific, but there are plans to change this: https://github.com/ipfs-shipyard/peerpad/issues/107

The snapshotting occurs over IPFS, producing a static and encrypted self-contained snapshot, published over IPFS.

It is possible to access the CRDT properties inside the core library, where the CRDT is formed: https://github.com/ipfs-shipyard/peerpad-core/blob/master/sr... . This could be exposed if you would require it..

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#13

Post by mbar »

I see these websocket connections in my network panel.

wss://sfo-3.bootstrap.libp2p.io/ wss://nyc-1.bootstrap.libp2p.io/ wss://nyc-2.bootstrap.libp2p.io/ wss://ams-1.bootstrap.libp2p.io/ wss://lon-1.bootstrap.libp2p.io/

I'm going to guess this is to establish the connections between peers. So this isn't really p2p because while there aren't any intermediaries for data transfer, there certainly are to establish the data connections.

If you want a p2p system because you care mostly about removing servers as a bottleneck, then this may not be an issue, but if you care about removing servers because you don't trust intermediaries, then this only gets you half way there.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#14

Post by diggan »

mbar wrote: post #13

I see these websocket connections in my network panel. wss://sfo-3.bootstrap.libp2p.io/ wss://nyc-1.bootstrap.libp2p.io/ wss://nyc-2.bootstrap.libp2p.io/ wss://ams-1.bootstrap.libp2p.io/ wss://lon-1.bootstrap.libp2p.io/ I'm going to guess this is to establish the connections between peers. So this isn't really p2p because while there aren't any intermediaries for data transfer, there certainly are to establish the da…

So the discovery is currently via Bootstrap Peers and a Signaling server that can help connect peers together. A P2P network needs some way of bootstrapping and having a couple of stable peers for doing that is the simplest and works in most scenarios.

But, it doesn't work in all scenarios. In the go-ipfs version of ipfs (or running js-ipfs) we can use mDNS for local discovery without any bootstraps, but unfortunately doesn't work in the browser (yet! FlyWeb would help https://github.com/ipfs/in-web-browsers/issues/45). Some sort of gossiping could work as well but we're always glad to hear other ideas of how we can make discovery work without having to hardcoded bootstrap nodes or signaling servers.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#15

Post by lgierth »

mbar wrote: post #13

I see these websocket connections in my network panel. wss://sfo-3.bootstrap.libp2p.io/ wss://nyc-1.bootstrap.libp2p.io/ wss://nyc-2.bootstrap.libp2p.io/ wss://ams-1.bootstrap.libp2p.io/ wss://lon-1.bootstrap.libp2p.io/ I'm going to guess this is to establish the connections between peers. So this isn't really p2p because while there aren't any intermediaries for data transfer, there certainly are to establish the da…

These websockets node you see aren't servers. They're nodes like any other node, except they have a well-known domain name. They don't do any work that a browser js-ipfs node wouldn't be capable of.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#16

Post by marknadal »

Hey, David/others I'm very impressed with your work on Y-JS (the CRDT library powering this). I'm the author of gun, which is currently (as far as I know from stats) the most popular CRDT-based data sync (MIT/ZLIB/Apache2 licensed, 6.6K+ stars, https://github.com/amark/gun ).

Usually I'm pretty skeptical of most systems, but I'm rather particularly impressed with yours. I also do not think you are getting the recognition you deserve, because the truly novel work here is Y-JS not IPFS (even though is IPFS is great), yet Y-JS is almost just a footnote on the page which is sad.

The first couple documents (on architecture and stuff) didn't seem to explore the CRDT that is used. I'll have to follow-through on other sub-documents/pages to read more, but wanted to get your thoughts first (and BTW, thanks for all the documentation you have already provided, it is impressive, and I know how hard it is to try and get everything shipped, so the last thing I want to be is an annoying HN person whining about why you didn't already have X or Y or Z finished, and wanted to give you a chance to reply directly).

I was able to rather quickly ( https://www.dropbox.com/s/8uxzq6d47szu4w1/Screenshot%202017-... ) get the document to go out of sync, but I understand this could just be a trivial implementation detail (and often is) NOT a problem with the CRDT, but wanted to check. Given your bachelor thesis, I know you are all too aware of these things, P2P collaborative rich-text is one of the hardest problems out there in the space. Your Y-JS system seems fairly "generalizable", no? Which is unique amongst CRDTs (ours at gun is as well, so I get the nuance). If that is true, usually P2P rich-text then requires a specialized CRDT on top of the generalizable one, is this what you all did?

For instance, Martin Kleppmann did an excellent explanation of explaining theirs here: https://youtu.be/yCcWpzY8dIA?t=16m26s . I was wondering if you did something similar?

We had to implement our own, which I did one of my infamous "cartoon explainers" on here: http://gun.js.org/explainers/school/class.html . However Martin actually brings up an edge case I hadn't thought of after 4 years of analyzing the system! But thankfully he provides a patch on how to fix it. ;)

Anyways, wanted to again repeat I'm very impressed with Y-JS (and say that it is not getting the attention it deserves for this being on top of HN, given that it is what is novel, not the other stuff). But I also wanted to inquire more about the generalizable-ness of Y-JS and then hear about any specific specialized-CRDT you may be using for P2P rich text, because it looks like it could be a breakthrough! Would love to hear more! Excellent work.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#17

Post by stebalien »

mbar wrote: post #13

I see these websocket connections in my network panel. wss://sfo-3.bootstrap.libp2p.io/ wss://nyc-1.bootstrap.libp2p.io/ wss://nyc-2.bootstrap.libp2p.io/ wss://ams-1.bootstrap.libp2p.io/ wss://lon-1.bootstrap.libp2p.io/ I'm going to guess this is to establish the connections between peers. So this isn't really p2p because while there aren't any intermediaries for data transfer, there certainly are to establish the da…

> If you want a p2p system because you care mostly about removing servers as a bottleneck, then this may not be an issue, but if you care about removing servers because you don't trust intermediaries, then this only gets you half way there.

This is, actually, truly peer-to-peer. Those are connections to the bootstrap nodes, you can set your own bootstrap nodes if you want but you do need to be seeded with a few initial peers to find new peers. If you want to establish a browser-to-browser connection, you will need some server to help setup the WebRTC connection; this is the unfortunate reality of browsers. However, you can (theoretically^) connect to arbitrary non-browser IPFS nodes over a websocket connection (that's what's happening here) without some "blessed" node acting as the rendezvous.

^Unfortunately, there's a bit of a wrinkle:

1. Most non-browser IPFS nodes don't listen on websockets by default. The transport is still experimental and and has some bugs.

2. If you load IPFS from an https origin, browsers won't generally allow you to establish connections to non-https websocket endpoints. Unfortunately, even if you enable listening on the websocket transport on your IPFS node, you'll end up listening on an http (not https) websocket. This is because IPFS manages the encryption of the connection itself (IPFS addresses, or "peer IDs", are cryptographic hashes of public keys) and doesn't use (or play well with) the CA system. To work around this, the bootstrap nodes have nginx proxies out in front that to handle HTTPs connections. Most (all other?) nodes don't.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#18

Post by marknadal »

mbar wrote: post #13

I see these websocket connections in my network panel. wss://sfo-3.bootstrap.libp2p.io/ wss://nyc-1.bootstrap.libp2p.io/ wss://nyc-2.bootstrap.libp2p.io/ wss://ams-1.bootstrap.libp2p.io/ wss://lon-1.bootstrap.libp2p.io/ I'm going to guess this is to establish the connections between peers. So this isn't really p2p because while there aren't any intermediaries for data transfer, there certainly are to establish the da…

Unfortunately browser technology sucks, WebRTC still requires a server to bootstrap the connections. It is incorrect to criticize this as not being P2P just because of existing limitations which immediately are removed if you were to run the exact same app in Electron or something. The best thing we can do is plead with browser vendors to improve this.

Re: PeerPad – A realtime P2P collaborative editing tool powered by IPFS

#19

Post by lgierth »

Note that https://peerpad.net is hosted right out of a go-ipfs gateway. You can instead also use https://ipfs.io/ipns/peerpad.net or http://localhost:8080/ipns/peerpad.net, or use a specific version of it:

    > open https://ipfs.io$(ipfs resolve /ipns/peerpad.net)
Post reply on HN