Would it be possible to build a service that resembles email using this technology? Like dropping a message file within a local folder and making it appear on someone else's local folder?
Hyperdrive v10 – a peer-to-peer filesystem
61–70 of 124 posts
Re: Hyperdrive v10 – a peer-to-peer filesystem
#62Hyperdrive is part of the peer-to-peer networking stack that Beaker Browser uses for websites. Today is release day and this tech update is the first announcement.
What is your relationship with Hyperdrive & the Beaker Browser?
The Beaker team works on Hyperdrive (the Hypercore Protocol) but we maintain a separate org at https://github.com/hypercore-protocol
Re: Hyperdrive v10 – a peer-to-peer filesystem
#63Earlier quoted context omitted.
Do you have any good solutions for this exact use case yet?
Checked Tahoe-LAFS, Perkeep, but they are too low level. Haven't tried Resilio Connect yet, although I would prefer something open-sourced. I think we have all the tech needed already (WebRTC, DHT, NAT hole punching, decades of p2p, encryption, onion security, StorJ/Filecoin) etc but what is lacking is dead simple UX and wide support of operating systems - Windows, Mac, Android, iOS, Linux (Raspberry, Synology, cheap…
Yeah there are no turnkey solutions that I'm aware of. Which is why every time there's a thread like this I come to see what the rumpus is.
I'm still waiting for a 'Drobo' like device without the proprietary physical layout. A light or dial goes into orange territory, you head to Best Buy or Amazon and buy the biggest drive that doesn't give you sticker shock, you push a button, out pops your worst drive and in goes the new one. Some lights flicker for a while and then go green.
I thought ZFS would have given us almost everything but the hardware ten years ago, but it turned out they oversold a few of the features back then, and then Larry happened.
Custom hardware is too expensive for small run consumer hardware, and Apple might have gotten into that space but never did. I wonder how many PCIe lanes you could shoehorn onto a Pi clone...
Re: Hyperdrive v10 – a peer-to-peer filesystem
#64Earlier quoted context omitted.
Thanks! - "Hyperdrive" was formerly the internal data-structure name of Dat archives. With this release, the team decided to rename the protocol from Dat to Hypercore Protocol. Subsequently, "Dat Archives" are now "Hyperdrives." The Dat community will post some updates about this soon. - The hyperdrive-daemon does not support dat:// URLs. I don't know what the future of dat:// URLs will be but Beaker is phasing them…
Another comment which is now gone asked what your relation to the Beaker project is, and how Hyperdrive is related to the Beaker browser?
Just replied there -
Beaker uses Hyperdrive, it's basically the source of its novel features. The Beaker team works on Hyperdrive (the Hypercore Protocol) but we maintain a separate org at https://github.com/hypercore-protocol
Re: Hyperdrive v10 – a peer-to-peer filesystem
#65Would it be possible to build a service that resembles email using this technology? Like dropping a message file within a local folder and making it appear on someone else's local folder?
@pfrazee threw together a demo app called PaulMail [1] that you can view in Beaker [0]. It's very simple and is just a proof-of-concept (messages are unencrypted, for example), but it does hint at possibilities there.
[1] hyper://1bc1faf01a22270fb5698a60e63ef7a596ad976457e6d9914a8fd56d87281917/
Re: Hyperdrive v10 – a peer-to-peer filesystem
#66Earlier quoted context omitted.
So do you opportunistically gather info about what other peers have via this gossip protocol, or just when you need something? I had looked at https://datprotocol.github.io/how-dat-works/ , but I don't remember anything about a gossip protocol or a peer building a "view of the world". Is that new?
It's only between the peers in your subset of the swarm for now. They exchange a series of WANT and HAVE messages where they subscribe to the sections of each others logs they are interested in. We are working on expanding this scheme so peers can help discover peers that have the section you are looking for. Due to the compressed bitfields these section are quite large. In most cases using a few kilobytes you can sh…
Yes, I saw the compressed bitfields. Most amount of bit twiddling I have ever seen in a pure javascript library...
Being able to identify a piece of content by an integer instead of a hash makes things more efficient compared to content-addressed storage a la IPFS.
Re: Hyperdrive v10 – a peer-to-peer filesystem
#67How do you handle merge conflicts? Since you aren't calling it a blockchain I take it that Hypercore is targeted at the trusted (writers) and semi-trusted (readers) use case?
Hypercore is a single writer append-only log. The website has a bit more info about how it works, but's basically a merkle log signed by a private key / public key scheme. We build collaborative data structures by combining multiple Hypercores.
Hyperdrives builds a p2p filesystem on top of Hypercore for a single writer. Using mounts you can mount other peoples drives so merge conflicts don't happen since there is no overlapping writes.
We are working on a union mount approach as well for overlapping drives (we talk a bit about this in the post)
Re: Hyperdrive v10 – a peer-to-peer filesystem
#68Quick note: it would be nice to ship the hyperdrive daemon as something more self-contained than a npm package.
I tried to install it and got some c compile errors. Probably my nodejs version is too old, but you want to have the absolute minimum of friction for users to install it.
Beaker comes as an appimage for linux, which worked flawlessly at the first try. Maybe do that for the hyperdrive daemon as well?
Re: Hyperdrive v10 – a peer-to-peer filesystem
#69Would it be possible to build a service that resembles email using this technology? Like dropping a message file within a local folder and making it appear on someone else's local folder?
@pfrazee threw together a demo app called PaulMail [1] that you can view in Beaker [0]. It's very simple and is just a proof-of-concept (messages are unencrypted, for example), but it does hint at possibilities there. [0] https://beakerbrowser.com [1] hyper://1bc1faf01a22270fb5698a60e63ef7a596ad976457e6d9914a8fd56d87281917/
Your recipients also need to be in your beaker address book for it work. It's a very rough PoC.
Re: Hyperdrive v10 – a peer-to-peer filesystem
#70Quick note: it would be nice to ship the hyperdrive daemon as something more self-contained than a npm package. I tried to install it and got some c compile errors. Probably my nodejs version is too old, but you want to have the absolute minimum of friction for users to install it. Beaker comes as an appimage for linux, which worked flawlessly at the first try. Maybe do that for the hyperdrive daemon as well?
Yes, we'll definitely be doing this. Thanks for the feedback.