Live data from Hacker News

Hyperdrive v10 – a peer-to-peer filesystem

blog.hypercore-protocol.org

41–50 of 124 posts

Re: Hyperdrive v10 – a peer-to-peer filesystem

#41
post #34

Does it support private clustering and encryption? My use case is to have shared volumes between servers, laptops but without all the networking hassle (VPNs, opening firewalls).

My go-to example for 'private cloud' (which as soon as we hit the Trough of Disillusionment on this Hype Train, we will all be talking about) is having a set of people in your social or family group maintaining a file share for your semi-private photos and videos.

You don't have to worry about a flood or tornado taking out all the photos of first birthdays, or Grandma when she was younger than you are now, and you also don't broadcast family dynamics, whereabouts (people have been robbed when social media made it clear they were not at home) or pictures of minors.

You, your uncle who Moved to the City and your cousin by marriage who wants to be a game designer all set up a file server and share the photos (your cousin is gonna throttle traffic while he's playing CoD or Dota of course, which it turns out he does all the time but at least he's an extra backup copy).

I don't know how you keep that one relative from uploading funny things they found on the internet that keep trying to install spyware or back doors though, but I suppose you'd have that problem now.

Re: Hyperdrive v10 – a peer-to-peer filesystem

#42
post #39

I've been following dat since a few years and I've been hearing about multiwriter since a few years but not anymore. Is it still something you are working on ?

Yes we are continuously exploring and research this. The mount support is our first stepping stone towards this. See the union mounts section of the post

Re: Hyperdrive v10 – a peer-to-peer filesystem

#43
post #40

Earlier quoted context omitted.

It should work on 14. Could you open an issue on the repository?

filed hyperdrive-daemon #47 problem with hyperdrive-daemon/node_modules/fuse-native/prebuilds/linux-x64/node.napi.node

Thanks, appreciate it

Re: Hyperdrive v10 – a peer-to-peer filesystem

#44
post #41
post #34

Does it support private clustering and encryption? My use case is to have shared volumes between servers, laptops but without all the networking hassle (VPNs, opening firewalls).

My go-to example for 'private cloud' (which as soon as we hit the Trough of Disillusionment on this Hype Train, we will all be talking about) is having a set of people in your social or family group maintaining a file share for your semi-private photos and videos. You don't have to worry about a flood or tornado taking out all the photos of first birthdays, or Grandma when she was younger than you are now, and you al…

Do you have any good solutions for this exact use case yet?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#45
post #30
post #19

Earlier quoted context omitted.

Wow. What a way to describe the tradeoff of a programming language. No, thanks

JavaScript is a programming language. NodeJS is not. It's the substrate that the NPM ecosystem runs on. choeger's comment very well may have not been made with a distinction in mind (and I agree that it was a low effort, low signal comment), but the distinction exists. choeger seems to be using Firefox. Firefox is implemented using an Emacs-like architecture. The Firefox codebase includes millions of lines of JS, and…

These days, using JavaScript outside of the browser sort of implies Node.js, in the same way that using Python implies that you're using CPython, unless you specify otherwise. In theory your distinction makes sense, but in practice, very few people are using anything but Node.js for backend JavaScript systems (at least for new projects).

Re: Hyperdrive v10 – a peer-to-peer filesystem

#46

Could this be used to create a kbfs (keybase file system) competitor?

Yea totally. The FUSE support is already very close to being elexactly that. We’ve been using it internally during our beta as an in office P2P dropbox using the mounts feature

In kbfs when I create a shared drive between you and me, the space is automatically available in read and write for both of us. Can this be done directly on top of hyperdrive or do we need to modify how hyperdrive (or the daemon I suppose) work ?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#47
post #34

Does it support private clustering and encryption? My use case is to have shared volumes between servers, laptops but without all the networking hassle (VPNs, opening firewalls).

It uses Noise to encrypt all transport connections and using the api you can whitelist which peers (by their public key) you want to replicate with. This API is not yet exposed through the daemon but it’s in the modules. Hyperdrives also have a built in capability system where you have to know the public key of the drive to download it from a peer, so if you only share the key with yourself no one else can access it.…

[deleted]

Re: Hyperdrive v10 – a peer-to-peer filesystem

#48
post #46

Earlier quoted context omitted.

Yea totally. The FUSE support is already very close to being elexactly that. We’ve been using it internally during our beta as an in office P2P dropbox using the mounts feature

In kbfs when I create a shared drive between you and me, the space is automatically available in read and write for both of us. Can this be done directly on top of hyperdrive or do we need to modify how hyperdrive (or the daemon I suppose) work ?

We can mount each others drives and collab that way.

For a full “union” mount experience we still have some research to do but we are working on it.

The mount setup is really good though and fully p2p

Re: Hyperdrive v10 – a peer-to-peer filesystem

#49
Congrats for the release. Mostly familiar with IPFS, but learned about DAT at dtn.is last year. The hypercore protocol is pretty well designed and documented.

How would hyperdrive deal with the following scenario: you got a large dataset such as wikipedia. Lots of people have browsed it, but most of them only have a tiny fraction.

How do yo know which peers to connect to to get a particular bit (offset?) you are interested in? The DHT only tells you which nodes participate in the hypercore, not what they have in detail, right?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#50
post #49

Congrats for the release. Mostly familiar with IPFS, but learned about DAT at dtn.is last year. The hypercore protocol is pretty well designed and documented. How would hyperdrive deal with the following scenario: you got a large dataset such as wikipedia. Lots of people have browsed it, but most of them only have a tiny fraction. How do yo know which peers to connect to to get a particular bit (offset?) you are inte…

The peers gossip using compressed bitfields in regards to what data they have. These bitfields are super small so we can pack quite a bit of information.

At the moment we don't do anything special in regards to discovery, but as we scale that's something we want to investigate. Since everything is running on append-only logs we can group the data into sections quite easily so there is some easy wins we can do there with announcing to the dht that you have data in a specific region.

Post reply on HN