Hyperdrive v10 – a peer-to-peer filesystem • Hacker News Board

Re: Hyperdrive v10 – a peer-to-peer filesystem

#21

Post by alphydan » Thu, May 14, 2020, 6:20 PM UTC

Could this be used to create a kbfs (keybase file system) competitor?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#22

Post by jhoechtl » Thu, May 14, 2020, 6:27 PM UTC

Can it be that you moved from the early days of dat which was written in Go afaik to primarily node/js? Why the move?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#23

Post by 0x006A » Thu, May 14, 2020, 6:28 PM UTC

mafintosh wrote: post #20

0x006A wrote: post #17

does it scale now? previously i was not able to import large folders (several TB) or many files (+100k files)

Yep! That is one of the things we’ve worked the hardest on. Completely new indexing structure, using an append-only hash trie which scales really well. We’ve tested it with many big datasets including importing all of Wikipedia as files in a single folder. Worked like a charm :)

this one? https://dumps.wikimedia.org/other/static_html_dumps/current/... how long does it take to import it?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#24

Post by pfraze » Thu, May 14, 2020, 6:29 PM UTC

jhoechtl wrote: post #22

Can it be that you moved from the early days of dat which was written in Go afaik to primarily node/js? Why the move?

I think dat has always been node/js

Re: Hyperdrive v10 – a peer-to-peer filesystem

#25

Post by aspenmayer » Thu, May 14, 2020, 6:41 PM UTC

pfraze wrote: post #13

rakoo wrote: post #8

Congratulations to the team ! 2 questions: - What is the difference between dat and hyperdrive ? - I see hyperdrive manages hyper:// urls -- what about dat:// urls ? Will they be still managed ?

Thanks! - "Hyperdrive" was formerly the internal data-structure name of Dat archives. With this release, the team decided to rename the protocol from Dat to Hypercore Protocol. Subsequently, "Dat Archives" are now "Hyperdrives." The Dat community will post some updates about this soon. - The hyperdrive-daemon does not support dat:// URLs. I don't know what the future of dat:// URLs will be but Beaker is phasing them…

Another comment which is now gone asked what your relation to the Beaker project is, and how Hyperdrive is related to the Beaker browser?

Re: Hyperdrive v10 – a peer-to-peer filesystem

#26

Post by blurbleblurble » Thu, May 14, 2020, 6:41 PM UTC

billme wrote: post #7

pfraze wrote: post #2

Hyperdrive is part of the peer-to-peer networking stack that Beaker Browser uses for websites. Today is release day and this tech update is the first announcement.

What is your relationship with Hyperdrive & the Beaker Browser?

He lives inside of a beaker browser. It's his home.

Re: Hyperdrive v10 – a peer-to-peer filesystem

#27

Post by Mizza » Thu, May 14, 2020, 6:43 PM UTC

Been following this for years, congratulations on your release. I looked at dat for building P2P services, but found that ZeroNet was actually more capable for "real" services - maybe now is the time to reevaluate that, especially given the improvements to large (both deep and wide) archives and the new hole punching.

Can you please mention your thoughts on:

- Discoverability of content in Hyperswarm (DHT Search/"Superpeers"/???)

- What happens to the old DEP proposals? (There is a critical feature that I need for my service that's still an open DEP proposal! https://github.com/datprotocol/DEPs/issues/61)

- What uses cases do you have in mind for this service?

Thanks!

Re: Hyperdrive v10 – a peer-to-peer filesystem

#28

Post by 0x006A » Thu, May 14, 2020, 6:43 PM UTC

what version of node.js does it require? i get this with v10.19.0 from Ubuntu 20.04:

    node_modules/hyperdrive-daemon/node_modules/hyperdrive-daemon-client/bin/commands/create.js:8        
      static usage = 'create [path]'                                                                                    
               ^                                                                                                    
                                                                                                                    
    SyntaxError: Unexpected token =

Re: Hyperdrive v10 – a peer-to-peer filesystem

#29

Post by andrewosh » Thu, May 14, 2020, 6:48 PM UTC

0x006A wrote: post #28

what version of node.js does it require? i get this with v10.19.0 from Ubuntu 20.04: node_modules/hyperdrive-daemon/node_modules/hyperdrive-daemon-client/bin/commands/create.js:8 static usage = 'create [path]' ^ SyntaxError: Unexpected token =

Oops, give it a shot in v12. Think static class properties were added after 10.

Re: Hyperdrive v10 – a peer-to-peer filesystem

#30

Post by cxr » Thu, May 14, 2020, 6:54 PM UTC

dsun179 wrote: post #19

choeger wrote: post #14

> written in Node.js Yeah. No, thanks.

Wow. What a way to describe the tradeoff of a programming language. No, thanks

JavaScript is a programming language. NodeJS is not. It's the substrate that the NPM ecosystem runs on.

choeger's comment very well may have not been made with a distinction in mind (and I agree that it was a low effort, low signal comment), but the distinction exists.

choeger seems to be using Firefox. Firefox is implemented using an Emacs-like architecture. The Firefox codebase includes millions of lines of JS, and that's been the case since before the language was JITted. I ran Firefox 1.0, 2.5, and 2.0 on an 800MHz PIII with 128 MB (later 192 MB, wow-wee!) of RAM. The JS in the Firefox codebase wasn't written in the NodeJS style, and if it were, that would have made it a non-starter.

Kneejerk JS haters are tiring, but so is the conflation of JS and the NodeJS ecosystem (by NodeJS supporters and NodeJS haters alike).

(Side note: if you insist on giving attention to folks engaged in low-value discussions, try to elevate the conversation to a level that makes it actually worth having.)