Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The BitTorrent Protocol Specification v2

bittorrent.org

Post reply on HN
Print view View original

101–110 of 114 posts

Re: The BitTorrent Protocol Specification v2

#101

Post by infogulch »

phire wrote: post #24
Scaevolus wrote: post #16

1) Chunks don't span files . Each file is validated by the hash of its merkle tree. This is the biggest user-visible change, since it means you can download one file without downloading others. 2) SHA1 is replaced with SHA2-256 (2x longer hashes and not broken). 3) Files are represented by a tree structure instead of a list of dictionaries with paths-- this reduces duplication in deeply-nested hierarchies. 4) Backwar…

Per-file metadata increases significantly, but it gets rid of the per piece data (which in bittorrent v1 is 20 bytes of sha1 hash per piece and made up the bulk of the .torrent file). The .torrent file only stores the merkle tree's root hash for each file, and the torrent client will query it's peers to get the rest of the merkle tree (verifiable against the root hash). The leafs of the merkle tree are the hash of ea…

> You can scan torrent files for duplicated files and download common files from multiple swarms.

This is one of the biggest things I feel is missing from the current protocol and I'm very glad it's in v2 draft. Now when a group of related torrents are repacked into a single torrent all the swarms are complementary instead of competitive. You don't have to choose between seeding the big pack instead of the individual files, just do what you want and the whole swarm still benefits.

Re: The BitTorrent Protocol Specification v2

#102

Post by infogulch »

caf wrote: post #49
jacksonsabey wrote: post #27

Earlier quoted context omitted.

would 2 different torrents that contain a file with the same exact physical hash share the same torrent file hash? if torrent A and B both contain the exact same file, but torrent A only has the first half available, and torrent B has the second half available, could I combine both torrents to download that file? this could help fix old dead torrents or at least make the file searchable elsewhere by it's sha256 for e…

As long as they also have the same piece size.

Not necessarily. If you have a smart client it should be able to combine them at the file-level since individual files are hashed as a whole now, regardless of the block size within it.

Re: The BitTorrent Protocol Specification v2

#103

Post by Piskvorrr »

Klathmon wrote: post #85
Piskvorrr wrote: post #82

Earlier quoted context omitted.

You will see it explode indeed (where "it" means "the browser" and "explode" means "tab gets throttled and/or crashes"). I understand the motivations for everything-in-a-browser, but this approach has major externalities - e.g. the client's power, both as in CPU and as in battery.

Something like a torrent network is a bunch of IO with some processing which fits perfectly with javascript. WebAssembly might be able to get around some of the performance issues in specific areas (like some of the more complicated hashing stuff) if there are some, but it's more than possible to write performant code in javsacript.

It might be possible, but I have yet to see it. So far, the performance increase has been thanks to the engine executing such code, despite the opposite effort ;) of the JS code.

Re: The BitTorrent Protocol Specification v2

#104

Post by ue_ »

wongarsu wrote: post #88
ue_ wrote: post #77

Earlier quoted context omitted.

I wanted to implement a distribured imageboard over bittorrent but I quickly realised it's hard to add data after the initial publication, and further to verify it, and the nature of trackers may make it prone to censorship. So I gave up.

Distributing the images/posts via bittorrent and the relations between them in the DHT might be the way to go with such a project. On the other hand, the an uncensorable imageboard would profit from the verifiable timestamping of a blockchain, with just the images distributed via a bittorrent-like mechanism. That also gives you a decent anti-spam mechanism (you can post in exchange for mining blocks, similar to the o…

I thought about this too, but one of the features of imageboards is that it doesn't splinter into subthreads, there's a big list of posts, unlike Reddit. And because a post can reply to multiple posts at once, you can't separate them in blockchain forks. If the blockchain forks, it becomes hard to reference posts in the other forks from any particular fork.

On the other hand, there has to be a way to avoid downloading (and sharing) certain parts of the chain, for example if someone uploads illegal content, they should have the option to never download that data, so I like the idea of keeping images separate.

For posting to be feasible, the time to mine has to be low, though of course it'll increase over time, meaning that either shorter blockchains are favoured for ease of use (nobody wants to wait 5 minutes and waste a lot of power just to make a post) but long enough to make them hard to forge.

There's also the issue of segmentation; there's an interest in certain users wanting not to share certain posts, for example people against political issue X may not want to share posts about X. With a small number of peers, this could mean that only one or two peers keeps track of the posts talking about issue X. And then you'd have to trust that you're not downloading illegal content from those people, so if you are committed to anti-censorship but also don't want to download illegal content, you have to trust those peers to only remove illegal content.

In the end, I'm not sure if it comes out better than NNTP, or even centralised discussion boards with multiple independent archive sites available (which can archive posts before they are deleted by moderators).

Re: The BitTorrent Protocol Specification v2

#105

Post by jorvi »

Klathmon wrote: post #84
lqdc13 wrote: post #80

Earlier quoted context omitted.

https://webtorrent.io/

sadly I believe that relies on special software bridges or gateways to talk to "true bittorrent networks".

Vuze already has a supporting plugin, and there's been talk for Transmission to include it. I'm not entirely sure, but I think that if 1 person has a compatible client it acts as a bridge connecting the two types of swarms via DHT!

Re: The BitTorrent Protocol Specification v2

#106

Post by kyberias »

pwg wrote: post #92
kyberias wrote: post #61

Earlier quoted context omitted.

uTorrent used to be really efficient, small memory-footprint, full featured bittorrent-client. One of the best software I've ever used to download... perfectly legal content [1] from the internets. Now it's full of ads and performs poorly. [1] Like all the different Linux distro install images over and over again.

Take a look at rtorrent: https://github.com/rakshasa/rtorrent It fits the efficient, small memory-footprint and no ads requirements. "Full Featured" is subjective as it depends upon what you consider "Full Featured".

Full featured means that it efficiently implements all the bells and whistles of the most recent bittorrent protocol. And only that, nothing extra.

Re: The BitTorrent Protocol Specification v2

#107

Post by johnwaynedoe »

boyce wrote: post #72
johnwaynedoe wrote: post #70

Earlier quoted context omitted.

The Brave browser has a bittorrent add-on pre-installed. I've been toying with it as my daily driver for a couple weeks and have been really enjoying it. The project still needs work but I love what they're doing. https://en.wikipedia.org/wiki/Brave_(web_browser) https://brave.com/

What's your job there

I wish I worked there, I do miserable tier 1 tech support. The reason I am promoting their browser is because it has the exact functionality the parent comment was looking for. Its also a really neat project by a former Firefox dev. It's a browser made to inherently block ads and tracking . Its essentially seamlessly does what several add-ons do for me in Firefox. I'm just excited to see a browser take tracking ads, fingerprinting and other browser privacy seriously.

Re: The BitTorrent Protocol Specification v2

#108

Post by rmc »

Klathmon wrote: post #71

I have to admit, BitTorrent is one of the things I took for granted. I never really thought about the details of how it works, or the really really impressive feats that were accomplished to get it to work. I knew it was a really good technology, but reading this and the comments here puts it on a whole other level. Why isn't this technology talked about more? Why are blockchains the big "thing" right now with people…

I suspect the close association with copyright infringement means that BitTorrent is a little toxic for many corporations.

Re: The BitTorrent Protocol Specification v2

#109

Post by rmc »

ue_ wrote: post #77
Klathmon wrote: post #71

I have to admit, BitTorrent is one of the things I took for granted. I never really thought about the details of how it works, or the really really impressive feats that were accomplished to get it to work. I knew it was a really good technology, but reading this and the comments here puts it on a whole other level. Why isn't this technology talked about more? Why are blockchains the big "thing" right now with people…

I wanted to implement a distribured imageboard over bittorrent but I quickly realised it's hard to add data after the initial publication, and further to verify it, and the nature of trackers may make it prone to censorship. So I gave up.

Have you seen ipfs? It might do part of what you want...

Re: The BitTorrent Protocol Specification v2

#110

Post by baby »

snakeanus wrote: post #93
baby wrote: post #65

Earlier quoted context omitted.

hopefully SHA-3 (or better, the Keccak permutation) is implemented in hardware first.

> SHA-3 (or better, the Keccak permutation) SHA-3 is just Keccak with specific capacities. If you mean that they should not implement only the standard SHA-3 functions then I agree, having c/raw/SHAKE256 as well would be nice.

Implementing the permutation would be even better, because other stuff like Strobe, Keyak, Ketje, etc... could use it.
Post reply on HN