Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The BitTorrent Protocol Specification v2

bittorrent.org

Post reply on HN
Print view View original

51–60 of 114 posts

Re: The BitTorrent Protocol Specification v2

#51

Post by loup-vaillant »

derefr wrote: post #44
loup-vaillant wrote: post #42

Earlier quoted context omitted.

There are better ways to prevent length extension attacks, such as choosing Blake2b. With the current scheme, only 12 bytes are missing, so Length extension attacks only get 96 bits of security… Replacing SHA-1 with SHA-2, what are they thinking ? Blake2 is faster and more secure than either.

They're probably thinking something about which hashes will get hardware-accelerated instructions provided on x86 and ARM.

Yeah, I keep forgetting about Hardware acceleration…

Re: The BitTorrent Protocol Specification v2

#52

Post by the8472 »

userbinator wrote: post #15
jzelinskie wrote: post #6

This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…

SHA256 truncated to 20 bytes What are the security implications of doing this? It seems it wouldn't increase the strength beyond the original 160 bits, no? Was there anything preventing redesigning the protocol to use full 32-byte SHA256 hashes throughout?

There are 2 uses of the hashes in bittorrent. a) integrity checking b) as unique identifier for the swarm.

The hash only gets truncated when used in places as unique identifier. When you start with a v2 magnet link or torrent file you get the full 32bytes hash, which means your integrity-checking is unaffected.

Re: The BitTorrent Protocol Specification v2

#53

Post by the8472 »

predakanga wrote: post #35
the8472 wrote: post #10

Earlier quoted context omitted.

The DHT BEPs specify a network that is only barely related to the bittorrent core protocol, they can already be used independently, and some people do. > I feel like trackers were largely overlooked in this update, but I'm biased because I work on a popular tracker. Yes, we did not pay much attention to trackers, but BEP52 basically seized the opportunity to do some incompatible changes we always wanted to do anyway…

How do you intend to handle incompatibilities in the tracker protocol that are introduced by BEP52? In particular, I note that there's nothing in there regarding which infohash should be used in the tracker updates. Should traffic with v1/v2 clients be reported separately, or should it be consolidated under the v2 infohash?

hybrid torrents essentially consist of two swarms, so you announce twice.

Re: The BitTorrent Protocol Specification v2

#54

Post by the8472 »

jzelinskie wrote: post #17
the8472 wrote: post #10

Earlier quoted context omitted.

The DHT BEPs specify a network that is only barely related to the bittorrent core protocol, they can already be used independently, and some people do. > I feel like trackers were largely overlooked in this update, but I'm biased because I work on a popular tracker. Yes, we did not pay much attention to trackers, but BEP52 basically seized the opportunity to do some incompatible changes we always wanted to do anyway…

>...and there were no such open issues with the http tracker protocol This is because the HTTP protocol is so much overhead that most trackers don't even really run it anymore. I think UDP being promoted to the spec would've been a step in the right direction. Modern trackers have a bunch of tricks like BEP34[0] to avoid getting pounded that would be great if every client conformed to. I hope I'm not coming off as ag…

I still don't see why this would need to piggyback on a breaking change of the metadata file format. "Please use UDP by default" is fairly orthogonal to the metadata format and could be added to the spec at any point if we want to.

HTTP trackers are considered fine for medium-scale torrent deployments. UDP trackers were originally introduced to cope with the traffic caused by running an open tracker that manages 100k+ infohashes for the whole world.

Also, both BEP3 and 52 already forward-reference the tracker extensions (compact and UDP), so someone who writes a new bittorrent implementation should already be aware of them.

Maybe we could make it more clear that some BEPs are almost-mandatory.

Re: The BitTorrent Protocol Specification v2

#55

Post by nenreme »

the8472 wrote: post #52
userbinator wrote: post #15

Earlier quoted context omitted.

SHA256 truncated to 20 bytes What are the security implications of doing this? It seems it wouldn't increase the strength beyond the original 160 bits, no? Was there anything preventing redesigning the protocol to use full 32-byte SHA256 hashes throughout?

There are 2 uses of the hashes in bittorrent. a) integrity checking b) as unique identifier for the swarm. The hash only gets truncated when used in places as unique identifier. When you start with a v2 magnet link or torrent file you get the full 32bytes hash, which means your integrity-checking is unaffected.

You download 32 bytes hash and check it using 20 byte hash from the magnet link.

Re: The BitTorrent Protocol Specification v2

#56

Post by the8472 »

Scaevolus wrote: post #39
phire wrote: post #24

Earlier quoted context omitted.

Per-file metadata increases significantly, but it gets rid of the per piece data (which in bittorrent v1 is 20 bytes of sha1 hash per piece and made up the bulk of the .torrent file). The .torrent file only stores the merkle tree's root hash for each file, and the torrent client will query it's peers to get the rest of the merkle tree (verifiable against the root hash). The leafs of the merkle tree are the hash of ea…

Right, in BitTorrent v1 the size of the .torrent file is O(number of files) + O(number of bytes), but with this it's just O(number of files) with a higher constant factor. Piece size is still baked into the file (as piece length), and is used for presence bitsets, which are a crucial part of the swarm algorithm. Clients download the rarest pieces first to boost efficiency, and this information is handled as bitsets s…

v1: O(path-depth * number of files + number of bytes)

v2: O(log(path-depth) * number of files)

that is assuming some constantish branching factor in your directory structure

> Merkle tree roots will only be unique for each piece length.

Merkle trees are independent of piece size, which means you can use them to dedup across torrents.

Re: The BitTorrent Protocol Specification v2

#57

Post by the8472 »

nenreme wrote: post #55
the8472 wrote: post #52

Earlier quoted context omitted.

There are 2 uses of the hashes in bittorrent. a) integrity checking b) as unique identifier for the swarm. The hash only gets truncated when used in places as unique identifier. When you start with a v2 magnet link or torrent file you get the full 32bytes hash, which means your integrity-checking is unaffected.

You download 32 bytes hash and check it using 20 byte hash from the magnet link.

http://bittorrent.org/beps/bep_0009.html#magnet-uri-format

Re: The BitTorrent Protocol Specification v2

#58

Post by the8472 »

mtgx wrote: post #40
jzelinskie wrote: post #6

This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…

The word encryption doesn't even seem to be mentioned there. At the very least it would help against traffic shaping, which you know is coming once net neutrality rules are dead.

Bittorrent already had an obfuscation protocol wrapper for years. It was effective for a while but the companies that implement traffic shaping equipment stepped up their game and probably rely on traffic flow matching now.

It is an arms race that is not won by updating a slowly evolving core protocol.

Re: The BitTorrent Protocol Specification v2

#59

Post by lucasverra »

nyolfen wrote: post #36
smegel wrote: post #34

Earlier quoted context omitted.

Thanks I haven't hear of that. And it is written in C++ which is nice. Is it considered the spiritual successor to the original uTorrent?

it is: >The qBittorrent project aims to provide an open-source software alternative to µTorrent. though in my experience it is more of a memory hog and buggier than utorrent. but that doesn't stop me from using it

I've switched to transmission client and never looked back (4 yrs ago)

Re: The BitTorrent Protocol Specification v2

#60

Post by phire »

nyolfen wrote: post #36
smegel wrote: post #34

Earlier quoted context omitted.

Thanks I haven't hear of that. And it is written in C++ which is nice. Is it considered the spiritual successor to the original uTorrent?

it is: >The qBittorrent project aims to provide an open-source software alternative to µTorrent. though in my experience it is more of a memory hog and buggier than utorrent. but that doesn't stop me from using it

It's nowhere near as perfect as utorrent once was. But it's pretty good and I have plenty of extra memory (it's currently using 305mb while seeding 40+ torrents, dropping down to 65mb if I pause them all.)
Post reply on HN