Do all Bittorrent clients support it already?
The BitTorrent Protocol Specification v2
11–20 of 114 posts
Re: The BitTorrent Protocol Specification v2
#12What's the stuff about "proof layers", is that new in this v2? The paper briefly talks about proof layer requests. Is this something merkle-tree related? What is the purpose? Is it to prevent clients from lying about having pieces they do not have by requesting a verifiable random hash chunk?
It's part of switching to merkle trees instead of flat piece lists. A merkle tree can only be verified if you either have a whole layer or send ancestor-siblings (uncle, great-uncle, etc.) along with a partial layer.
Merkle trees allow torrents to start faster from magnet links since only the tree roots need to be front-loaded while the tree can be fetched incrementally.
Re: The BitTorrent Protocol Specification v2
#13Rationale for hash function change: https://github.com/bittorrent/bittorrent.org/issues/58
Discussion of other changes: https://github.com/bittorrent/bittorrent.org/pull/59
Re: The BitTorrent Protocol Specification v2
#14Earlier quoted context omitted.
The main differences I can see is a change from SHA1 -> SHA2 and also seems to have added official spec for webtorrent.
It also appears to be using a merkle hash tree for piece hashing now along with a few new peer wire messages to support that.
It also switches to a merkle hash per-file and specifies that each file is aligned to the start of a piece, and the size of the last piece of each file matches the amount of remaining data.
This means in large multi-file torrents you don't have to download (and store) the two extra 1-4mb pieces at the start/end of each file anymore.
Re: The BitTorrent Protocol Specification v2
#15This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…
SHA256 truncated to 20 bytes
What are the security implications of doing this? It seems it wouldn't increase the strength beyond the original 160 bits, no? Was there anything preventing redesigning the protocol to use full 32-byte SHA256 hashes throughout?
Re: The BitTorrent Protocol Specification v2
#161) Chunks don't span files. Each file is validated by the hash of its merkle tree. This is the biggest user-visible change, since it means you can download one file without downloading others.
2) SHA1 is replaced with SHA2-256 (2x longer hashes and not broken).
3) Files are represented by a tree structure instead of a list of dictionaries with paths-- this reduces duplication in deeply-nested hierarchies.
4) Backwards compatible-- you can make a .torrent file with both old and new pieces, and a swarm can speak either. This requires padding files from BEP47, which most clients probably don't support.
Per-file metadata increases pretty significantly, from ~19B (just length) to ~68B (length + hash).
Re: The BitTorrent Protocol Specification v2
#17This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…
The DHT BEPs specify a network that is only barely related to the bittorrent core protocol, they can already be used independently, and some people do. > I feel like trackers were largely overlooked in this update, but I'm biased because I work on a popular tracker. Yes, we did not pay much attention to trackers, but BEP52 basically seized the opportunity to do some incompatible changes we always wanted to do anyway…
>...and there were no such open issues with the http tracker protocol
This is because the HTTP protocol is so much overhead that most trackers don't even really run it anymore. I think UDP being promoted to the spec would've been a step in the right direction. Modern trackers have a bunch of tricks like BEP34[0] to avoid getting pounded that would be great if every client conformed to.
I hope I'm not coming off as aggressive. I really appreciate this work and I'm really glad to see a spec revision. It's just as you said, there's been many years and many good improvements that I'd like to see made while there's still a change to break things.
Re: The BitTorrent Protocol Specification v2
#18This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…
SHA256 truncated to 20 bytes What are the security implications of doing this? It seems it wouldn't increase the strength beyond the original 160 bits, no? Was there anything preventing redesigning the protocol to use full 32-byte SHA256 hashes throughout?
The problem with SHA1 being broken is unrelated to the amount of bits in the hash, so the point of switching to sha256 isn't to "increase the strength beyond the original 160 bits" but to avoid Shattered and future potential attacks.
Re: The BitTorrent Protocol Specification v2
#19This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT. Beyond the hashing algorithm, some important additions that were previously propos…
> They've decided to move forward with SHA256 truncated to 20 bytes [...]
I wonder why not SHA512? It's actually faster to compute than SHA512 on 64-bit architectures.
Re: The BitTorrent Protocol Specification v2
#20Do all Bittorrent clients support it already?
Currently no bittorrent clients support it. This is still just a draft. I've only just started working on an implementation for libtorrent, it will be quite a while before it is production ready.