Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BitTorrent v2

blog.libtorrent.org

Post reply on HN
Print view View original

361–370 of 576 posts

Re: BitTorrent v2

#361

Post by oconnor663 »

johnisgood wrote: post #151
sleavey wrote: post #71

SHA1 has a collision, so now it uses SHA256. How long until a SHA256 collision? Shouldn't the new protocol just add support for many modern hash functions, and client updates can disable support for hashes that become insecure later? Or does this introduce its own headaches? That's what SSH does, right?

Why not BLAKE3? I am really curious. I mean, since it exists, why not that over SHA256? Because it is relatively new?

In addition to loeg's point about timing, I'll add that BLAKE3 has its own internal tree structure. It would be unfortunate to have two tree structures layered on top of one another, both because it's "ugly" and because it won't do as good a job of parallelizing things. However, unifying the tree structures would be a big commitment. Every detail of the layout would need to be exactly as it is in BLAKE3. There wouldn't be any space for custom metadata on interior tree nodes, for example. I'm not familiar with the protocol details of BitTorrent myself, but I wouldn't be surprised if that unified approach turned out to be too limiting. (But for a file/tree project that does use the exact BLAKE3 structure, see github.com/oconnor663/bao.)

Re: BitTorrent v2

#362

Post by octoberfranklin »

aurelian15 wrote: post #356
sedatk wrote: post #294

Earlier quoted context omitted.

Only if blocks have the same offset in the same binary and if they align with the block boundaries. Otherwise, different hashes would be generated. I don't expect that to happen a lot.

There are ways around this. See "content-aware chunking", e.g. implemented using rolling hashes [1]. This is for example what rsync does. The idea is to make blocks (slightly) variable in size. Block boundaries are determined based on a limited window of preceding bytes. This way a change in one location will only have a limited impact on the following blocks. [1] https://en.wikipedia.org/wiki/Rolling_hash

Rolling hashing is really only useful for finding nonaligned duplicates.

There isn't a way to advertise some "rolling hash value" in a way that allows other people with a differently-aligned copy to notice that you and them have some duplicated byte ranges.

Rolling hashes only work when one person (or two people engaged in a conversation, like rsync) already has both copies.

Re: BitTorrent v2

#363

Post by loeg »

johnisgood wrote: post #235
loeg wrote: post #168

Earlier quoted context omitted.

I would guess that the reasoning is similar to why Git is moving to SHA256 (from SHA1) rather than to BLAKE3 — SHA256 was around 5 years ago and the major design change has been in the works for a while (BEP 52 dates to 2017). BLAKE3 (2019) would be a fine choice today.

I see. Thank you! By the way, I have not thought much about it, so in case you may know: would not it be possible to implement this in a way that allows swapping the hash function? So for example when we run into issues with SHA-256, change the hash function to something else.

We already can: the "swap" will just be a v3 along the same lines.

Re: BitTorrent v2

#364

Post by Hamuko »

oauea wrote: post #197
Hamuko wrote: post #139

Earlier quoted context omitted.

Been a fan of Transmission on macOS for many years. I just wish I could install Transmission on a Linux box and still keep using the fantastic Cocoa interface it has on macOS.

You mean https://github.com/transmission-remote-gui/transgui or another gui? You should be able to connect remotely, that is pretty much the entire idea behind transmission. A torrent daemon with various UIs that use the API.

I mean the native Transmission Cocoa GUI.

Re: BitTorrent v2

#365

Post by Hamuko »

brokenmachine wrote: post #263
Hamuko wrote: post #148

Earlier quoted context omitted.

> I use around ~200-300 GB/month Pretty sure I use more than that in a day.

That's a lot of linux isos to download per day. Seriously, why would you use that much data per day for? Even getting full blu-rays you wouldn't be able to watch that many.

It's more up than down.

Re: BitTorrent v2

#366

Post by octoberfranklin »

huhtenberg wrote: post #106
proverbialbunny wrote: post #25

It feels like this isn't a large enough leap forward. It would be nice if BitTorrent v2 made it harder for ISPs to identify what is bit torrent traffic. AT&T artificially slows down upload speeds.

It's nearly impossible to obfuscate a protocol to work around filtering. You'd want to look like some other protocol and you want that protocol be encrypted by default. Otherwise yours will get fingerprinted via the deep packet inspection. The most obvious choice is to run your protocol over TLS. But then they can just throttle long-lived bulky TLS connections where neither side is on 443. You can then require the re…

The problem is that most residential users don't care about upload speed.

This allows ISPs to throttle uploads with very low risk of pissing off the bulk of their customer base when their detection algorithm gives a false positive.

Re: BitTorrent v2

#367

Post by retox »

nayuki wrote: post #302

Since the article mentioned bencoding and provides sample .torrent files, I'd like to show my code for parsing and visualizing bencode structures: https://www.nayuki.io/page/bittorrent-bencode-format-tools

Can I suggest including an example .torrent file (for something legal) for those that don't have one to upload.

Re: BitTorrent v2

#368

Post by octoberfranklin »

LordDragonfang wrote: post #239
moonbug wrote: post #223

Earlier quoted context omitted.

>Other choices here are OpenVPN, WireGuard and, possibly, IPsec or, get a better ISP? not all technical problems need a technical solution.

Not a viable solution in the US. Most ISPs operate on exclusivity agreements, and like 90% of users effectively have only a single choice of ISP for internet of a reasonable speed. A better solution would be to have rules to make that kind of throttling illegal, but we've seen how that played out.

> 90% of users effectively have only a single choice of ISP for internet of a reasonable speed

Which is mainly because we have privatized roads here in the USA.

Yes, you read that right. The "gold standard" is underground fiber, which lies in the public right-of-way yet is 100% privately owned with no "duty to serve" like the electric utility has. Oftentimes the fiber owner doesn't even have to dig up the dirt -- if they lay fiber along a newly built highway the government does all the digging for them (to create the roadside drainage ditch). The phone company just unreels a spool of armored OS2 into the ditch and hey, it's Miller Time.

Until privatized right-of-way stops we will continue to get screwed.

[*] Transcontinental railroad right-of-ways are the exception to the above, but there's only four of them. And, frankly, that land was privatized through outright fraud. Read the book _Railroaded_ sometime, it's shocking.

Re: BitTorrent v2

#369

Post by captainmuon »

perardi wrote: post #324
int_19h wrote: post #277

Earlier quoted context omitted.

It's also fairly successful as a way to transfer money.

I am pretty lucky that I only have to transfer money to between 2 countries that have heavily linked banking systems (US -> Canada), but I am not entirely sure what Bitcoin would get me in terms of transferring money. The fees are quite small on Transferwise, and also, I have…I don’t know what you want to call it…accountability? Reversibility? Reliability? Whatever fee I have to pay, the fact I get a known third-part…

Some time ago I had a bunch of Chinese yuan on a Chinese account that I wanted to transfer to Europe. I couldn't do a wire transfer, and I couldn't bring them as cash since my German bank didn't want them. I could either pull 500 € a day via ATM (with large fees), or just send it via Bitcoin.

Re: BitTorrent v2

#370

Post by rolleiflex »

andrei_says_ wrote: post #300
rolleiflex wrote: post #83

I make P2P tools too. [0] Let me tell you this: Bittorrent is one of the few things in the space that actually ... works. It works not in the sense that there is a white paper that should work. Not in the sense that there are a few company-made swarms hosted on industrial servers that keep everyone up and alive, so that the thing gives the impression the 'P2P' network does work. Not in the sense that there is a very-…

Aether looks so interesting. Do you have communities that have adopted it ? In what use cases? How do they use it daily? I like how the data expiration is set as an intentional feature.

We do have a small but dedicated community. It’s quite nice, you should check it out.
Post reply on HN