Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BitTorrent v2

blog.libtorrent.org

Post reply on HN
Print view View original

371–380 of 576 posts

Re: BitTorrent v2

#371

Post by salty_biscuits »

cercatrova wrote: post #144
rolleiflex wrote: post #83

I make P2P tools too. [0] Let me tell you this: Bittorrent is one of the few things in the space that actually ... works. It works not in the sense that there is a white paper that should work. Not in the sense that there are a few company-made swarms hosted on industrial servers that keep everyone up and alive, so that the thing gives the impression the 'P2P' network does work. Not in the sense that there is a very-…

> It's a tool that doesn't pitch that it's a P2P tool - it doesn't try to convince you with sob stories in how using P2P helps fight against the big bad evil web. Instead, you use it because it's genuinely the best at what it does: it being P2P is not a selling point, it's just how it happens to work, and that is exactly what it should be. This should be something that every creator who markets or sells products shou…

I think everyone knows that, it is just much, much easier to say about how it works with fancy buzz words than to actually make something good.

Re: BitTorrent v2

#372

Post by btschaegg »

octoberfranklin wrote: post #362
aurelian15 wrote: post #356

Earlier quoted context omitted.

There are ways around this. See "content-aware chunking", e.g. implemented using rolling hashes [1]. This is for example what rsync does. The idea is to make blocks (slightly) variable in size. Block boundaries are determined based on a limited window of preceding bytes. This way a change in one location will only have a limited impact on the following blocks. [1] https://en.wikipedia.org/wiki/Rolling_hash

Rolling hashing is really only useful for finding nonaligned duplicates. There isn't a way to advertise some "rolling hash value" in a way that allows other people with a differently-aligned copy to notice that you and them have some duplicated byte ranges. Rolling hashes only work when one person (or two people engaged in a conversation, like rsync) already has both copies.

I think you misunderstood how the rolling hash is used in this context. It's not used to address a chunk; you'd use a plain old cryptographic hash function for that.

The rolling hash is used to find the chunk boundary: Hash a window before every byte (which is cheap with a rolling hash) and compare it against a defined bit mask. For example: Check if the first 20 bytes are zero. If so, you'd get chunks with about 2^20 bytes (1 MiB) average length.

As a good explanation, I'd encourage you to look at borgbackup's internals documentation: https://borgbackup.readthedocs.io/en/stable/internals.html

Re: BitTorrent v2

#373

Post by Semaphor »

bonoboTP wrote: post #273
dane-pgp wrote: post #266

Earlier quoted context omitted.

It was good design to decouple the transfer protocol from the search protocol, to allow different technologies to be explored, but in practice people ended up relying on the "good enough" solution of using the web for search. Setting the precedent that media companies can seize domain names and force ISPs to block access to search engines if they don't like some of the search results (while ruining the lives of the p…

Yes. For all the P2P distributed praise ITT, torrent as a whole, as it is used in the real world, is very centralized. If the Pirate Bay is shut down, that stuff is gone. Uploading became a privilege, something that normal average people don't do. Earlier people ripped their own CDs, DVDs, magazine scans, digitized their VHS, recorded shows with their TV card etc. and shared these files. I don't know anybody who has…

> earlier people ripped their own CDs, DVDs, magazine scans, digitized their VHS, recorded shows with their TV card etc. and shared these files.

For that, you want Direct Connect [0]. As a one sentence pitch: IRC for file sharing ;)

[0] https://en.wikipedia.org/wiki/Direct_Connect_(protocol)

Re: BitTorrent v2

#374

Post by 3np »

dchuk wrote: post #208
cercatrova wrote: post #144

Earlier quoted context omitted.

> It's a tool that doesn't pitch that it's a P2P tool - it doesn't try to convince you with sob stories in how using P2P helps fight against the big bad evil web. Instead, you use it because it's genuinely the best at what it does: it being P2P is not a selling point, it's just how it happens to work, and that is exactly what it should be. This should be something that every creator who markets or sells products shou…

This is exactly the problem with all cryptocurrency currently. It’s a massive user experience issue, in the sense that users have to experience the technical bullshit of how the currencies work, completely missing the brilliant part of real money: it just works. I hand people money, they give me things. I swipe my credit card, I get things. I can’t remember who aid it originally, but there’s a great test you can give…

TBH, I don't think that's a bad thing at the current moment - the tools and base layers (most notably ETH scalability) are not in a position to handle a huge influx of mainstream users with expectations that things Just Work.

There's great progress being made, but we'll not get there during 2020.

Re: BitTorrent v2

#375

Post by knodi123 »

tomxor wrote: post #215

> not only uses a hash tree, but it forms a hash tree for every file in the torrent [...] > Files that are identical can also more easily be identified across different swarms, since their root hash only depends on the content of the file. Wait, content addressed blobs across swarms... does that mean torrents made by completely different people at different times that happen to contain one or more identical files can…

> this feels like a significant feature that would boost the long term health of a lot of torrents

Except for so many that are a single archive file. It seems logical that you'd want to compress what you're sending as much as possible before uploading... but in today's bandwidth environment, it actually makes sense to leave certain things uncompressed so you can benefit from swarm overlaps?!? Wild!

Re: BitTorrent v2

#376

Post by riffraff »

Shared404 wrote: post #349
sanderjd wrote: post #267

Earlier quoted context omitted.

It's simple: it is an abject failure as a currency. It is fairly successful as a security for the purpose of speculation. If you aren't a financial speculator, you really don't need to pay attention to it.

> it is an abject failure as a currency. In the US perhaps. In at least one other country however... https://www.bbc.com/news/business-47553048 https://www.somagnews.com/bitcoin-becomes-the-dominant-curre...

The second article makes a pretty strong claim but does not seem to have any proof other than "Venezuela uses crypto a lot", doesn't it?

Re: BitTorrent v2

#377

Post by riffraff »

captainmuon wrote: post #369
perardi wrote: post #324

Earlier quoted context omitted.

I am pretty lucky that I only have to transfer money to between 2 countries that have heavily linked banking systems (US -> Canada), but I am not entirely sure what Bitcoin would get me in terms of transferring money. The fees are quite small on Transferwise, and also, I have…I don’t know what you want to call it…accountability? Reversibility? Reliability? Whatever fee I have to pay, the fact I get a known third-part…

Some time ago I had a bunch of Chinese yuan on a Chinese account that I wanted to transfer to Europe. I couldn't do a wire transfer, and I couldn't bring them as cash since my German bank didn't want them. I could either pull 500 € a day via ATM (with large fees), or just send it via Bitcoin.

Why couldn't you do a wire transfer?

Re: BitTorrent v2

#378

Post by enedil »

gruez wrote: post #274
nicbou wrote: post #257

Earlier quoted context omitted.

It has been a few years, and despite a few attempts, I still don't really understand bitcoin. Yes yes ledgers and proof of work and yada yada, but I still don't know what I would need to do to buy something with bitcoin right now. As I understand I need to do something that's pretty much exactly like opening a bank account, but transactions take forever, they cost money, and the currency is not accepted by any busine…

>As I understand I need to do something that's pretty much exactly like opening a bank account Not really. If I want a bank account I need to fill out a page of paperwork requiring my name, address, social security number, citizenship information, income, occupation, date of birth, scans of various pieces of ID and/or other documents. Compare this to bitcoin where you only need to install an app, and new identities c…

The parent commenter is European, so they have probably a lots better experience with banking than you might have in the US.

Re: BitTorrent v2

#379

Post by hinkley »

btschaegg wrote: post #372
octoberfranklin wrote: post #362

Earlier quoted context omitted.

Rolling hashing is really only useful for finding nonaligned duplicates. There isn't a way to advertise some "rolling hash value" in a way that allows other people with a differently-aligned copy to notice that you and them have some duplicated byte ranges. Rolling hashes only work when one person (or two people engaged in a conversation, like rsync) already has both copies.

I think you misunderstood how the rolling hash is used in this context. It's not used to address a chunk; you'd use a plain old cryptographic hash function for that. The rolling hash is used to find the chunk boundary: Hash a window before every byte (which is cheap with a rolling hash) and compare it against a defined bit mask. For example: Check if the first 20 bytes are zero. If so, you'd get chunks with about 2^2…

I think they understood just fine.

If I discover that the file I want to publish shares a range with an existing file, that does very little because the existing file has already chosen its chunk boundaries and I can’t influence those. That ship has sailed.

I can only benefit if the a priori chunks are small enough that some subset of the identified match is still addressable. And then I may only get half of a two thirds of the improvement I was after.

Post reply on HN