Live data from Hacker News

New improvements to IPFS Bitswap for faster container image distribution

blog.ipfs.io

51–60 of 61 posts

Re: New improvements to IPFS Bitswap for faster container image distribution

#51

Yay. I last used ipfs for leeching abandonware around November. Although it had a tough time getting started and it would occasionally freeze up for several minutes, it worked well when it worked. It's seems to be getting better from when I first tried it.

I wasn't aware of this use case, can you post a link/CID (if it's legal)?

Re: New improvements to IPFS Bitswap for faster container image distribution

#53
post #15

Earlier quoted context omitted.

I thought that ipfs is about high availability, fault tolerance, including some resistance against addressed censorship. It never looked like an anonymizing tool to me; did anybody advertise it as such?

Well to be fair it would be quite imprudent to have a file system where everyone can see what everyone is doing. In particular it's pointless to be able to circumvent censorship if you can't do so anonymously.

Circumventing censorship without strong anonymity is not necessarily pointless: you can publish something sensitive from a place where you'd not be prosecuted (e.g. from abroad). The point is to bring the message to those who are denied information.

Re: New improvements to IPFS Bitswap for faster container image distribution

#54
post #4

Would be curious to know how using IPFS for internal container distribution compares to using BitTorrent. IIRC BitTorrent has found similar uses in the past. Also, how well does BitSwap work when the underlying network is congested? Do IPFS nodes do any kind of congestion control?

You can't update a torrent, if the content changes you have to create a new one. IPNS helps with that. And you can't share pieces across different torrents, if some still have the old torrent, they share it separately from the new one even if the differences are minimal.

Does anyone use IPNS for anything real? It performs terribly whenever I've tried it. It almost turned me off entirely from using IPFS until I realized that it's just an optional extra and the rest of IPFS is still useful without it. I really wonder how many people try out IPFS, run into issues with IPNS, and then write off the whole project because they thought IPNS was a central piece to it. I think the project would do really well to strike all references to IPNS from their getting-started guides, bring up reliable alternatives like DNSlink records (or even ENS), and then maybe bring up IPNS as an optional extra.

Re: New improvements to IPFS Bitswap for faster container image distribution

#55

Earlier quoted context omitted.

I think that was a dig at ipfs's issues with real world usage where a lot of traffic until recently was wasted on metadata and every node used lots of bandwidth to gossip. Meanwhile the actual throughout on a non-tuned node was not great at all.

Has this changed now?

I just realised that "recently" happened at the end of 2018 https://blog.ipfs.io/53-go-ipfs-0-4-18/ - oops.

There was also a change later which turned off nodes being the middleman by default, but not sure which version.

It's supposed to be much better these days, but I haven't tried it again in a few months.

Re: New improvements to IPFS Bitswap for faster container image distribution

#56

Earlier quoted context omitted.

Has this changed now?

I just realised that "recently" happened at the end of 2018 https://blog.ipfs.io/53-go-ipfs-0-4-18/ - oops. There was also a change later which turned off nodes being the middleman by default, but not sure which version. It's supposed to be much better these days, but I haven't tried it again in a few months.

Ah, that agrees with my experience, I noticed the IPFS node behaving much better some time in the last year.

Re: New improvements to IPFS Bitswap for faster container image distribution

#57
post #43
post #32

Earlier quoted context omitted.

IPNS is a system that runs on top of IPFS. You do not need IPNS to use IPFS.

That's technically correct, but in practice the term "IPFS" is commonly used to refer to both ipfs and its optional feature ipns.

Not in my circles. IPFS is used a lot for storing and archiving files but IPNS is rarely used or mentioned. The only situations I've seen where IPNS would be useful, ENS was used instead as it's more reliable.

Re: New improvements to IPFS Bitswap for faster container image distribution

#58
post #4

Would be curious to know how using IPFS for internal container distribution compares to using BitTorrent. IIRC BitTorrent has found similar uses in the past. Also, how well does BitSwap work when the underlying network is congested? Do IPFS nodes do any kind of congestion control?

You can't update a torrent, if the content changes you have to create a new one. IPNS helps with that. And you can't share pieces across different torrents, if some still have the old torrent, they share it separately from the new one even if the differences are minimal.

although there was an IP to bittorrent for that :(

Re: New improvements to IPFS Bitswap for faster container image distribution

#59
post #4

Would be curious to know how using IPFS for internal container distribution compares to using BitTorrent. IIRC BitTorrent has found similar uses in the past. Also, how well does BitSwap work when the underlying network is congested? Do IPFS nodes do any kind of congestion control?

You can't update a torrent, if the content changes you have to create a new one. IPNS helps with that. And you can't share pieces across different torrents, if some still have the old torrent, they share it separately from the new one even if the differences are minimal.

that should be irrelevant for container distribution as container images are immutable.

Re: New improvements to IPFS Bitswap for faster container image distribution

#60
post #29

Earlier quoted context omitted.

> And you can't share pieces across different torrents So make each layer an own torrent?

That's still not great. Imagine the only difference between two versions of the layer is that you updated a single jar in a 200MB app bundle. The effective difference could be a few tens of blocks, but you still need to redownload the whole thing. If we can manage the assignment/padding to match ipfs fragments, that could result in a massive saving.

That is a limitation of the Torrent clients.

On top of that, most of them should already be able to understand that certain files already exist; but it seems like it’s more of a file-level feature at this point rather than block-level.

Is there something I am misunderstanding?

Post reply on HN