How We Made IPFS Content Publishing 10x Faster

Also used it several times for almost-live video broadcasts, served over cloudflare ipfs gateway while it was working.

It used HLS with .ts files, in a special way which circumvented cloudflare protection against .mp4 and .ts files, or something along the lines. Don't remember the details, but it was a cheap way to deliver your stream to any video player using standard protocols only.

Couldn't you also just build a bittorrent client that hosts a local webserver to provide http access? I could never get what IPFS actually did that bittorrent didn't.

1. IPFS has addressable content

The main drawback of BitTorrent v1 file hashing scheme inside the .torrent file is that it makes a virtual stream of the directory you want to share, splits that stream into blocks, and hash it altogether. That means that each file inside the torrent is unique, even if your directory have very common files, each is which exist in BitTorrent network per se.

IPFS solves this issue by just hashing the contents of the file. The "directory" of the files is a list of each individual file. If somebody has created a directory of 100 mp3 files and you happen to have one of it and added it into your IPFS daemon, you will be serving this file even if you've downloaded it from elsewhere (not from this "directory").

2. IPFS has both immutable and mutable files and directories

In BitTorrent, if you want to update your torrent file (in Russian we use «раздача» (bittorrent "upload"/"seeding"), a very precise word for bittorrent which doesn't have direct equivalent in English, so I'll stick to "torrent file), all the swarm needs to be switched to the updated version due to #1.

This means you have split swarm: the majority of people still seed the old version of torrent, and the minority seed the new version. Because of non-content addressation, all the old, already exising files in new .torrent file are treated as "new", and the old swarm can't seed them.

In IPFS, you can either create new immutable directory with all the old files and 1 new file, and all the old files would be seeded by the existing peers, or you can create mutable directory, and you can just modify it to your like without the need to update the link.

---

Both of these issues are solved in BitTorrent v2 more or less, but it's still not very popular, even if the specification is from 2017.

IPFS however is much more featureful than that. It allows to build decentralized distributed (serverless) websites and services, with user data and such.

There used to be a ZeroNet project which directly aimed at decentralized distributed web services, and is was very cool, there were many blogs, forums, boards. It all could be implemented in IPFS, but I saw only very simple text editors over IPFS and such, much simpler applications than it was in ZeroNet.

BitTorrent had their own version of distributed websites, Maelstrom: https://www.ctrl.blog/entry/bittorrent-maelstrom.html

the containerd stargz snapshotter has an IPFS integration so you can use IPFS instead of a traditional OCI registry to store your OCI containers

Not using it in production but i found it pretty cool to test

His statement at the end was pretty interesting:

"If we do want to change our relationship to technology, I think we’d have to do it intentionally. My basic thoughts are roughly:

    We should accept the premise that people will not run their own servers by designing systems that can distribute trust without having to distribute infrastructure. This means architecture that anticipates and accepts the inevitable outcome of relatively centralized client/server relationships, but uses cryptography (rather than infrastructure) to distribute trust. One of the surprising things to me about web3, despite being built on “crypto,” is how little cryptography seems to be involved!
    We should try to reduce the burden of building software. At this point, software projects require an enormous amount of human effort. Even relatively simple apps require a group of people to sit in front of a computer for eight hours a day, every day, forever. This wasn’t always the case, and there was a time when 50 people working on a software project wasn’t considered a “small team.” As long as software requires such concerted energy and so much highly specialized human focus, I think it will have the tendency to serve the interests of the people sitting in that room every day rather than what we may consider our broader goals. I think changing our relationship to technology will probably require making software easier to create, but in my lifetime I’ve seen the opposite come to pass. Unfortunately, I think distributed systems have a tendency to exacerbate this trend by making things more complicated and more difficult, not less complicated and less difficult."

Funnily enough, later that year ChatGPT came out and blew away the excitement around cryptocurrencies by making software easier to manufacture to some degree. Though even with the latest LLM tools, I don't think this has changed at all so far: "Even relatively simple apps require a group of people to sit in front of a computer for eight hours a day, every day, forever." Maybe those people can program by texting from the coffee machine, but they're still working.