Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nix × IPFS – Milestone 1

blog.ipfs.io

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Nix × IPFS – Milestone 1

#41

Post by bityard »

akerro wrote: post #34
Shared404 wrote: post #20

Earlier quoted context omitted.

I don't know what your software config on the pi is, but if it's writing to the SD card a lot it'll wear out and fail pretty quickly. Just a heads up from someone who's been bitten.

I hear it in every discussion, yet I'm still waiting for any of my SD cards to fail. I've had the same rpi0 for 3 years as CCTV with motiond (motion detector). Recording all activity from my window, filling the card in 100% and removing all recorded data, every 2 days, no errors, no failure. I no longer need that CCTV so using the rpi, with the same SD card, as IPFS node.

IME, the people who complain about SD cards failing on raspberry pi bought cheap cards that were intended for bulk storage, not frequent writes or running an OS card. Regular SD cards are not SSDs.

On my network of random Pis and other stuff, I use only high-endurance SD cards which can withstand lots of writes have durability much closer to SSDs.

Re: Nix × IPFS – Milestone 1

#42

Post by MarcScott »

adsjhdashkj wrote: post #40

Tangent, what's the best way to use Nix-like features in a mainstream distro? Eg, i've been evaluating PopOS recently because i want a Mac equivalent OS. Ie, i don't want driver/config issues. However one problem i never see solved from any "normal" distros is reproducible systems. Hypothetically if i used Nix as a Desktop my config would be bulletproof. .. but then i'm going through a fair amount of work configuring…

I just bit the bullet and use NixOS as my Desktop. There's a bit of work after initial install, to setup your config files the way you want, but after that you're all set. I've had no problems transferring my configs when I change hardware, and end up with an identical environment that I had on my previous computer.

Re: Nix × IPFS – Milestone 1

#43

Post by yjftsjthsd-h »

adsjhdashkj wrote: post #40

Tangent, what's the best way to use Nix-like features in a mainstream distro? Eg, i've been evaluating PopOS recently because i want a Mac equivalent OS. Ie, i don't want driver/config issues. However one problem i never see solved from any "normal" distros is reproducible systems. Hypothetically if i used Nix as a Desktop my config would be bulletproof. .. but then i'm going through a fair amount of work configuring…

I've been using ansible to configure my laptops for years now; it automatically installs most of the packages that I'm going to want, clones things like my dotfiles into my home directory, etc.

Edit: Oh, and obviously I keep the playbooks+roles in version control.

Re: Nix × IPFS – Milestone 1

#44

Post by akerro »

bityard wrote: post #41
akerro wrote: post #34

Earlier quoted context omitted.

I hear it in every discussion, yet I'm still waiting for any of my SD cards to fail. I've had the same rpi0 for 3 years as CCTV with motiond (motion detector). Recording all activity from my window, filling the card in 100% and removing all recorded data, every 2 days, no errors, no failure. I no longer need that CCTV so using the rpi, with the same SD card, as IPFS node.

IME, the people who complain about SD cards failing on raspberry pi bought cheap cards that were intended for bulk storage, not frequent writes or running an OS card. Regular SD cards are not SSDs. On my network of random Pis and other stuff, I use only high-endurance SD cards which can withstand lots of writes have durability much closer to SSDs.

Yes, that's correct. I only buy original Samsung or SanDisk SD cards that come with 5+ years warranty, and then I keep original packaging anyway.

Re: Nix × IPFS – Milestone 1

#45

Post by Avamander »

na85 wrote: post #31
imhoguy wrote: post #9

Open source plus distributed p2p file sharing is the killer combo. I don't get why public stuff like NPM, DEB or Docker registries haven't switched to use it as primary way of distribution. P2P, such as IPFS and recently BitTorrent 2.0 with its hash tree per file, is the only free (as beer and speech) reliable way to host things online forever - at least as long there is the last of the veterans who keeps a copy and…

In a lot of the world, upload speeds are very limited and ISPs charge an arm and a leg if you exceed your upstream data caps. On top of that, Bit torrent has a tendency to overwhelm routers and make them slow to a crawl. Those are a lot of downsides to ask your users to accept just so you can cheap out on download servers.

With LPD, we could have two PCs upload to each other, it would be even beneficial to ISPs if everyone didn't have to download updates at a similar time from the furthest-away server. Especially in developing countries.

Re: Nix × IPFS – Milestone 1

#46

Post by jFriedensreich »

arianvanp wrote: post #3

Does IPFS actually solve the problem they set out here though? IPFS is a distributed CDN; but not very good for storing things persistently or reliably from my experience. At the moment; the nixos cache is stored in very durable and reliable S3 storage; with very high durability guarantees. Why is that not good enough? sure it's centralised. But IPFS doesn't offer distributed durability; it offers a CDN. It doesn't s…

"Distributed CDN" sells ipfs a bit short. It has properties of a distributed CDN but usually people think of a CDN as not a good fit to store the primary assets, but ipfs can have pretty good persistence and reliability guarantees if setup accordingly.

Usually a project like Nixos would not use the simple standard ipfs daemon but something like cluster.ipfs.io and i think i have also seen ipfs servers on top minio or s3 for storing the data. So the actual databits durability and reliability can be the same or better than the s3 comparison with ipfs.

A discussion like this is best split up into the api and storage system parts: so S3 becomes the S3 Rest API and an S3 object store and IPFS becomes the IPFS access protocol (and IPFS gateway REST api) and whatever IPFS storage system a projects choses to use.

The cool thing about IPFS for package managers lies more in the protocol and architecture part than the low level data storage.

- content addressable storage as core design principle solves a lot of problems that need to be manually build on top of s3 anyways if you want to build a CDN or Package manager directly on S3, but in a standard and documented way . I would argue even without the p2p distributed part of ipfs, it would be a great fit for a content addressable storage system.

- "discovery": finding packages across different servers, this is such a game changer. Traditionally you would have 1 or 2 repositories configured and packages are fetched from those, if these servers do not have the right package or versions or are offline you are screwed (having to manually find alternative mirrors or finding copies of the package elsewhere). In ipfs, if your peers don't have a file, it can just ask the network for available "mirrors" and even users who have a copy and contribute to the network can provide you their copy automatically.

Re: Nix × IPFS – Milestone 1

#47

Post by jFriedensreich »

ArchD wrote: post #19

IPFS is not censorship resistant. If people start using it to subvert tyranny, politically neutral things like Nix's use of IPFS can get collaterally damaged in its infancy by state power.

Its slightly more censorship resistant than S3 but yes, that seems to not have been on the list of problems they try to solve. On the other hand a list of all the systems that solve the problems in a half decent performant manner and are also censorship resistant would be 0 to my knowledge.

Re: Nix × IPFS – Milestone 1

#48

Post by georgyo »

Taek wrote: post #8
arianvanp wrote: post #3

Does IPFS actually solve the problem they set out here though? IPFS is a distributed CDN; but not very good for storing things persistently or reliably from my experience. At the moment; the nixos cache is stored in very durable and reliable S3 storage; with very high durability guarantees. Why is that not good enough? sure it's centralised. But IPFS doesn't offer distributed durability; it offers a CDN. It doesn't s…

We built an alternative to IPFS called Skynet that attempts to solve a couple of the major issues with IPFS. The biggest one being data durability and uptime. On Skynet, pinning doesn't mean hosting the file from your machine, it means paying a bunch of service providers to host the file for you. When you pin content to Skynet, you can turn off your computer 5 minutes later and the data will still be available global…

Say what you will about IPFS and ProtocolLabs, but I really admire that they made IPFS completely blockchain agnostic.

There is no requirement to buy or receive coins to use IPFS. And multiple different implementations of persistent stores can use the same IPFS network to ensure durability.

This means that FileCoin, other coins, static hosting providers, self-hosting can all coexist and strengthen the same network.

Every other distributed content addressable file store seems to only be interested in supporting their own coin only. For that reason alone, I find IPFS the most interesting.

Re: Nix × IPFS – Milestone 1

#49

Post by brabel »

Taek wrote: post #8
arianvanp wrote: post #3

Does IPFS actually solve the problem they set out here though? IPFS is a distributed CDN; but not very good for storing things persistently or reliably from my experience. At the moment; the nixos cache is stored in very durable and reliable S3 storage; with very high durability guarantees. Why is that not good enough? sure it's centralised. But IPFS doesn't offer distributed durability; it offers a CDN. It doesn't s…

We built an alternative to IPFS called Skynet that attempts to solve a couple of the major issues with IPFS. The biggest one being data durability and uptime. On Skynet, pinning doesn't mean hosting the file from your machine, it means paying a bunch of service providers to host the file for you. When you pin content to Skynet, you can turn off your computer 5 minutes later and the data will still be available global…

This looks interesting. Why is the payment method based on a crypto coin? How does using a blockchain help make Skynet and Sia work?

I am wary of crypto coins as they tend to have wild swings in value over time as they are used primarily for speculation... can I be sure that if I upload something today worth $2/month, in 2 years I will still be paying this much or less?

Re: Nix × IPFS – Milestone 1

#50

Post by teacpde »

arianvanp wrote: post #5
nmfisher wrote: post #2

Interesting use case for IPFS, which has often felt like an (admittedly cool) solution in search of a problem. Can someone enlighten me as to real-world examples where actually reproducible builds are critical?

For one. They allow for distributed incremental builds. Improving your developer productivity. IT doesn't matter anymore where an object file is compiled. You just need to know its hash, ask if it exists, or otherwise compile yourself. One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT…

I understand why changing comments doesn’t matter, but how do you decide which dependencies don’t need to be rebuilt when certain part of the source code is changed?
Post reply on HN