Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nix × IPFS – Milestone 1

blog.ipfs.io

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Nix × IPFS – Milestone 1

#61

Post by kevincox »

MarcScott wrote: post #42
adsjhdashkj wrote: post #40

Tangent, what's the best way to use Nix-like features in a mainstream distro? Eg, i've been evaluating PopOS recently because i want a Mac equivalent OS. Ie, i don't want driver/config issues. However one problem i never see solved from any "normal" distros is reproducible systems. Hypothetically if i used Nix as a Desktop my config would be bulletproof. .. but then i'm going through a fair amount of work configuring…

I just bit the bullet and use NixOS as my Desktop. There's a bit of work after initial install, to setup your config files the way you want, but after that you're all set. I've had no problems transferring my configs when I change hardware, and end up with an identical environment that I had on my previous computer.

I recently switched as well and am very happy. I don't think that re-configuring takes much longer that the average distro would especially because a lot of services you can just say `services.foobar.enable = true` and get a decent default config. You only need to spend your time configuring the things that you are picky about.

And yes, the fact that I can configure all of my systems with a shared set of configs and have consistent and reproducible environments is fantastic.

I wrote up my thoughts about the switch here: https://kevincox.ca/2020/09/06/switching-to-desktop-nixos/

Re: Nix × IPFS – Milestone 1

#62

Post by kevincox »

teacpde wrote: post #50
arianvanp wrote: post #5

Earlier quoted context omitted.

For one. They allow for distributed incremental builds. Improving your developer productivity. IT doesn't matter anymore where an object file is compiled. You just need to know its hash, ask if it exists, or otherwise compile yourself. One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT…

I understand why changing comments doesn’t matter, but how do you decide which dependencies don’t need to be rebuilt when certain part of the source code is changed?

The most "proper" way to do this is splitting the result into interface + implementation. For example if you are dynamically linking to a library instead of being given access to the entire library you are given a simplified "signature" which has the minimal amount of info required to linking. This interface would change when you add or remove a function, but not when a function body changes.

The example was given for linking but this can be done for many steps of the build process.

Re: Nix × IPFS – Milestone 1

#63

Post by Ericson2314 »

georgyo wrote: post #48
Taek wrote: post #8

Earlier quoted context omitted.

We built an alternative to IPFS called Skynet that attempts to solve a couple of the major issues with IPFS. The biggest one being data durability and uptime. On Skynet, pinning doesn't mean hosting the file from your machine, it means paying a bunch of service providers to host the file for you. When you pin content to Skynet, you can turn off your computer 5 minutes later and the data will still be available global…

Say what you will about IPFS and ProtocolLabs, but I really admire that they made IPFS completely blockchain agnostic. There is no requirement to buy or receive coins to use IPFS. And multiple different implementations of persistent stores can use the same IPFS network to ensure durability. This means that FileCoin, other coins, static hosting providers, self-hosting can all coexist and strengthen the same network. E…

Yeah the layering in Protocol Lab's work is really nice.

The tendency of funded / for-profit software is to eschew layering, so it's easy to try out but ultimately ill-fitting and inflexible. This is a core problem of capitalism with IT.

IPFS, libp2p, IPLD, Filecoin etc. all resist that temptation, and I think it will help them greatly in the end.

Re: Nix × IPFS – Milestone 1

#64

Post by Ericson2314 »

vin047 wrote: post #18
canndrew2020 wrote: post #14

Earlier quoted context omitted.

> Does IPFS actually solve the problem they set out here though? No, none of these distributed P2P networks (that I've seen) do. The problem isn't just building a DHT (kademlia-based networks have existed for years) the problem is incentivizing people to seed - ideally people with high-bandwidth and massive amounts of storage who are seeding data that people want. In other words you need to build an economy on top of…

Protocol Labs (the guys behind IPFS) have been working on https://filecoin.io to address this exact concern – incentivising people via micropayments in their cryptocurrency (filecoin) to pin and seed files.

Yes it's really good to have the economics (mechanism design) and infrastructure in separate layers.

Also, I'd argue more important than even having the hosters is having the content addresses. We need well-known immutable data for people to want in the first place. And traditional system bury data under so much mutation/indirection that it's hard to know what that content is, or that content-addressing even exists.

I highly recommend https://www.softwareheritage.org/2020/07/09/intrinsic-vs-ext..., which is about software heritage trying to get the word out to the larger library/archival/standardization community that content addressing and other "intrinsic" identifiers are possible and desirable.

Git and torrents I think is the best counterexample to the above, and there is probably more legally-kosher git and bittorrent usage, so I am especially bullish on Git hashing being the bridge to a more distributed/federated world.

Re: Nix × IPFS – Milestone 1

#65

Post by Ericson2314 »

teacpde wrote: post #50
arianvanp wrote: post #5

Earlier quoted context omitted.

For one. They allow for distributed incremental builds. Improving your developer productivity. IT doesn't matter anymore where an object file is compiled. You just need to know its hash, ask if it exists, or otherwise compile yourself. One problem with NixOS currently is that certain dependencies in our package tree are very painful to change. If we touch glibc, we need to recompile 60.000 packages and it takes A LOT…

I understand why changing comments doesn’t matter, but how do you decide which dependencies don’t need to be rebuilt when certain part of the source code is changed?

Nix does this simply and stupidly --- if you can observ it, it's part of the cache key.

This is wonderful because it's totally sound. All efficiency methods (basically, hiding things, as the other comment says) is left to the build plan itself.

Switching to Nix for batch jobs is switching from DOS to an OS with actual process isolation. A complete paradigm shift.

Re: Nix × IPFS – Milestone 1

#66

Post by Ericson2314 »

adsjhdashkj wrote: post #40

Tangent, what's the best way to use Nix-like features in a mainstream distro? Eg, i've been evaluating PopOS recently because i want a Mac equivalent OS. Ie, i don't want driver/config issues. However one problem i never see solved from any "normal" distros is reproducible systems. Hypothetically if i used Nix as a Desktop my config would be bulletproof. .. but then i'm going through a fair amount of work configuring…

Just use NixOS. Our defaults are more like Debian/Fedora than Arch/Gentoo. You're not going to be struggling to hunt down drivers.

Re: Nix × IPFS – Milestone 1

#67

Post by shp0ngle »

imhoguy wrote: post #9

Open source plus distributed p2p file sharing is the killer combo. I don't get why public stuff like NPM, DEB or Docker registries haven't switched to use it as primary way of distribution. P2P, such as IPFS and recently BitTorrent 2.0 with its hash tree per file, is the only free (as beer and speech) reliable way to host things online forever - at least as long there is the last of the veterans who keeps a copy and…

BitTorrent 2.0 is not recent, no.

The current bittorrent company is interested in chasing ICOs, I don’t think they work on new versions of the protocol anymore?

edit: yeah it’s from 2008

Re: Nix × IPFS – Milestone 1

#68

Post by Taek »

brabel wrote: post #49
Taek wrote: post #8

Earlier quoted context omitted.

We built an alternative to IPFS called Skynet that attempts to solve a couple of the major issues with IPFS. The biggest one being data durability and uptime. On Skynet, pinning doesn't mean hosting the file from your machine, it means paying a bunch of service providers to host the file for you. When you pin content to Skynet, you can turn off your computer 5 minutes later and the data will still be available global…

This looks interesting. Why is the payment method based on a crypto coin? How does using a blockchain help make Skynet and Sia work? I am wary of crypto coins as they tend to have wild swings in value over time as they are used primarily for speculation... can I be sure that if I upload something today worth $2/month, in 2 years I will still be paying this much or less?

Sia is a decentralization first protocol. We believe strongly that the biggest advantage is an immunity to de-platforming and a commitment to uncompromisingly open protocol.

Crypto is the only means of payment I am aware of that does not have a centralized middleman with the power to deny a transaction.

Decentralization aside, there are efficiency gains as well. Every transaction on the Sia network is point-to-point, and in some cases we've had nodes that average more than 1 million discreet transactions per day for over a month. The total cost of doing that was something like $100 (including the cost of all the resources bought with those millions of transactions), I struggle to imagine a traditional payment system providing that kind of value.

There's also a lot more flexibility to innovate. For example, every single one of our payments is accompanied by a cryptographic proof that the accompanying storage or computation (not many people know, but the Sia network does support a limited form of computation) action was completed correctly. The payment and computation are fundamentally/cryptographically tied together in way that we could not reasonably achieve on a traditional payment system.

Re: Nix × IPFS – Milestone 1

#69

Post by wesnel »

adsjhdashkj wrote: post #40

Tangent, what's the best way to use Nix-like features in a mainstream distro? Eg, i've been evaluating PopOS recently because i want a Mac equivalent OS. Ie, i don't want driver/config issues. However one problem i never see solved from any "normal" distros is reproducible systems. Hypothetically if i used Nix as a Desktop my config would be bulletproof. .. but then i'm going through a fair amount of work configuring…

Personally, I use NixOS in conjunction with Home Manager[0] to more conveniently use Nix to manage my user-level config. The configuration that I manage with Home Manager includes my Emacs config, my Git config, and my Bash config. Additionally, almost all of the other programs I use on a daily basis are at least installed through it. Since it’s primarily only used for managing some packages, the configuration process was simple compared to setting up NixOS for me.

I would imagine that you could use Home Manager on a non-NixOS system to at least create reproducible configs for the programs you use, although the OS as a whole would of course still be non-reproducible. However, I do not know how well Home Manager works on non-NixOS systems.

As you mention, just using Nix itself can be sufficient to get a reproducible set of packages on your system. I recall reading a blog post about someone who does this on both Ubuntu and MacOS.[1] The way this person does it is interesting because it’s more sophisticated than spawning the occasional ‘nix-shell’ or something. For example, they get the benefit of Nix “generations,” with a new generation being created each time they modify their declarative config files.

[0] https://github.com/nix-community/home-manager [1] https://www.nmattia.com/posts/2018-03-21-nix-reproducible-se...

Re: Nix × IPFS – Milestone 1

#70

Post by capableweb »

shp0ngle wrote: post #67
imhoguy wrote: post #9

Open source plus distributed p2p file sharing is the killer combo. I don't get why public stuff like NPM, DEB or Docker registries haven't switched to use it as primary way of distribution. P2P, such as IPFS and recently BitTorrent 2.0 with its hash tree per file, is the only free (as beer and speech) reliable way to host things online forever - at least as long there is the last of the veterans who keeps a copy and…

BitTorrent 2.0 is not recent, no. The current bittorrent company is interested in chasing ICOs, I don’t think they work on new versions of the protocol anymore? edit: yeah it’s from 2008

People are working on BitTorrent yes! libtorrents blog made a blog post quite recently (September 7th, 2020) - https://blog.libtorrent.org/2020/09/bittorrent-v2/
Post reply on HN