Earlier quoted context omitted.
You could use nix the same way guix uses scheme. The lowest level operation in nix is not "run-this-bash-script" but "run- this-executable-with-these-arguments".
But even still you are mixing languages, so the problem remains. The issue is not that Nix uses Bash, but that Nix uses different languages for the host-side and build-side. AFAIK people aren't writing the build-side code in the Nix language, and I'm not sure if it even has the features necessary to do it. Guix unifies the different layers of code execution with a single language, and using a Lisp enabled that design…
What is Nix and why you should try it
181–190 of 201 posts
Re: What is Nix and why you should try it
#182Earlier quoted context omitted.
None of what he mentioned was desktop-specific. GPUs are useful, and if CUDA is involved then it's being used for acceleration of some kind, which can perfectly well be done on servers. I agree that Nix is hard to deal with when it comes to binary distributions of any kind, which includes CUDA.
> I agree that Nix is hard to deal with when it comes to binary distributions of any kind, which includes CUDA. That's really interesting to me. Why is a binary package not considered a source package where "compiling" it is a no-op?
A source package using autoconf will, in general, just work. A binary package always needs extra work.
Re: What is Nix and why you should try it
#183Nice post, and I'm glad you're enjoying NixOS!
I'm a NixOS contributor/committer, and the principle author (and co-maintainer) of the Bundler-based packaging integration. If you or anyone reading this has any feedback and/or questions about NixOS and Ruby (or NixOS in general) feel free to shoot me a message. I idle on #nixos on Freenode (as cstrahan), and you can email me at charles {{at}} cstrahan.com.
Cheers!
-Charles Strahan (cstrahan)
Re: What is Nix and why you should try it
#184I can vouch for the immense improvement Nix has made to my software development process. I use NixOS on my desktop and laptop. At the OS-level, it gets a lot of things right: reproducible, immutable system configs; lightweight containers; devops with declarative configs. At a software project level, nix-shell is an indispensible tool. Compilers and interpreters aren't even part of my system-wide config; instead each…
Sounds like this solves a similar problem to Docker. Can you comment on what the differences are, and the relative strengths and weaknesses of each approach?
NixOS committer here.
Docker attempts to achieve reproducibility by capturing the entire state of a system in an image file. It also attempts to conserve space by taking a layered approach to images, so when you base your Dockerfile on some base image, your resulting image is the union of the base image's layers and your own changes.
Here's where Docker's approach falls down, and how this could be fixed (and indeed is, by Nix).
Flaw #1:
Building an image from a given Dockerfile is not guaranteed to be reproducible. You can, for example, access a resource over the network in one of your build steps; if the contents of that resource changes between two `docker build`s (e.g. a new version of whatever you're downloading is released, or an attacker substitutes the resource), you'll silently get different resulting images, and very likely will run into "well, it works on my machine" issues.
Solution:
Prohibit any step of your build process from accessing the network, unless you've supplied the expected hash (say, sha256) of any resulting artifacts.
For the projects I work on in my free time, using NixOS on my personal computers, I've never been bitten by nondetermism.
I wish I could say the same about my work projects that use Docker. My team members and I have run into countless issues where our Dockerfiles stop working and then we have to drop everything and play detective so e.g. new hires can get to work, or put fires out in our C.I. env when the cached layers are flushed, etc. So many wasted hours.
Flaw #2:
What happens if you have two Dockerfiles that don't share the same lineage, but you install some of the same packages? You end up with multiples layers on your disk that contain the same contents. That's wasted space.
Solution:
I'll use NixOS as an example, again. In NixOS, you can look at any package and compute the entire dependency graph. It should be noted that this graph includes not only the names of the packages, but also precisely which version of each package was used as a build input. This goes for both build inputs and runtime dependencies.
Note: by "version" I mean not only the version as listed in the release notes, but every detail of how the package was built: which version of Python was used? And then transitively: what version of C was used to compile that Python? etc, etc.
NixOS exploits this by allowing you to share packages with the host machine, and then each NixOS container you spin up has the necessary runtime dependencies bind-mounted into the container's root file system. As a result, NixOS has better deduplication (read: zero). Also, by the same graph traversal mechanism, it's trivial to take any environment, serialize the runtime dependency graph, and send that graph to another machine, back it up somewhere, create a bootable ISO for CD/USB -- whatever you can dream up.
Thanks to Nix's enforced determinism, you can trivially build container environments (and all of their required packages) in parallel across a fleet of build machines. In fact, Nix's superiority at building packages is so strong that people have gone so far as to build Docker images using Nix instead of `docker` (where they can't avoid Docker entirely for whatever reason): https://github.com/NixOS/nixpkgs/blob/2a036ca1a5eafdaed11be1...
You can read more about the Docker image building support here (along with my own rationale for this in the comments): http://lethalman.blogspot.com/2016/04/cheap-docker-images-wi...
I'm keeping things simple here and trying to address the most salient "Docker vs Nix" points, though I could continue talking about other strengths of NixOS outside of the scope of Docker/container tech, if desired.
Docker's union filesystem approach is great in a world where you can't use a better package manager. For everyone else, there are package managers that obviate the need for such hacks, don't waste space, provide determinism at both runtime and build time, etc.
Re: What is Nix and why you should try it
#185I install any PACKAGE of version VERSION in:
~/stow/pkgs/PACKAGE/VERSION/
So taking nim for example, I'd have: ~/stow/pkgs/nim/0.17.1/
Under that, I'd have the whole FHS system for just nim 0.17.1: ~/stow/pkgs/nim/0.17.1/bin/
~/stow/pkgs/nim/0.17.1/lib/
~/stow/pkgs/nim/0.17.1/share/
I set ~/stowed/ as my STOW_TARGET. So on running stow, symlinks to all: ~/stow/pkgs/PACKAGE/VERSION/{bin,lib,share,..}/..
get created in: ~/stowed/{bin,lib,share,..}/..
I then have aliases set up to do something like "unstow PKG/OLD_VER" and "restow PKG/NEW_VER" when I want to change the package version.I skip the "restow" step if I just want to "soft uninstall" a package i.e. remove it from PATH, PKG_CONFIG_PATH, MAN_PATH, .. by removing it only from the STOW_TARGET ~/stowed.
"Hard uninstall" would be simply:
\rm -rf ~/stow/pkgs/PACKAGE/VERSION_I_DONT_WANT/
Or even: \rm -rf ~/stow/pkgs/PACKAGE_I_DONT_WANT/Re: What is Nix and why you should try it
#186I switched from Arch Linux to NixOS last year and it was a great experience https://ramsdenj.com/2017/06/19/switching-to-nixos-from-arch... . The community is relatively small and easy to get into, with lots of people ready to help out beginners. It took a while to get used to, especially considering the way to learn how something works seems to always be to go and read source code on GitHub. There are good things an…
Re: What is Nix and why you should try it
#187I can vouch for the immense improvement Nix has made to my software development process. I use NixOS on my desktop and laptop. At the OS-level, it gets a lot of things right: reproducible, immutable system configs; lightweight containers; devops with declarative configs. At a software project level, nix-shell is an indispensible tool. Compilers and interpreters aren't even part of my system-wide config; instead each…
these functions' documentation is either buried in a long manual This is a problem with lots of feature-rich software, even with meticulously-documented APIs. What we need is reverse-indexed documentation. That is, an extensive API reference is only useful for someone who already knows what functions are in the API and just needs to remember how to use them. But even the most thorough API reference does nothing to pr…
Re: What is Nix and why you should try it
#188Earlier quoted context omitted.
I believe GP was asking if you could have packages A and B installed simultaneously if - A depended on version X of C - B depended on version Y of C - X != Y
Yes you can because packages are not really installed at the system level. Those dependencies are all solved at runtime. You can't have A and B in a same runtime environment obviously.
Re: What is Nix and why you should try it
#189Earlier quoted context omitted.
Yes you can because packages are not really installed at the system level. Those dependencies are all solved at runtime. You can't have A and B in a same runtime environment obviously.
Um, just to be clear: can I write a program that depends on both A and B? Nix lets you do that.
At different runtimes (e.g: different package.py configurations) yes.
I can have a software called my_application which has a version that uses A and another version that uses B.
That differentiation can be a flag at runtime, a "variance" e.g: gcc-4 vs gcc-3 etc.
How does nix allow you to do that? For example, say I am using library foo and it has the function bar
In version 1
bar definition is
```
def bar(a, b):
return a + b
```In version 2
bar definition is
```
def bar(a):
return a + 10
```I don't understand how you can use v1 and v2 of foo at the same runtime? Unless nix does some namespacing based on the lib version invisibly for the enduser but that looks very prone to error...
If you release them as separated libraries it won't complain, for example, shotgun API imports can be by version
```
import shotgun3
import shotgun2
```
so theoretically you can release a rez package whose name is shotgun3 and another one whose name is shotgun2.
That way you would effectively have shotgun2 and shotgun3 in a same runtime environment.
Re: What is Nix and why you should try it
#190Earlier quoted context omitted.
I do this without needing any extra tool[0]. I basically define a custom meta-package with all the packages I want as dependencies and install it with `nix-env -iA`. [0]: https://github.com/asymmetric/dotfiles/blob/master/config.ni...
Thanks for linking to this! Do you know of a way to get a list of installed packages then? "nix-env -q" lists the meta-package but not the packages installed within. Also, when you update packages, does the update propagate to the sub-packages?
And as you can see you can also override their config.
I don't know how to list installed packages, as I almost never feel like I need to know that (but if you find out how, I'd be happy to know!)