Live data from Hacker News

Nix as OS X Package Manager

ariya.ofilabs.com

171–180 of 209 posts

Re: Nix as OS X Package Manager

#171
post #33
post #15

When I first heard about Docker, I thought it did what Nix does. Then someone handed me a docker image saying, "I got this working on my laptop, deploy this. Isn't this great!". I had no idea how they arrived at that configuration. If something happened to them or that image, I wouldn't have known how to reproduce that "working state". So I am happy to see Nix / Guix become popular. It is what I imagined package mana…

> I wouldn't have known how to reproduce that "working state". So what? You don't need to reproduce that working state: The work has already been done! Think about how the interactive approach Lisp and Smalltalk programmers use is superior to the compile/run/rerun approach used by C++ and Java programmers -- by exploring the problem in a nonlinear way, you can find your way to a solution more quickly. Docker package…

It sure sounds like you are advocating Docker as a way to produce and distribute a http://martinfowler.com/bliki/SnowflakeServer.html

We've been down this road before. Hacking something until it works, then imaging it and passing it on to your coworkers is a maintenance nightmare.

Re: Nix as OS X Package Manager

#172
post #165

Earlier quoted context omitted.

Not true any longer, at least for the binary packages I produce[0]. I switched them over to trunk-based builds a while ago, which are updated every few days. [0] https://pkgsrc.joyent.com/install-on-osx/

How is situation with python3 on osx? Last time I used pkgin, it was not compiled as Framework preventing some libraries to work properly

They're all available:

    $ pkgin se python3
    python35-3.5.1nb2    Interpreted, interactive, object-oriented programming language
    python34-3.4.4       Interpreted, interactive, object-oriented programming language
    python33-3.3.6nb3    Interpreted, interactive, object-oriented programming language
along with a bunch of pre-packaged modules:

    $ pkgin avail | grep ^py3 | wc -l
        1412
Let me know if anything you need is missing and I'll add it.

Re: Nix as OS X Package Manager

#173
post #33

Earlier quoted context omitted.

> I wouldn't have known how to reproduce that "working state". So what? You don't need to reproduce that working state: The work has already been done! Think about how the interactive approach Lisp and Smalltalk programmers use is superior to the compile/run/rerun approach used by C++ and Java programmers -- by exploring the problem in a nonlinear way, you can find your way to a solution more quickly. Docker package…

It sure sounds like you are advocating Docker as a way to produce and distribute a http://martinfowler.com/bliki/SnowflakeServer.html We've been down this road before. Hacking something until it works, then imaging it and passing it on to your coworkers is a maintenance nightmare.

I appreciate how difficult it is to understand someone when you begin the conversation by assuming you understand them.

For example, did you read that article?

You know Docker has none of those problems

> The first problem with a snowflake server is that it's difficult to reproduce. Should your hardware start having problems, this means that it's difficult to fire up another server to support the same functions.

    docker tag/push/pull
> If you need to run a cluster, you get difficulties keeping all of the instances of the cluster in sync.

    docker start tag
> You can't easily mirror your production environment for testing.

    docker commit/export/import
> When you get production faults, you can't investigate them by reproducing the transaction execution in a development environment.

    docker push/pull
> The true fragility of snowflakes, however, comes when you need to change them. Snowflakes soon become hard to understand and modify. Upgrades of one bit software cause unpredictable knock-on effects.

    docker history/diff
The issue that Martin is describing is one where people work on the server - update it in place, and do not have any change management process besides "do stuff" and any quality checking process besides "check stuff".

He spent over 500 words to say that's probably not a good idea.

No kidding.

Now.

What I'm actually advocating is interactive development: The exact same way we develop our APL workspaces and our Smalltalk images, can be used to design and build server infrastructure.

Instead of a development cycle that goes:

* Edit nixfile/dockerfile/vagrantfile

* Wait (sometimes hours)

* Test things

* Repeat

You get a very short:

* Do stuff

* Get happy

* Tag the results

Then when you want to run "in production", you publish your tags/image, review the history with your team, not dissimilar to how Smalltalk people will clean up their changes file, and then introduce the new version into the cluster. Once you're satisfied the world hasn't ended, you can decommission the old images.

You can even make scripts that do these many of these steps automatically (divert x% of traffic to new instances, collect results, and so on). If you have a well-designed infrastructure, you can have junior developers use those scripts to make changes on their first day.

Now.

I get it, you have realised that if you start over often enough you get good at it.

I get that you've seen people who treat systems development as a completely linear descent into unmaintainable bat barf that can only be recovered by a bad day reinstalling everything and "starting over",

I also get that you think amortising that cost by taking those hours to rebuild your entire system when glibc changes is necessary to avoid that bad day.

I also get you like static external configuration like Dockerfiles or Vagrantfiles, and view them as instrumental in the audit of that process.

However while you understood that I disagree with that, you didn't understand why: If you had a decade or so of Smalltalk or Lisp experience instead of just ruby experience, you might have known exactly what I meant, but instead of asking, you just ignored the part you didn't understand, making a straw man argument that you did understand.

After all, what's more likely, someone knows something important that you don't? Or someone knows something unimportant and just isn't as smart as you?

Re: Nix as OS X Package Manager

#174
post #97

Earlier quoted context omitted.

YES. Finally someone understands what really annoys me when I say that docker isn't 100% reproducible if you aren't using version pinning or something similar. If you have 2 developers, and one of them does a build the next day, you could have them with two different versions of a package when the version went up.

sounds like 'emerge hell' from gentoo 12 years ago.

Ex-Gentoo dev here. It is not the same. Gentoo emerge runs in place, and you can definitely break your system with a broken build.

NixOS on the other hand, builds everything in sandboxes that only expose the requested dependencies. Builds are almost 100% deterministic. The cherry on top is that your system installation is simply a package consisting of all other packages and configuration symlinked together, and you replace your entire system in one go (atomically, by writing a symlink).

If the build fails at any point along the chain, you get an error and your system remains unchanged. You can totally switch from one major release to the next (and back) without hassle.

Re: Nix as OS X Package Manager

#175
post #46

Could someone post (or link to) the history of Nix, the package manager? Who developed it, key project milestones, etc.? I understand it comes from, or led to, NixOS Linux distribution which uses Nix of course. The nixos.org website is a bit sparse on the Nix project's history, and same with wikipedia (though separately mentions that NixOS was a research project started by Eelco Dolstra in 2003).

* 2003 Nix & nixpkgs Language and packages

* 2007 NixOS Linux distribution

* 2009 Hydra Continuous Integration

* 2011 NixOps Cloud deployer

* 2013 First stable NixOS branch

(from http://wmertens.github.io/nixos-cfgmgmtcamp-slides/#/1/1 - non-authorative source :))

Re: Nix as OS X Package Manager

#176
post #46

Could someone post (or link to) the history of Nix, the package manager? Who developed it, key project milestones, etc.? I understand it comes from, or led to, NixOS Linux distribution which uses Nix of course. The nixos.org website is a bit sparse on the Nix project's history, and same with wikipedia (though separately mentions that NixOS was a research project started by Eelco Dolstra in 2003).

Nix (package manager):

- There is a version history in the release notes appendix of the manual. [0]

- As others have posted here, the project is currently hosted on github [1], though it was originally in a svn repo. [0]

- Eelco Dolstra is the active project maintainer. [2]

- First checkin for Nix was ~March 12, 2003 (in the file nix.c/nix.cc). [1]

- Version 1.0 release on May 11, 2012 (also first release from github). [0]

[0] Release Notes: http://nixos.org/nix/manual/#sec-relnotes

[1] https://github.com/NixOS/nix

[2] https://github.com/edolstra

Edit: Thanks to sibling posts for the other info they were able to dig up for us.

Re: Nix as OS X Package Manager

#177
post #83

Earlier quoted context omitted.

Nix has a way of doing this as well, just needs a better UX (like most of nix).

I was quite surprised that `nix install vim` didn't work out of the box. More so after realising that there's not even a `nix` command. I see that there's a bunch of `nix-whatever` tools around, but even their names aren't much sensible. I'm not in doubt that nix is technically superior to homebrew, but UX-wise it's at least been a disappointing experience for me.

What Nix for Mac needs more than anything is a great, simple overview of how to use it in practice, for casual users. I use it as my main package manager, but I install software with it rarely enough that I need to dig around in the manpages every time.

Re: Nix as OS X Package Manager

#179

Earlier quoted context omitted.

sounds like 'emerge hell' from gentoo 12 years ago.

Ex-Gentoo dev here. It is not the same. Gentoo emerge runs in place, and you can definitely break your system with a broken build. NixOS on the other hand, builds everything in sandboxes that only expose the requested dependencies. Builds are almost 100% deterministic. The cherry on top is that your system installation is simply a package consisting of all other packages and configuration symlinked together, and you…

the non-deterministic nature is what I was getting at.

I worked at a place where they'd spin up a new gentoo box for a dev with stock "emerge foo" and let everything run up.

Then 3 weeks later, they'd do it again for the next dev, and they'd have many diff versions.

I know they weren't "doing it right", but they were doing it the default way they learned, and it caused a lot of problems.

Thanks for the info/clarification though.

Re: Nix as OS X Package Manager

#180
post #153
post #69

Earlier quoted context omitted.

We wrote an article about the same thing https://blog.wearewizards.io/why-docker-is-not-the-answer-to... (Tom has a background in physics as well)

You are assuming "that reproducibility is good". I believe this assumption needs to be challenged.

Any significant contribution you might have for software engineering in regards to that?

Aside from merely believing that.

I can't even tell what you mean. If I want to rebuild my machine same as before, or deploy build 100s of servers at different points in time with the same setup, reproducibility is necessary.

There's not even one argument against that. If you have any, please oblige us.

Should deployments be a surprise?

Post reply on HN