Nix as OS X Package Manager

> Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse.

What exactly do you mean by flexible reuse, and why is it good? What exactly do you mean by NON updated parts and what does it mean that they are "producible"?

I don't understand your words. Speak plainly!

I don't use a Dockerfile, and the interactive development approach means normally you start up your clone, work on it for a bit, then commit the clone.

Here's an example:

    $ docker run -t -i debian:jessie bash
    root@264b51d62d0e:/# apt-get install nvi
    E: Unable to locate package nvi
    root@264b51d62d0e:/# apt-get update
    root@264b51d62d0e:/# apt-get install nvi
    $ v=`docker commit 264b51d62d0e`
    $ docker tag $v whatever

The mistake with the first line didn't mean I needed to start over. In a realistic example that could save me an hour of rebuilding. I can push my tag to other developers I'm working with, and they can use `docker diff` to find out what the differences are.

> But until one reaches that, they have to experiment with various configurations (to end with the specific Docker image they want). Without reproducibility, they can't be guaranteed that their various test configurations under test are the same and only their latest changes differ. Perhaps another thing they left as is broke too.

This is a danger that requires discipline. Committing instances is very cheap, so it is good practice to commit often, and keep notes (commit messages). I think version control is common enough that most sysadmins know how to do this, and other packaging systems suffer from the exact same problem.

However because it's interactive, that need to experiment is satisfied fully and the sysadmin/user gets to use all of their tools to develop the working prototypes.

> But if you mess with the clone it's not a clone anymore. And if a whole team has to mess with the clone over some period of time to update or fix it, there's hell to keep track what went on.

This is not true.

With Docker you are encouraged to make many clones and branches and try different things out. The clones that are useful get tagged and forwarded to others.

A sysadmin who isn't using version control has other problems!

> Only offering cloning also means that when you need to update only a part (e.g. because of security reasons) you can't just use the same dockerfile with the changed component and be sure that you'll get an otherwise identical system. So you don't get flexible updating.

I don't understand this complaint, but it mentions dockerfiles again. Let me be clear: I never use dockerfiles.

If I want to update for security reasons (as I mentioned in my heartbleed example), then I will find it easier and faster to implement the fixes interactively. Having to write my recipe for nix-os and run it and wait while it runs is insecure because you are vulnerable longer.

> Besides, reproducibility doesn't preclude cloning as part of the process -- it's a superset of it, offering way more flexibility.

The non-interactive (and "reproducible") approach is slower, less secure, more prone to errors. I don't believe it can possibly be "more flexible" since the interactive approach can do everything the non-interactive approach can do and faster.

I do admit that Docker's build artefacts use more disk space and more Internet bandwidth than plain text nix configuration artefacts, but this is not fundamental to interactive development, which is specifically what I'm advocating.

> Any significant contribution you might have for software engineering in regards to that?

> That's just an argument from popularity. And not necessarily a popularity they opted for, while also being provided with the alternative.

Arguing that "nobody does it so it must be wrong" isn't any better than "just because everyone does it doesn't make it right".

I really think you need to learn about something and make up your own mind, instead of repeating "Docker is Bad" because you read something like that on a blog.

I also think you should really try to understand what I am suggesting before you go arguing against it.

> This is called dogma,

Then you don't know what the word "dogma" means.

What I said was "In my experience." i.e. practice learned over time.

This cannot be interpreted by any reasonable person as "blobs are bad, just 'cause."

That kind of straw man argument leads me to conclude that similar issues are being your inability to understand the use of reproducability.

> How do you know that the docker image was created with the right set of software?

Define "the right set". If I ask someone to prepare something I might use

    docker export/diff

in part of the review process.

> How do you know that there wasn't anything else added to it?

    docker diff

> Copying the final image means you're just copying an opaque binary blob. You can't do the work yourself to build an image, and get the same docker image.

So what?

Why exactly do you think "copying an opaque binary blob" is bad?

Don't you ever copy files on your hard disk?

Have you ever used `dd` to copy a filesystem as a binary blob to a disk or a USB key?

Did you know "git" copies "opaque binary blobs" around? Do you really prefer RCS because the version control is stored in plain text files?

What exactly is your complaint here?

My position is simple, but apparently radical: I recommend interactive development of systems because it is better, and I define better as faster and more secure. "Reproducible" simply is not a goal.

> For me, it's not about ease of use. It's about security and trust.

You should rethink your position.

Nix packages from sources you don't trust aren't more secure than docker images (even if you use nix.useChroot=true), which, barring bugs in docker and Linux, at least get virtual memory, disk, and networking.

Nix packages built by yourself are harder to write than docker tags you build yourself.

> Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse.

> What exactly do you mean by flexible reuse, and why is it good?

I already explained. What exactly don't you understand?

It's about being able to reuse a recipe (dockerfile, etc.) for creating your final artifact while ALSO being able to change it -- as opposed to the "take it or leave it" case with cloning some binary blob.

> What exactly do you mean by NON updated parts

I wrote "Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be (re)producible".

My typo aside, the meaning is clear: with cloning available but no reproducibility, when you want to only change part of a container (e.g. update a specific piece of software there), you don't know (when you re-create the container) that the other parts you didn't change are as they were before.

Basically that's the very definition of non-reproducibility.

> I don't use a Dockerfile, and the interactive development approach means normally you start up your clone, work on it for a bit, then commit the clone.

That's neither a good, nor a new process. That's like 90's sysadmin work. For one, you're doing all this work manually. Second, if you want to revert/change something you made, you have to mess with your clone, potentially putting it in some weirdo state -- you basically remove the scriptability part and just mess with your clone manually. Or you write some custom scripts or use some provisioning software -- which brings you right back to the reproducibility discussion.

With reproducibility you wouldn't have to commit the clone -- just the "recipe" to make. And you would still COULD commit the clone if you wanted (as I said, reproducibility is a superset of merely working with clones).

> The mistake with the first line didn't mean I needed to start over. In a realistic example that could save me an hour of rebuilding.

Doesn't save you anything over reproducibility -- since the latter does not prevent cloning. But having only cloning (which is what the parent lamented) does deprive you of the benefits of reproducibility.

"In a realistic example", for example, things could get much more hairy (and many more tests and changes could be needed), beyond merely forgetting to update apt before installing nvi.

And all of these changes and process would be totally opaque to your clonable blob.

> I can push my tag to other developers I'm working with, and they can use `docker diff` to find out what the differences are.

The can use docker diff and try to guess what the differences are -- and what the intention was, etc. Because docker diff is just a record of file changes, not of procedures followed and the intentions behind them.

> This is a danger that requires discipline.

And that's (part) of the whole problem. Anything that requires discipline on the human part and that could be automated is busy-work -- and error prone.

> With Docker you are encouraged to make many clones and branches and try different things out.

Yes, but that's meant for safe-keeping, like vm-snapshots. It doesn't replacing actually having a written record of what you did, what's supposed to be istalled inside, and why.

> I don't understand this complaint, but it mentions dockerfiles again. Let me be clear: I never use dockerfiles.

So again, a throwback to the manual devops age.

> Arguing that "nobody does it so it must be wrong" isn't any better than "just because everyone does it doesn't make it right".

Perhaps. But I fail to see where I did that.

> repeating "Docker is Bad"

Who said Docker is bad? Docker is great. It's non reproducibility that's the issue.

Is there any good guides to get started on nix with python?