Live data from Hacker News

LXC vs. Docker

earthly.dev

61–70 of 147 posts

Re: LXC vs. Docker

#61
post #54
post #52

Earlier quoted context omitted.

Running the same script every time doesn't necessarily guarantee the same result. Lots of docker build scripts have the equivalent of date > file.txt or curl https://www.random.org/integers/?num=1&min=1&max=1000&col=1&base=10&format=plain&rnd=new > file.txt Buried deep somewhere in the code. But yeah I don't see any reason why you couldn't theoretically make a reproducible build with Docker.

Docker images actually contain the timestamp each layer was built at, so are basically de-facto non-reproducible. Buildah from Red Hat has an argument to set this programmatically instead of using the current date, but AFAIK there's no way to do that with plain old docker build.

The nixpkgs workaround is to build images with build-date == 1970-01-01. Annoying but reproducible.

Re: LXC vs. Docker

#62
post #44

LXC via Proxmox is great for stateful deployments on baremetal servers. It's very easy to backup entire containers with the state (SQLite, Postgres dir) to e.g. NAS (and with TrueNAS then to S3/B2). Best used with ZFS raid, with quotas and lazy space allocation backups are small or capped. Nothing stops one from running Docker inside LXC. For development I usually just make a dedicated priviledged LXC container with…

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

Also can't do live migrations or backups and moving storage around is a headache.

We've pretty much stopped using LXC containers in Proxmox because of all the little issues.

Re: LXC vs. Docker

#63
post #42

LXD (Canonical's daemon/API front end to lxc containers) is great -- as long as you aren't using the god awful snap package they insist on. The snap is probably fine for single dev machines, but it has zero place in anything production. This is because canonical insists on auto-updating and refreshing the snap at random intervals, even when you pin to a specific version channel. Three times I had to manually recover…

Makes you wonder whether Canonical has any idea about operating servers. Auto-updating packages is the last thing you want. Doing that for a container engine, without building in some jitter to avoid the scenario you described is absolutely insane. Who even uses snap in production? If I squint my eyes I can see the use for desktops, but why insist on it for server technologies as well?

Canonical would gladly hand back full control of updates if you pay them for an "enterprise edition" snap store. https://ubuntu.com/core/docs/store-overview#:~:text=Brand%20....

Re: LXC vs. Docker

#64
post #52
post #46

Earlier quoted context omitted.

Can you expand on how builds aren't reproducible? I though Dockerfile ensured that builds are indeed reproducible?

Running the same script every time doesn't necessarily guarantee the same result. Lots of docker build scripts have the equivalent of date > file.txt or curl https://www.random.org/integers/?num=1&min=1&max=1000&col=1&base=10&format=plain&rnd=new > file.txt Buried deep somewhere in the code. But yeah I don't see any reason why you couldn't theoretically make a reproducible build with Docker.

If you have dynamism like this in your build, doesn't that imply that no build system is reproducible?

Re: LXC vs. Docker

#65

Earlier quoted context omitted.

I recently started using containerd inside Nomad, a breath of fresh and simple air after failed k8s setups!

Oh, Nomad looks interesting. Why should someone reach for it vs K8S?

Cloudflare recently posted a great blog article on how/why they use nomad: https://blog.cloudflare.com/how-we-use-hashicorp-nomad/

Re: LXC vs. Docker

#66

Apples to oranges. LXC can be directly compared with a small, and quite insignificant, part of Docker: container runtime. Docker became popular not because it can run containers, many tools before Docker could do that (LXC included). Docker became popular because it allows one to build, publish and then consume containers.

I would say docker's killer feature is the Dockerfile. It makes it understandable, reproducible and available to a broad range of people.

At least they mentioned it in their apple:oranges comparison.

there's also the global namespace thing. "FROM ubuntu:18.04" is pretty powerful.

I run a proxmox server with LXC but if I could use a dockerfile or equivalent, my containers would be much much more organized. I wouldn't like to pull images from the internet however.

Re: LXC vs. Docker

#67
post #66

Apples to oranges. LXC can be directly compared with a small, and quite insignificant, part of Docker: container runtime. Docker became popular not because it can run containers, many tools before Docker could do that (LXC included). Docker became popular because it allows one to build, publish and then consume containers.

I would say docker's killer feature is the Dockerfile. It makes it understandable, reproducible and available to a broad range of people. At least they mentioned it in their apple:oranges comparison. there's also the global namespace thing. "FROM ubuntu:18.04" is pretty powerful. I run a proxmox server with LXC but if I could use a dockerfile or equivalent, my containers would be much much more organized. I wouldn't…

Understandable? Yes.

Reproducible? No. Most Dockerfiles are incredibly unreproducible.

Re: LXC vs. Docker

#68
post #19

I like the docker way of one thing, one process, per container. LXC seems a bit different. However, an exciting thing to me is the Cambrian explosion of alternatives to docker: podman, nerdctl, even lima for creating a linux vm and using containerd on macos looks interesting.

Docker can have N processes per container though, just depends how you set up your image

Yes, and it makes sense in some cases. Supervisord is awesome for this.

Re: LXC vs. Docker

#69

Earlier quoted context omitted.

I got so annoyed with snapd that I finally patched the auto-update functionality to provide control via environment variable. It's ridiculous that this is what I have to personally go through in order to maintain control of when updates are applied on my own systems. If enough people were to ever decide to get together and properly fork snapd and maintain the patched version I'd totally dedicate time to helping out.…

We blocked the annoying snapd autoupdate behavior by setting a http proxy to a nonexistent server. Whenever we had a maintenance window we would unset the proxy, allow the update, then set the set the nonexistent proxy server again. Very annoying.

this feels both clever and stupid at the same time - not you but the software games you have to play.

Re: LXC vs. Docker

#70

Apples to oranges. LXC can be directly compared with a small, and quite insignificant, part of Docker: container runtime. Docker became popular not because it can run containers, many tools before Docker could do that (LXC included). Docker became popular because it allows one to build, publish and then consume containers.

> Docker became popular because it allows one to build, publish and then consume containers. True, but Docker is an awful choice for those things (builds are performed "inside out" and aren't reproducible, publishing produces unauditable binary-blobs, consumption bypasses cryptographic security by fetching "latest" tags, etc.)

> builds are performed "inside out" and aren't reproducible

This is probably a good argument because of how hard it is to do anything in a reproducible manner, if you care even about timestamps and such matching up.

Yet, i'd like to disagree that it's because of inherent flaws with Docker, merely how most people choose to build their software. Nobody wants to use their own Nexus instance as a storage for a small set of audited dependencies, configure it to be the only source for all of them, build their own base images, seek out alternatives for all of the web integrated build plugins etc.

Most people just want to feed the machine a single Dockerfile (or the technology specific equivalent, e.g. pom.xml) and get something that works out and thus the concerns around reproducibility get neglected. Just look at how much effort the folks over at Debian have put into reproducibility: https://wiki.debian.org/ReproducibleBuilds

That said, a decent middle ground is to use a package cache for your app dependencies, specific pinned versions of base images (or build your own ones on top of the common ones, e.g. a customized Alpine base image) and multi stage builds, you are probably 80% of the way there, since if need be, you could just dump the image's file system and diff it against a known copy.

Nexus (some prefer Artifactory, some other solutions): https://www.sonatype.com/products/repository-oss

Multi stage builds: https://docs.docker.com/develop/develop-images/multistage-bu...

The rest 20% might take a decade until reproducibility is as user friendly as Docker currently is, just look at how slowly Nix is adopted.

> publishing produces unauditable binary-blobs

It's just a file system that consists of a bunch of layers, isn't it? What prevents you from doing:

  docker run --name dump-test alpine:some-very-specific-version sh -c exit
  docker export -o alpine.tar dump-test
  docker rm dump-test
You get an archive that's the full file system of the container. Of course, you still need to check everything that's actually inside of it and where it came from (at least the image persists the information about how it was built normally), but to me it definitely seems doable

> consumption bypasses cryptographic security by fetching "latest" tags

I'm not sure how security is bypassed if the user chooses to use whatever is the latest released version. That just seems like a bad default on Docker's part and a careless action on the user's part.

Actually, there's no reason why you should limit yourself to just using tags, since something like "my-image:2022-02-18" might be accidentally overwritten unless your repo specifically prevents this from being allowed. If you want, you can actually run images by their hashes, for example, Harbor makes this easy to do by letting you copy those values from their UI, though you can also do so manually.

For example, let's say that we have two Dockerfiles:

  # testA.Dockerfile
  FROM alpine:some-very-specific-version
  RUN mkdir /test && echo "A" > /test/file
  CMD cat/test/file
  
  # testB.Dockerfile
  FROM alpine:some-very-specific-version
  RUN mkdir /test && echo "B" > /test/file
  CMD cat/test/file
If we use just tags to refer to the images, we can eventually have them be overriden, which can be problematic:

  # Example of using version tags, possibly problematic
  docker build -t test-a -f testA.Dockerfile .
  docker run --rm test-a
  docker build -t test-a -f testB.Dockerfile .
  docker run --rm test-a
In the second case we get the "B" output even though the tag is "test-a" because of a typo, user error or something else. Yet, we can also use hashes:

  # Example of using hashes, more dependable
  docker build -t test-a -f testA.Dockerfile .
  docker image inspect test-a | grep "Id"
  docker run --rm "sha256:93ee9f8e3b373940e04411a370a909b586e2ef882eef937ca4d9e44083cece7c"
  docker build -t test-a -f testB.Dockerfile .
  docker image inspect test-a | grep "Id"
  docker run --rm "sha256:8dd9ba5f1544c327b55cbb75f314cea629cfb6bbfd563fe41f40e742e51348e2"
  docker build -t test-a -f testA.Dockerfile .
  docker image inspect test-a | grep "Id"
  docker run --rm "sha256:93ee9f8e3b373940e04411a370a909b586e2ef882eef937ca4d9e44083cece7c"
Here we see that if your underlying build is reproducible, then the resulting image hash for the same container will be stable. Furthermore, someone overwriting the test-a tag didn't break you being able to run the first correctly built image because the tags are just convenience, so you'll be able to run the previous one.

Of course, that loops back to the reproducible build discussion if you care about hashes matching up, rather than just tags not being overwritten.

Post reply on HN