Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LXC vs. Docker

earthly.dev

Post reply on HN
Print view View original

91–100 of 147 posts

Re: LXC vs. Docker

#91

Post by ansible »

buybackoff wrote: post #16

LXC via Proxmox is great for stateful deployments on baremetal servers. It's very easy to backup entire containers with the state (SQLite, Postgres dir) to e.g. NAS (and with TrueNAS then to S3/B2). Best used with ZFS raid, with quotas and lazy space allocation backups are small or capped. Nothing stops one from running Docker inside LXC. For development I usually just make a dedicated priviledged LXC container with…

We do something similar with btrfs as the filesystem. There have been some issues with btrfs itself, but the LXC side of this has worked pretty good. Any significant storage (such as project directories) is done with a bind mount into the container, so that it is easy to separately snapshot the data or have multiple LXC containers on the same host access the same stuff. That was more important when we were going to run separate LXC containers for NFS and Samba fileservers, but we ended combining those services into the same container.

Re: LXC vs. Docker

#92

Post by wanderr »

adamgordonbell wrote: post #2

I like the docker way of one thing, one process, per container. LXC seems a bit different. However, an exciting thing to me is the Cambrian explosion of alternatives to docker: podman, nerdctl, even lima for creating a linux vm and using containerd on macos looks interesting.

That seems weird for some stacks though, like nginx, php-fpm, php. At least I still haven't wrapped my head around what's the right answer for the number of containers involved there.

Re: LXC vs. Docker

#93

Post by aaronius »

petre wrote: post #44
buybackoff wrote: post #16

LXC via Proxmox is great for stateful deployments on baremetal servers. It's very easy to backup entire containers with the state (SQLite, Postgres dir) to e.g. NAS (and with TrueNAS then to S3/B2). Best used with ZFS raid, with quotas and lazy space allocation backups are small or capped. Nothing stops one from running Docker inside LXC. For development I usually just make a dedicated priviledged LXC container with…

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

That highly depends on the underlying storage. If it is something that supports snapshots (ZFS, Ceph, LVM thin) then it should work fine, also backups will be possible without any downtime as they will be read from a temporary snapshot.

Re: LXC vs. Docker

#94

Post by stickyricky »

zelphirkalt wrote: post #87
stickyricky wrote: post #24

Earlier quoted context omitted.

Is there a benefit to this over SSH or VSCode remote?

Neither SSH not VSCode offer any kind of isolation out of the box.

I mean running docker on the remote machine and just sshing into it. I assume changing the docker host on OSX just means a command is being sent over the network. Just wondering why prioritize "local" development if its all remote anyway.

Re: LXC vs. Docker

#95

Post by _448 »

yokem55 wrote: post #42

LXD (Canonical's daemon/API front end to lxc containers) is great -- as long as you aren't using the god awful snap package they insist on. The snap is probably fine for single dev machines, but it has zero place in anything production. This is because canonical insists on auto-updating and refreshing the snap at random intervals, even when you pin to a specific version channel. Three times I had to manually recover…

> The snap is probably fine for single dev machines

It is not good even on single dev machines.

Re: LXC vs. Docker

#96

Post by sreevisakh »

pkulak wrote: post #59
yokem55 wrote: post #42

LXD (Canonical's daemon/API front end to lxc containers) is great -- as long as you aren't using the god awful snap package they insist on. The snap is probably fine for single dev machines, but it has zero place in anything production. This is because canonical insists on auto-updating and refreshing the snap at random intervals, even when you pin to a specific version channel. Three times I had to manually recover…

Not even kidding, a huge part of what made me move to Arch was that it's one of the few distros that packages LXD. Apparently it's a pain, but I'm forever grateful!

Alpine is another distro that packages LXD. I have Arch on my workstation, but I'm not confident about securing Arch on a server. Alpine, on the other hand, is very much suited to be an LXD host. It's tiny, runs entirely from RAM and can tolerate disk access failures. Modifications to host filesystem won't persist after reboot, unless the admin 'commits' them. The modifications can be reviewed before a commit - so it's easy to notice any malicious modifications. I also heard a rumor that they are integrating ostree.

The only gripe I have with Alpine is its installation experience. Like Arch, Alpine has a DIY type installation (but a completely different style). But unlike Arch, it isn't easy to properly install Alpine without a lot of trial and error. Alpine documentation felt like it neglects a lot of important edge cases that trip you up during installation. Arch wiki is excellent on that aspect - they are likely to cover every misstep or unexpected problem you may encounter during installation.

Re: LXC vs. Docker

#97

Post by rajishx »

yokem55 wrote: post #42

LXD (Canonical's daemon/API front end to lxc containers) is great -- as long as you aren't using the god awful snap package they insist on. The snap is probably fine for single dev machines, but it has zero place in anything production. This is because canonical insists on auto-updating and refreshing the snap at random intervals, even when you pin to a specific version channel. Three times I had to manually recover…

I am simple man with simple need, I am perfectly happy with a distro as long I have my editor my terminal and my browser.

I could not bear the snaps on ubuntu always coming back and hard to disable on every update, I gave up and just switched to arch and happy to have control on my system again.

I had a lot of crash running on Ubuntu when running huge rust based test suite doing a lot of IO (on btrfs), never had that issue on arch. not sure why, not sure how I can even debug it (full freeze, nothing in systemd logs) so I guess I just gave up.....

Re: LXC vs. Docker

#98

Post by buybackoff »

aaronius wrote: post #93
petre wrote: post #44

Earlier quoted context omitted.

It's an annoying that you can only make snapshots on a stopped container. With VMs it works in a running VM.

That highly depends on the underlying storage. If it is something that supports snapshots (ZFS, Ceph, LVM thin) then it should work fine, also backups will be possible without any downtime as they will be read from a temporary snapshot.

Even with ZFS you still have to wait for RAM to dump, haven't you? And it will freeze at least for the dump write time. Do they have CoW for container memory?

But even if they had, the RAM snapshot needs to be written, but without freezing the container. I would appreciate an option when I could ignore everything that was not fsyned, e.g. Postgres use case. In that case the normal ZFS snapshot should be enough.

Re: LXC vs. Docker

#99

Post by aaronius »

buybackoff wrote: post #98
aaronius wrote: post #93

Earlier quoted context omitted.

That highly depends on the underlying storage. If it is something that supports snapshots (ZFS, Ceph, LVM thin) then it should work fine, also backups will be possible without any downtime as they will be read from a temporary snapshot.

Even with ZFS you still have to wait for RAM to dump, haven't you? And it will freeze at least for the dump write time. Do they have CoW for container memory? But even if they had, the RAM snapshot needs to be written, but without freezing the container. I would appreciate an option when I could ignore everything that was not fsyned, e.g. Postgres use case. In that case the normal ZFS snapshot should be enough.

RAM and other state can be part of a snapshot for VMs, in which case the VM will continue right where it was.

The state of a container is not part of the snapshot (just checked), as it is really hard to capture the state of the container (CPU, network, all kinds of file and socket handles) and restore it because all an LXC container is, is local processes in their separate cgroups. This is also the reason why a live migration is not really possible right now, as all that would need to be cut out from the current host machine and restore in the target machine.

This is much easier for VMs as Qemu offers a nice abstraction layer.

Re: LXC vs. Docker

#100

Post by rascul »

pkulak wrote: post #59
yokem55 wrote: post #42

LXD (Canonical's daemon/API front end to lxc containers) is great -- as long as you aren't using the god awful snap package they insist on. The snap is probably fine for single dev machines, but it has zero place in anything production. This is because canonical insists on auto-updating and refreshing the snap at random intervals, even when you pin to a specific version channel. Three times I had to manually recover…

Not even kidding, a huge part of what made me move to Arch was that it's one of the few distros that packages LXD. Apparently it's a pain, but I'm forever grateful!

openSUSE packages lxd
Post reply on HN