Live data from Hacker News

LXD containers on macOS at near-native speeds

beringresearch.github.io

101–110 of 134 posts

Re: LXD containers on macOS at near-native speeds

#101

Earlier quoted context omitted.

Rosetta doesn’t help here because the kernel would be ARM, and the parent commenter wants an x86-64 kernel.

X86 userspace on an arm kernel is exactly what Rosetta would be used for.

Sure, but they are talking about an Intel kernel.

Re: LXD containers on macOS at near-native speeds

#103
post #73

Earlier quoted context omitted.

I've recently switched to using nix-shell and overmind with a Procfile to officially retired docker-compose from all of my projects. Such a breath of fresh air. Like Mitchell Hashimoto (Hashicorp), I no longer do any development on macOS directly; I have a Linux VM where all my projects live and have their environments managed with a shell.nix file. Highly highly recommend it.

If you’re using nix, why are you using a VM? Nix on macOS works pretty well these days.

VMs are better for my particular use case as I am regularly shifting between my Macbook and my Windows desktop with WSL2.

Re: LXD containers on macOS at near-native speeds

#104

Earlier quoted context omitted.

Even systemd will run in Docker on macOS. Can you clarify what makes macOS unacceptable for emulating Linux?

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in.

What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I have had to deal with lately:

  $ (for i in `docker images | tail -n +2 | awk '{ printf "%s:%s\n", $1, $2 }'`; do docker inspect $i | jq '.[].Architecture'; done) | grep -c amd64  
7

  $ (for i in `docker images | tail -n +2 | awk '{ printf "%s:%s\n", $1, $2 }'`; do docker inspect $i | jq '.[].Architecture'; done) | grep -c arm64
23

Most of the 23 arm64 Docker images is pretty obscure stuff, with one or two manually rebuilt for the arm64 architecture. Yet, I have downloaded all of my obscure images directly from the Docker Hub – not traded them for a pinch of spice from a shady Docker image dealer at a drench inducing corner of the dark web. All of the arm64 images run at the native speed, locally and in the cloud.

The only reason I store x86 (i.e. «amd64») images locally is my laptop being a transient transit point between their source and the destination. I would have rebuilt them as well but the build process is either broken for each of them or is too convoluted to justify spending a time trying to fix it.

Re: LXD containers on macOS at near-native speeds

#105
post #73

Earlier quoted context omitted.

I've recently switched to using nix-shell and overmind with a Procfile to officially retired docker-compose from all of my projects. Such a breath of fresh air. Like Mitchell Hashimoto (Hashicorp), I no longer do any development on macOS directly; I have a Linux VM where all my projects live and have their environments managed with a shell.nix file. Highly highly recommend it.

What do you use to run the VM? Do you try to minimize overhead at all?

My general setup mirrors this[1] pretty closely, though of course on Windows I can just use WSL2.

[1]: https://m.youtube.com/watch?v=ubDMLoWz76U

Re: LXD containers on macOS at near-native speeds

#106
post #40

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”. I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

> slow ass filesystem always ruined it in the past.

Make sure that you're running a recent version of Lima, and use a 9p mount.

Edit: by editing one of these yamls: https://github.com/lima-vm/lima/tree/master/examples

Re: LXD containers on macOS at near-native speeds

#107

Earlier quoted context omitted.

Even systemd will run in Docker on macOS. Can you clarify what makes macOS unacceptable for emulating Linux?

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

At this point I’ve only got one non-arm64 container, and it’s solely optional support for testing some obscure cases that our CI infrastructure handles more robustly anyways.

Your information is pretty out of date.

Re: LXD containers on macOS at near-native speeds

#108
post #70
post #11

This is part of the bigger Macpine project, which to me is much more interesting than LXD: https://github.com/beringresearch/macpine """ The goal of this project is to enable MacOS users to: Easily spin up and manage lightweight Alpine Linux environments. Use tiny VMs to take advantage of containerisation technologies, including LXD and Docker. Build and test software on x86_64 and aarch64 systems """

It feels like lima and colima are much more mature. Being unable to remap the SSH port (or other ports) alone makes this a non-starter.

At the end of the day, are they all just wrappers around QEMU?

The aarch64 -> x86_64 virtualization performance is pretty rough.

Like, to the point where compiling a small hobbyist Rust project that needs to be deployed on Linux x86_64 in the cloud (cheap VPS)... you might as well just build it in the cloud and not locally on Mac host because it takes 10 minutes instead of 10 seconds

Re: LXD containers on macOS at near-native speeds

#109
post #30

I like the progress that is being made for running containerized workloads on macOS. In my case I like some of the benefits of running the workload on a remote machine; such as no fan noise, less heat, less power consumption (especially on laptops). However the downsides can be also quite annoying, such as file sync times or IDE input lag. My current setup is to have both data and workload run on a remote machine and…

I've never noticed any input lag with VSCode remote SSH. I was under the impression the editor cached file contents (pulled entire files at a time over SSH) to resolve this issue.

Or are you just talking about when using neovim?

Re: LXD containers on macOS at near-native speeds

#110

Earlier quoted context omitted.

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I h…

But then it's not the same container, right? Either you'd need to switch your whole cloud infrastructure to run on ARM (which I see is possible now, and I'd be curious to hear if people have done that successfully) or we're back to "works on my machine ^W architecture".
Post reply on HN