Live data from Hacker News

LXD containers on macOS at near-native speeds

beringresearch.github.io

111–120 of 134 posts

Re: LXD containers on macOS at near-native speeds

#111

Earlier quoted context omitted.

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I h…

I was exaggerating slightly - you will get some images in ARM.

But it's hard to not see this as evidence for my point when you were only able to rebuild "one or two" of the 8-9 amd64 images you're running without significant time investment.

By your own stats roughly one third of the images you use don't support ARM. And if you want to run one of those on ARM - you're likely screwed!

Even some _official_ images don't support ARM, random example: https://hub.docker.com/_/percona

Docker ARM support is improving, but acting like it's ready for prime time is laughable. Try doing your day job with no amd64 containers for any reason for a month and come back to me.

Re: LXD containers on macOS at near-native speeds

#112

Earlier quoted context omitted.

Even systemd will run in Docker on macOS. Can you clarify what makes macOS unacceptable for emulating Linux?

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

> persistent wasted RAM/disk space, awful disk IO performance

I'm getting cranky about arguments against VMs.

1) I have had more RAM than I ever use for about 25 years. I upgrade about every 4 to 6 years. Is everyone else writing code on Amigas?

2) I keep getting told by everyone (IT, devops, DBA, my boss, his boss) that disk space is cheap.

3) VirtualBox has been consistently ok for about 10 years and I almost always can outrun scripts in a Linux VM on my Windows machine compared to our production AWS instance running K8s. Of course my machine would never keep up with a high user count but this is about disk performance, and locally I can prove both my monolith and my VM are faster.

I don't see how arguments like these are good anymore with the modern hardware of a local dev system. That makes containers an over complication. The only reason to do them is to save devops the headache of having to create a docker file. I mean the guys read books and watch YouTube while I'm writing code, so...they can write the damn docker file.

Re: LXD containers on macOS at near-native speeds

#113

Earlier quoted context omitted.

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

> persistent wasted RAM/disk space, awful disk IO performance I'm getting cranky about arguments against VMs. 1) I have had more RAM than I ever use for about 25 years. I upgrade about every 4 to 6 years. Is everyone else writing code on Amigas? 2) I keep getting told by everyone (IT, devops, DBA, my boss, his boss) that disk space is cheap. 3) VirtualBox has been consistently ok for about 10 years and I almost alway…

Disk space is cheap; IOPS aren't. Same stuff with RAM: it's cheap, but fast access isn't.

Re: LXD containers on macOS at near-native speeds

#114

Earlier quoted context omitted.

Even systemd will run in Docker on macOS. Can you clarify what makes macOS unacceptable for emulating Linux?

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

I don't use a VM for Docker. I ran systemd directly inside Docker for macOS.

https://github.com/LoganDark/initbuntu

Re: LXD containers on macOS at near-native speeds

#115

Earlier quoted context omitted.

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I h…

I was exaggerating slightly - you will get some images in ARM. But it's hard to not see this as evidence for my point when you were only able to rebuild "one or two" of the 8-9 amd64 images you're running without significant time investment. By your own stats roughly one third of the images you use don't support ARM. And if you want to run one of those on ARM - you're likely screwed! Even some _official_ images don't…

You had made a response to the following statement – the one I was responding to:

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in.

The statement is false. x86 containers do not use the translation, hence they can't run on M1. People have been successfully running aarch64 (ARM) containers, including on AWS Graviton (re:Invent 2018) and on Graviton2/3 (more recently), for a few years now. Oracle cloud have been offering Ampere cloud compute resources for… a couple of years now?

But you have conveniently dodged it, for it did not suit your agenda and presented it as «exaggerating slightly». It is not slight and it is not exaggeration, it is a blantantly distorted and also a likely ignorant statement.

> But it's hard to not see this as evidence for my point when you were only able to rebuild "one or two" of the 8-9 amd64 images you're running without significant time investment.

> By your own stats roughly one third of the images you use don't support ARM. And if you want to run one of those on ARM - you're likely screwed!

Whether one is screwed or not is a matter of perspective (I am certainly not the one), just as 23 aarch64 images are hardly «were only able to rebuild "one or two"». I have revised my 9 locally stored x86 images, thanks to you, and purged one stray one, with the remaining ones being, in fact, an images itself, and an Docker image tag slapped on it. Therefore, I only have 4x remaining x86 Docker images, but of course it is not going to convince you and you will resort to a distorted narrative again.

> Docker ARM support is improving, but acting like it's ready for prime time is laughable.

«Docker for ARM» means «Docker running on Linux/ARM». Docker has supported Linux, and Linux has support aarch64 for a long time now. Solutions I design for my clients have been running as/in aarch64 containers for a few years now, and my clients indeed now find it laughable how they could not switch to ARM sooner due to 30-40% lesser cloud compute bills. Most solutions do not require neither containers nor servers, though, it is only the platform related stuff.

> Try doing your day job with no amd64 containers for any reason for a month and come back to me.

If you don't understand the difference between the aarch64, and that the Apple instantiation of the aarch64 architecture is not the only one, then you it is unlikely that you understand what Docker is and what Docker is not. Therefore, it will be difficult for you to self-assess and quantify the magnitude of your own uproarious laughter. We are at a point now when the CPU has become a configuration setting at the infrastructure provisioning time, and is not an insurmountable barrier.

All of the containers I need to run for my current needs, run, and they do it perfectly well in mix and match Fargate container clusters. Most containers are Graviton2 containers, with a few select ones (the ones that are cumbersome to rebuild), run in AWS x86 containers, and both types coexist peacefully. Last but not least, I do not even run most containers locally, I occasionally help developers with build related problems or to cobble a POC together to verify an idea, evolve it into a solution or a product, and scrap the container setup. Everything else, including production workloads, runs in the cloud, and it just works 24x7.

Re: LXD containers on macOS at near-native speeds

#116
post #110

Earlier quoted context omitted.

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I h…

But then it's not the same container, right? Either you'd need to switch your whole cloud infrastructure to run on ARM (which I see is possible now, and I'd be curious to hear if people have done that successfully) or we're back to "works on my machine ^W architecture".

> Either you'd need to switch your whole cloud infrastructure to run on ARM …

Nope, the whole cloud infrastructure does not need to switch.

Since containers are fully self-contained, self-sufficient and isolated from each other, they can coexist peacefully and run alongside each other in a mixed setup in AWS ECS/Fargate clusters.

And they do. AWS Lambda also supports aarch64, and the degree of isolation for the lambdas is even higher and they can also run out of a x86 or an aarch64 Docker image (if required), and 30-40% cheaper with performance loss incurred for the workloads I am interested in.

Re: LXD containers on macOS at near-native speeds

#117

Earlier quoted context omitted.

Using a VM for Docker gives you the worst of both worlds. You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc). On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what y…

I don't use a VM for Docker. I ran systemd directly inside Docker for macOS. https://github.com/LoganDark/initbuntu

> I don't use a VM for Docker.

Yes you are.

https://github.com/moby/hyperkit

(Or maybe you mean a non-Docker For Mac VM)

Re: LXD containers on macOS at near-native speeds

#118
post #117

Earlier quoted context omitted.

I don't use a VM for Docker. I ran systemd directly inside Docker for macOS. https://github.com/LoganDark/initbuntu

> I don't use a VM for Docker. Yes you are. https://github.com/moby/hyperkit (Or maybe you mean a non-Docker For Mac VM)

Interesting. Docker for Mac uses that library for virtualization? I suppose that would explain how I was able to get systemd to run, then.

Re: LXD containers on macOS at near-native speeds

#119
post #40

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”. I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

Because docker for Mac has always kind of sucked, I’ve moved on to making nix-based development environments and I’ve been very pleased so far. Lately I’ve been trying https://devenv.sh/ and it works great! I haven’t tried it for ruby, though I have used vanilla nix shell for ruby projects before and it worked quite well after I over-rode GEM_PATH and GEM_HOME to the correct values.

While it looks really nice, "Without containers" at this point in time with the supplychain attacks doesn't seem to be a good thing.

Can devenv environment put itself in a chroot / sandbox where it cannot access my files, and is firewalled by default?

Re: LXD containers on macOS at near-native speeds

#120
post #30

I like the progress that is being made for running containerized workloads on macOS. In my case I like some of the benefits of running the workload on a remote machine; such as no fan noise, less heat, less power consumption (especially on laptops). However the downsides can be also quite annoying, such as file sync times or IDE input lag. My current setup is to have both data and workload run on a remote machine and…

If you're on Kubernetes remotely, Telepresence [0] might be worth a look. [0] https://www.telepresence.io

I was going to recommend DevSpace for kubernetes, looks like Telepresence has a lot of similar capabilities. I'll have to check that out as well.

https://www.devspace.sh/

Post reply on HN