Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LXD containers on macOS at near-native speeds

beringresearch.github.io

Post reply on HN
Print view View original

121–130 of 134 posts

Re: LXD containers on macOS at near-native speeds

#121

Post by e12e »

barefeg wrote: post #30

I like the progress that is being made for running containerized workloads on macOS. In my case I like some of the benefits of running the workload on a remote machine; such as no fan noise, less heat, less power consumption (especially on laptops). However the downsides can be also quite annoying, such as file sync times or IDE input lag. My current setup is to have both data and workload run on a remote machine and…

One useful tip for reducing the latency with ssh commands (like docker, with DOCKER_HOST pointing to ssh server - for example enabling tab/autocomplete for subcommands (which does ssh to lookup list on tab)) - is to use PersisControl with ssh:

https://docs.rackspace.com/blog/speeding-up-ssh-session-crea...

For reducing the latency of an interactive ssh session, there's mosh: https://mosh.org/

Re: LXD containers on macOS at near-native speeds

#122

Post by e12e »

amelius wrote: post #32
mekster wrote: post #10

Earlier quoted context omitted.

lxd is so much easier than alternatives like systemd-nspawn. 3 minutes search reveals an easy to follow tutorial. https://ubuntu.com/server/docs/containers-lxd

Thanks but I wasn't talking about lxd, but about the underlying technology.

Your initial comment mentions experimentation with lxd - did you mean experiment with lxc? The level of abstraction is very different (this is the raison-d'être for lxd, after all!).

Lxc is somewhat to lxd as jails are to ezjails...

Re: LXD containers on macOS at near-native speeds

#123

Post by tracker1 »

lstodd wrote: post #113
notjoemama wrote: post #112

Earlier quoted context omitted.

> persistent wasted RAM/disk space, awful disk IO performance I'm getting cranky about arguments against VMs. 1) I have had more RAM than I ever use for about 25 years. I upgrade about every 4 to 6 years. Is everyone else writing code on Amigas? 2) I keep getting told by everyone (IT, devops, DBA, my boss, his boss) that disk space is cheap. 3) VirtualBox has been consistently ok for about 10 years and I almost alway…

Disk space is cheap; IOPS aren't. Same stuff with RAM: it's cheap, but fast access isn't.

Even then, access is still significantly better on a local machine in a VM than it is on a shared server... You are effectively the only real user, generally speaking. The Gen4 NVME drive and 64gb+ ram on my local system is still better than shared server resources are generally going to be, short of dedicated (ie: expensive) server environments to do the same.

I'm not a huge fan of VM development, generally speaking... that said, my current work setup is Windows + WSL + Rancher Desktop + VS Code (remoting extensions) for nearly everything. My windows terminal default is set to WSL, so it's pretty close to in the box... I can launch windows apps from the wsl environment... the integration is pretty seamless. In terms of performance, if you're stuck on an older Windows version (I am), then you can limit the max memory that the WSL environment gets, similar to a VM... I usually reserve 8-12gb for my host environment and give the rest to my dev environment.

My personal desktop is Linux (Ubuntu-Budgie). My previous job the laptop was an M1 Max, which had faster ram/disk than the windows laptop for the current job, or my personal desktop (which is impressive to me).

In the end, for most things, it's not significant enough to worry about... The aarch macs and i/o have been more problematic than wsl and other x86 environments... I think as rosetta support improves with qemu and docker, it will change and get significantly better.

That said, with VS Code's remoting extensions, I can be working on a remote system nearly as effectively as local anyway... which is how I typically use my laptop. Wireguard to home, vs code w/ ssh to my personal desktop.

Re: LXD containers on macOS at near-native speeds

#124

Post by tracker1 »

cmeacham98 wrote: post #111
inkyoto wrote: post #104

Earlier quoted context omitted.

> On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. What on Earth are you talking about? arm64 Docker images predate the emergence of M1, and their number has proliferated in the last two years like there was no tomorrow. Here is an excerpt from my own tiny, non-representative collection of Docker images I h…

I was exaggerating slightly - you will get some images in ARM. But it's hard to not see this as evidence for my point when you were only able to rebuild "one or two" of the 8-9 amd64 images you're running without significant time investment. By your own stats roughly one third of the images you use don't support ARM. And if you want to run one of those on ARM - you're likely screwed! Even some _official_ images don't…

I'm a follower on the MS SQL Server Docker github repo... it's definitely a bigger issue for some than others. Depending on what the container/service does, the amd64 translation isn't too bad, it's the disk i/o that really sucks, which is being worked on/improved. Even then, more people need to understand in Mac/Windows how/when to use Volume containers for storage, not local/native/host disk directly, which helps a lot.

In the end, it'll get better... I'm happier with WSL than Mac, even if I personally prefer straight Linux.

Re: LXD containers on macOS at near-native speeds

#125

Post by tracker1 »

dchuk wrote: post #40

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”. I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

It's less than perfect and being worked on afaik... Main thing to avoid are any volume mounts to the host environment. Declare/use volume containers in your docker-compose.yaml, will help a lot, but still more sluggish than host/native.

If you use VS Code the remoting extensions pack includes SSH, for my personal laptop, I'm usually using code to my linux desktop, and editing remotely... it's pretty slick. Code will run a "code server" in the remote environment, and is effectively just displaying the text view to you, the actual work is happening in the remote envornment. You can do similar with containers, or in windows+wsl.

Re: LXD containers on macOS at near-native speeds

#126

Post by inkyoto »

inkyoto wrote: post #116
nxrabl wrote: post #110

Earlier quoted context omitted.

But then it's not the same container, right? Either you'd need to switch your whole cloud infrastructure to run on ARM (which I see is possible now, and I'd be curious to hear if people have done that successfully) or we're back to "works on my machine ^W architecture".

> Either you'd need to switch your whole cloud infrastructure to run on ARM … Nope, the whole cloud infrastructure does not need to switch. Since containers are fully self-contained, self-sufficient and isolated from each other, they can coexist peacefully and run alongside each other in a mixed setup in AWS ECS/Fargate clusters. And they do. AWS Lambda also supports aarch64, and the degree of isolation for the lambd…

> […] and 30-40% cheaper with performance loss incurred for the workloads I am interested in.

Should read: «… with no performance loss incurred for the workloads I am interested in». A typo crept in.

Re: LXD containers on macOS at near-native speeds

#127

Post by cmeacham98 »

inkyoto wrote: post #115
cmeacham98 wrote: post #111

Earlier quoted context omitted.

I was exaggerating slightly - you will get some images in ARM. But it's hard to not see this as evidence for my point when you were only able to rebuild "one or two" of the 8-9 amd64 images you're running without significant time investment. By your own stats roughly one third of the images you use don't support ARM. And if you want to run one of those on ARM - you're likely screwed! Even some _official_ images don't…

You had made a response to the following statement – the one I was responding to: > On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in. The statement is false. x86 containers do not use the translation, hence they can't run on M1. People have been successfully running aarch64 (ARM) containers, including on AWS…

> The statement is false. x86 containers do not use the translation, hence they can't run on M1.

Sorry, let me admit I actually don't own an M1/M2 Mac and have only used one by proxy helping my coworkers. To be completely honest, I just assumed it was possible to run amd64 containers on them through some form of translation (QEMU/Rosetta).

That actually makes my whole point stronger though - if the container you want doesn't support amd64, you either fix it yourself (often a time consuming and difficult process) or you just can't use it at all.

The rest of your post is just chasing ghosts. We're talking about M1 Macs, remember, not cloud servers? Thanks for letting me know about Graviton, but I am literally employed by AWS and build aarch64 containers as part of my day job, I'm well aware.

Re: LXD containers on macOS at near-native speeds

#128

Post by barefeg »

seabrookmx wrote: post #109
barefeg wrote: post #30

I like the progress that is being made for running containerized workloads on macOS. In my case I like some of the benefits of running the workload on a remote machine; such as no fan noise, less heat, less power consumption (especially on laptops). However the downsides can be also quite annoying, such as file sync times or IDE input lag. My current setup is to have both data and workload run on a remote machine and…

I've never noticed any input lag with VSCode remote SSH. I was under the impression the editor cached file contents (pulled entire files at a time over SSH) to resolve this issue. Or are you just talking about when using neovim?

For neovim I definitely experienced input lag. In VSCode is a bit more subtle; for example when opening new files and saving files with autoformatter

Re: LXD containers on macOS at near-native speeds

#129

Post by maherbeg »

tracker1 wrote: post #125
dchuk wrote: post #40

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”. I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

It's less than perfect and being worked on afaik... Main thing to avoid are any volume mounts to the host environment. Declare/use volume containers in your docker-compose.yaml, will help a lot, but still more sluggish than host/native. If you use VS Code the remoting extensions pack includes SSH, for my personal laptop, I'm usually using code to my linux desktop, and editing remotely... it's pretty slick. Code will…

I'm thinking about giving this a try but using an iPad connected to a monitor setup for this. I rarely code on the go for personal use, so this could work pretty well...

Re: LXD containers on macOS at near-native speeds

#130

Post by ColCh »

It uses almost same mounting tech as colima (9p)

Macpine: https://github.com/beringresearch/macpine/blob/71788e9c3c09c...

colima: https://github.com/abiosoft/colima/blob/7ebcf14a69158afa43b2...

So it seems that it has same performance as colima project as well.

As for IO performance, see this colima issue https://github.com/abiosoft/colima/issues/146#issuecomment-1...

Post reply on HN