Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LXC and LXD: a different container story

lwn.net

Post reply on HN
Print view View original

91–100 of 102 posts

Re: LXC and LXD: a different container story

#91

Post by zekrioca »

sreevisakh wrote: post #47
jordemort wrote: post #12

Hi, author of the article, pleased to see it here! I'd really like to hear more from folks about how they're using LXC and/or LXD, and what they think their greatest strengths are compared to Docker or Kubernetes.

I use Kubernetes within an LXD container with Btrfs backing storage. This isn't anything special. But it has two advantages. The first is that you could try out multi-node K8s clusters on a single system. The second is that the containers can be deleted and rebuilt easily without affecting the host system. Both are very useful when you're learning multi-node K8s. I plan to expand my homelab to true multi-node setup.…

Do you have instructions (e.g., tutorial or documentation) on what needs to be done to have this setup? There are many configurations that would need to make it work properly. Thanks!

Re: LXC and LXD: a different container story

#92

Post by zekrioca »

kwk1 wrote: post #52
discardedrefuse wrote: post #15

I ran LXD for about a year in my home lab. Unfortunately, the easiest way to get it running is by installing Snap; which I didn't do. Instead I ran it on Alpine Edge (one of the few distros that actually has it in their package manager). LXD kept breaking after system updates and I got tired of troubleshooting. I suppose that's just part of the perils of running bleeding edge. When LXD was running, I found that it fe…

> Unfortunately, the easiest way to get it running is by installing Snap; The Debian package for LXD just entered the archive about 2 weeks ago: https://tracker.debian.org/news/1361535/accepted-lxd-500-1-s... This means the next Debian release will make for stable LXD hosts without needing snap.

Finally!

Re: LXC and LXD: a different container story

#93

Post by ncrmro »

locusofself wrote: post #62
0xbadcafebee wrote: post #36

Earlier quoted context omitted.

> we needed our SaaS to also work on-prem for financial companies and government entities, and thus we could not count on kubernetes or any specific cloud vendor ....? kubernetes is not a cloud vendor, it runs on any Linux distribution, and it's FOSS.... ?!

I mean that bringing kubernetes to an on-premises customer, just to run our software, was (probably correctly) deemed to be too much complexity. In the end it was complex enough as it was. 5 years ago kubernetes on-prem was not particularly easy, it probably still isn't, and doing it for a customer who just wants to run our software is a lot of support burden.

K3os makes on prem quite nice these days

Re: LXC and LXD: a different container story

#94

Post by tinus_hn »

nullwarp wrote: post #46
jordemort wrote: post #12

Hi, author of the article, pleased to see it here! I'd really like to hear more from folks about how they're using LXC and/or LXD, and what they think their greatest strengths are compared to Docker or Kubernetes.

We use LXC on physically distributed Proxmox nodes. It let's us easily launch new VMs and migrate around when we need to. Nothing we have is very complicated but it works well. We use SaltStack to tear up special debug containers with a bunch of integrated tools. To update Saltstack just downloads the new container and creates a new container in Proxmox. I guess you could technically replace what we built with pure D…

Even though I am sure I am perfectly capable of figuring out the command line incantations to make these containers work, I still really appreciate the convenience of the Proxmox environment where it’s something I don’t have to think about.

Re: LXC and LXD: a different container story

#95

Post by fuzzy2 »

fariszr wrote: post #88
fuzzy2 wrote: post #82

Earlier quoted context omitted.

Yes, it’s Hetzner. The magic ingredient is ipvlan , a special interface type loosely related to macvlan . An ipvlan interface is tied to a physical interface. Whether traffic goes to the virtual interface is decided based on the IP address alone. The ipvlan virtual interface can be moved to a different network namespace and will still work. Note that Hetzner added support for multiple MAC addresses in the meantime. S…

> Note that Hetzner added support for multiple MAC addresses in the meantime. So at least for IPv4, you don’t need this. Does this apply to the IP supplied with the server? Because I can't find any Mac address option for it.

No, the primary IP address cannot have its MAC address changed.

Re: LXC and LXD: a different container story

#96

Post by dizhn »

fariszr wrote: post #50
fuzzy2 wrote: post #27

I fucking love LXC! It allowed me to “virtualize” several physical servers on a single newer server, keeping all of the original setup (except some networking changes). Combined with ipvlan , I could even work around the MAC address restriction of my server provider, with both IPv4 and IPv6. I also use it to host a Jitsi instance. Jitsi is rather picky about the underlying distribution. All this without the overhead…

You are using hetzner right? I'm interested to see how you bypassed the Mac address problem, any blogs, guides ?

What mac address restriction is there? One sets them up on their robot interface so things will actually route. You get the assigned mac and use that in your virt ip or bridge settings.

Are we talking about something else?

Re: LXC and LXD: a different container story

#97

Post by dizhn »

dspillett wrote: post #57
fariszr wrote: post #50

Earlier quoted context omitted.

You are using hetzner right? I'm interested to see how you bypassed the Mac address problem, any blogs, guides ?

I'm not sure what the problem is, but if it is stopping you bridging the physical interface effectively, could you not define a virtual bridge with local only addresses and have the host NAT to/from that and the physical interface? Adds a little extra latency to be sure, more so if you have two or more boxes on the same network with containers that would otherwise talk directly to each other on a private vlan, but if…

They also provide a virtual bridge for dedicated servers.

Me I could not get it to work reliably so I added another bridge on my servers which link via wireguard. It works like a lan for both the physical servers and VMs.

Re: LXC and LXD: a different container story

#98

Post by fuzzy2 »

dizhn wrote: post #96
fariszr wrote: post #50

Earlier quoted context omitted.

You are using hetzner right? I'm interested to see how you bypassed the Mac address problem, any blogs, guides ?

What mac address restriction is there? One sets them up on their robot interface so things will actually route. You get the assigned mac and use that in your virt ip or bridge settings. Are we talking about something else?

The Robot interface thing is actually relatively new (some years). Before they introduced that, all your IP addresses (and maybe subnets, too?) were delivered to your physical MAC address, period. Wanted to use a VM? Maybe use proxy_arp or some 1:1 NAT. Bridging? Not possible.

Re: LXC and LXD: a different container story

#99

Post by [deleted] »

discardedrefuse wrote: post #15

I ran LXD for about a year in my home lab. Unfortunately, the easiest way to get it running is by installing Snap; which I didn't do. Instead I ran it on Alpine Edge (one of the few distros that actually has it in their package manager). LXD kept breaking after system updates and I got tired of troubleshooting. I suppose that's just part of the perils of running bleeding edge. When LXD was running, I found that it fe…

[deleted]

Re: LXC and LXD: a different container story

#100

Post by vngzs »

pxc wrote: post #25
discardedrefuse wrote: post #15

I ran LXD for about a year in my home lab. Unfortunately, the easiest way to get it running is by installing Snap; which I didn't do. Instead I ran it on Alpine Edge (one of the few distros that actually has it in their package manager). LXD kept breaking after system updates and I got tired of troubleshooting. I suppose that's just part of the perils of running bleeding edge. When LXD was running, I found that it fe…

> Unfortunately, the easiest way to get it running is by installing Snap; which I didn't do. Instead I ran it on Alpine Edge (one of the few distros that actually has it in their package manager). Enabling LXD is a one-liner on NixOS: virtualisation.lxd.enable = true; LXC and LXD are a somewhat common approach for NixOS users who want to quickly try something out in the environment of a traditional distro. :)

I use LXD like this on NixOS. It's great for Steam, which works best on Arch Linux.
Post reply on HN