Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LXC and LXD: a different container story

lwn.net

Post reply on HN
Print view View original

41–50 of 102 posts

Re: LXC and LXD: a different container story

#41

Post by hobs »

0xbadcafebee wrote: post #36
locusofself wrote: post #31

My last company had hundreds possibly thousands of LXC containers, and we orchestrated everything via saltstack (which is similar to ansible or puppet if you aren't familiar). The justification was that we needed our SaaS to also work on-prem for financial companies and government entities, and thus we could not count on kubernetes or any specific cloud vendor to be available, so we rolled our own orchestration built…

> we needed our SaaS to also work on-prem for financial companies and government entities, and thus we could not count on kubernetes or any specific cloud vendor ....? kubernetes is not a cloud vendor, it runs on any Linux distribution, and it's FOSS.... ?!

And that's why the sentence contains an or - because they are not a cloud provider but you cant guarantee a local corp is gonna run kube.

Re: LXC and LXD: a different container story

#42

Post by UnlockedSecrets »

I love using LXC, Finding documentation and such online regarding things however is an absolute pain in the ass in trying to figure out if the LXC commands they are talking about older depreciated syntax's, LXD which uses 'lxc' for their command line, Or the modern version of LXC which you are trying to make use of....

Re: LXC and LXD: a different container story

#43

Post by tonnydourado »

hobs wrote: post #41
0xbadcafebee wrote: post #36

Earlier quoted context omitted.

> we needed our SaaS to also work on-prem for financial companies and government entities, and thus we could not count on kubernetes or any specific cloud vendor ....? kubernetes is not a cloud vendor, it runs on any Linux distribution, and it's FOSS.... ?!

And that's why the sentence contains an or - because they are not a cloud provider but you cant guarantee a local corp is gonna run kube.

A previous employer had the same issue, and solved it by writing the application to kubernetes and deploying minikube to the clients that didn't have their own kubernetes cluster.

To be fair, the situation ended up being: bunch of clients running minikube, 1 major client with their own kubernetes cluster, and a lot of other clients using the multi tenant, cloud hosted, offering. Made it a bit of a pain to support that one client with a cluster we could not control, but it was one of our biggest clients, so, them the breaks.

Re: LXC and LXD: a different container story

#44

Post by tonnydourado »

I tried to invent docker and kubernetes with LXC back when docker was super alpha and kubernetes was barely a fetus. I failed, obviously, but learned a lot. It's kinda the only reason I managed to pick up kubernetes later on, honestly. Sometimes, not knowing what you're doing is an asset.

Re: LXC and LXD: a different container story

#45

Post by ansible »

chubot wrote: post #38

Do either of them support the concept of “layers” like Docker does? I think that feature combined with overlayfs2 is quite useful, despite my many criticisms of Docker. It is sort of a middle ground between Nix like granularity which requires rewriting upstream, and big LXC blobs created with shell scripts. Although I also think we need some kind of middle ground between docker and nix :)

> Do either of them support the concept of “layers” like Docker does?

Never used LXD, but LXC does not have layers per-se.

I usually run LXC containers on a btrfs filesystem, which easily supports snapshots and sending containers to other container hosts via "btrfs send | ssh otherhost btrfs receive".

If you are treating your servers like pets (and not cattle) LXC is a very convenient means to consolidate servers onto fewer hardware systems.

Re: LXC and LXD: a different container story

#46

Post by nullwarp »

jordemort wrote: post #12

Hi, author of the article, pleased to see it here! I'd really like to hear more from folks about how they're using LXC and/or LXD, and what they think their greatest strengths are compared to Docker or Kubernetes.

We use LXC on physically distributed Proxmox nodes. It let's us easily launch new VMs and migrate around when we need to. Nothing we have is very complicated but it works well. We use SaltStack to tear up special debug containers with a bunch of integrated tools. To update Saltstack just downloads the new container and creates a new container in Proxmox.

I guess you could technically replace what we built with pure Docker but the Proxmox UI is a godsend for our on the ground support engineers who aren't the most technically savvy types.

We started going down the Kubernetes route initially but it quickly turned into an absolute nightmare for us and we gave up quickly and we're all pretty strong with Kubernetes. I guess though when all you need is a few stateful services running the "dynamic scaling" of kubernetes is kind of pointless.

Re: LXC and LXD: a different container story

#47

Post by sreevisakh »

jordemort wrote: post #12

Hi, author of the article, pleased to see it here! I'd really like to hear more from folks about how they're using LXC and/or LXD, and what they think their greatest strengths are compared to Docker or Kubernetes.

I use Kubernetes within an LXD container with Btrfs backing storage. This isn't anything special. But it has two advantages. The first is that you could try out multi-node K8s clusters on a single system. The second is that the containers can be deleted and rebuilt easily without affecting the host system. Both are very useful when you're learning multi-node K8s. I plan to expand my homelab to true multi-node setup. However, I will probably retain K8s inside LXD. It is useful to run certain applications that demand full control of the system - like PiHole or MailInABox. There are probably better ways of hosting them. However, LXD gives me a lot of flexibility to experiment and make mistakes.

Re: LXC and LXD: a different container story

#48

Post by leetnewb »

discardedrefuse wrote: post #19
leetnewb wrote: post #16

Earlier quoted context omitted.

Someone packages it for opensuse. It has been stable for me on tumbleweed for a couple of years.

Thanks for the info. Never used OpenSuse, but next time my curiosity swings back around to LXD I'll check it out!

In the off chance you remember this conversation, the package does not require attr (even though lxd does), so you need to zypper in attr to get lxd running if another dependency didn't pull it.

Re: LXC and LXD: a different container story

#49

Post by MuffinFlavored »

sreevisakh wrote: post #47
jordemort wrote: post #12

Hi, author of the article, pleased to see it here! I'd really like to hear more from folks about how they're using LXC and/or LXD, and what they think their greatest strengths are compared to Docker or Kubernetes.

I use Kubernetes within an LXD container with Btrfs backing storage. This isn't anything special. But it has two advantages. The first is that you could try out multi-node K8s clusters on a single system. The second is that the containers can be deleted and rebuilt easily without affecting the host system. Both are very useful when you're learning multi-node K8s. I plan to expand my homelab to true multi-node setup.…

> The first is that you could try out multi-node K8s clusters on a single system.

Why can't you do this without BTRFS/LXD and just with Docker/OCI?

Re: LXC and LXD: a different container story

#50

Post by fariszr »

fuzzy2 wrote: post #27

I fucking love LXC! It allowed me to “virtualize” several physical servers on a single newer server, keeping all of the original setup (except some networking changes). Combined with ipvlan , I could even work around the MAC address restriction of my server provider, with both IPv4 and IPv6. I also use it to host a Jitsi instance. Jitsi is rather picky about the underlying distribution. All this without the overhead…

You are using hetzner right?

I'm interested to see how you bypassed the Mac address problem, any blogs, guides ?

Post reply on HN