Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

K3s – Lightweight Kubernetes

k3s.io

Post reply on HN
Print view View original

181–190 of 194 posts

Re: K3s – Lightweight Kubernetes

#181

Post by pas »

Intermernet wrote: post #168
sheeshkebab wrote: post #87

Earlier quoted context omitted.

> great K8S API, that Swarm lacks Hmm... nothing against k8s, but it’s deployment api is an abomination on par with aws cloudformation. You need teams of yaml engineers to manage these things.

Is YAML engineer a real thing? I always considered YAML to be a slightly hacky configuration DSL that I needed to be familiar with, not an actual core, career, competence. For example, can I become an INI engineer?

I think that's the joke :)

Of course k8s's YAML API is just a clunky interface, because it's evolving very rapidly, and it's a rather low-level thing. So basically if you work a lot with k8s, it seems like all you do all day is to copy paste [or generate, or parse] YAMLs.

Re: K3s – Lightweight Kubernetes

#182

Post by pas »

seabrookmx wrote: post #167
GordonS wrote: post #155

Earlier quoted context omitted.

I use Docker Compose on a small, cloud-hosted VM to run all my development dependencies and stuff I need for testing (I've got RabbitMQ, Postgres and Splunk running at present). It was really simple to setup, and "just works" - and the nodes don't have to compete with a greedy K8s orchestrator for resources!

Ditto. For a single machine you don't need "orchestration" - you just need something cleaner than a bash script to start/stop and potentially restart your containers. I set up my home server with minikube more as a learning exercise, but went back to docker-compose for running all the things I actually rely on (Plex for instance) just because it's so much simpler.

Agreed. k8s makes no sense on 1 node. But in my experience Swarm is just not enough when you have more than one node. You have to work around a few missing key features. (Like the management of deployments. With Swarm you have stacks, and you can update them, but you can't easily roll back the update, you can't easily manage/configure the update. And if the update gets stuck, it just gets stuck, it's not managed.)

Re: K3s – Lightweight Kubernetes

#183

Post by pas »

sheeshkebab wrote: post #87
pas wrote: post #47

Earlier quoted context omitted.

Based on the README [0], quickly comparing to Swarm: - this has the same great K8S API, that Swarm lacks. (Deployment is a first class citizen in kube land, but you only have to make do with the service YMLs in the Swarm sphere.) - k3s lacks some in-tree plugins, that swarm might have (mount cloud provider managed block device), but there are out of tree addons - sqlite instead of etcd3 [but available], so out of the…

> great K8S API, that Swarm lacks Hmm... nothing against k8s, but it’s deployment api is an abomination on par with aws cloudformation. You need teams of yaml engineers to manage these things.

The syntax is ugly, the features are amazing. And it's easy to abstract over ugly syntax, but it's hard to work around Swarm's missing features.

Re: K3s – Lightweight Kubernetes

#184

Post by peterwwillis »

sliken wrote: post #176
peterwwillis wrote: post #150

Earlier quoted context omitted.

"Cluster" doesn't really square with "small business". Like, why have 3 nodes? HA? They're all probably plugged into the same power run and switch and breathing the same A/C. They're not actually redundant but they are more complicated. So it makes no sense unless you have strangely huge/complicated computational requirements.

Why not? Small business can lose significant money from a few hours of downtime and support that's onsite within an hour is quite expensive. I'm not thinking cluster for performance, just for HA. For small business why not 3 x Raspberry Pi to enable as much functionality as possible without network and/or power? A cheap UPS would likely run a few Pi for days. Chick-fil-a (3 NUC in a k8s cluster) seems pretty proud of…

It's still a bad idea.

First of all, do you know what k8s is for? Bin packing. Does your small business have a bin packing problem? (And I don't mean crates) Does your small business even need containers at all?

Second, 3 nodes is more than you need. You only need 2 nodes for HA. There is no universe in which two nodes in the closet of a coffee shop would go down, but three would not.

Third, it's too complicated. It takes teams of people loads of time and money to get it working properly, and then they have to keep supporting it, because release cycles and changing standards, etc. Distributed systems are the most complex and the most costly, unless you're at a huge scale, and then it can be cheaper.

Four, it's unnecessarily expensive, because again, you don't need 3, and it's too complicated.

Five, you don't need 3 nodes to have redundant network paths. A DSL line and a cell modem are pretty easy to plug into one machine.

But six, the real reason this wouldn't work: small businesses do not buy HA.

> Question is can the right combination of hardware standardization, software standardization, support, and application store get together to enable chick-fil-a like functionality at a price point acceptable to small businesses?

Yes, it's called Windows on a Dell.

Re: K3s – Lightweight Kubernetes

#185

Post by antmanler »

xur17 wrote: post #172
farisjarrah wrote: post #169

Earlier quoted context omitted.

Have you seen or used Microk8s ? Seems like it might be what you are looking for. Can be installed and removed via snap package too

Microk8s looks interesting as well, but it doesn't support osx as far as I can tell.

multipass + microk8s on macos https://github.com/CanonicalLtd/multipass

Re: K3s – Lightweight Kubernetes

#186

Post by sandGorgon »

pas wrote: post #182
seabrookmx wrote: post #167

Earlier quoted context omitted.

Ditto. For a single machine you don't need "orchestration" - you just need something cleaner than a bash script to start/stop and potentially restart your containers. I set up my home server with minikube more as a learning exercise, but went back to docker-compose for running all the things I actually rely on (Plex for instance) just because it's so much simpler.

Agreed. k8s makes no sense on 1 node. But in my experience Swarm is just not enough when you have more than one node. You have to work around a few missing key features. (Like the management of deployments. With Swarm you have stacks, and you can update them, but you can't easily roll back the update, you can't easily manage/configure the update. And if the update gets stuck, it just gets stuck, it's not managed.)

The stuck issue is not a limitation of Swarm, but was a bug that has gotten fixed in 18.06 - https://github.com/moby/moby/issues/37493

Rollback has existed for a while now - https://docs.docker.com/engine/reference/commandline/service... . you can actually tune the parallelism and the delay

Re: K3s – Lightweight Kubernetes

#187

Post by antranigv »

biggestlou wrote: post #157
antranigv wrote: post #149

I mean, I've built a Kubernetes-y thing for my own needs, mostly "scheduling" LXC and FreeBSD Jails, and it needs hella less than 256MB of RAM and it's around 20MBs. Maybe I should clean the code and publish it.

That's literally not Kubernetes, though. Does it support custom resource definitions? Services? Deployments? ConfigMaps? Multiple containers on a single host, i.e. Pods? Your project does sound very cool but I don't think provides a fair basis of comparison re: size and RAM usage.

very good question! source definition -> no. services -> yes. deployments -> yes. configmaps -> no. multiple containers on single host -> yes. I should add the most common features I think. I also haven't integrated it with any reverse-proxy service yet, but will try that too.

Re: K3s – Lightweight Kubernetes

#188

Post by detaro »

omeid2 wrote: post #49
detaro wrote: post #4

Looks quite interesting for small setups, curious to read more about the limitations (E.g. " Added sqlite3 as the default storage mechanism. etcd3 is still available, but not the default. " - what availability promises can this make?)

In terms of etcd3 vs sqlite3, it is as reliable as most airplane systems that depend on it. https://www.sqlite.org/famous.html I think the "high availability by redundancy" story is oversold.

Not worried about reliability of sqlite, but the system running it, and what happens if it goes down. E.g. if it relies on a single node, but the cluster just can't make changes anymore but continues to run just fine and cleanly recovers once the main DB is back, that's probably a tradeoff that works often. If stuff starts breaking quickly, not so much.

Re: K3s – Lightweight Kubernetes

#189

Post by otabdeveloper1 »

12298765 wrote: post #120
otabdeveloper1 wrote: post #94

Earlier quoted context omitted.

You missed the point. A day will come when CPU speeds and RAM sizes will stop increasing. When that day comes you don't be able to ignore performance, because a lack of performance will mean actual, real-world dollar costs for the client, and so performance will be the #1 requirement of the client.

CPU speeds haven't changed much in 10 years, and to be honest, I would argue that performance has been getting worse and is not a priority for many/most developers. The client doesn't seem to care, and just sees the 5-20 second webpage load times as totally normal (at my company/industry at least). Our applications are horrifically slow, but we're still the market leaders in our segment

> The client doesn't seem to care, and just sees the 5-20 second webpage load times as totally normal

Yes, but will they tolerate 2 minute page load times? 10 minute?

At some point reality kicks in and you start to have to care about the nuts-and-bolts hardware.

Re: K3s – Lightweight Kubernetes

#190

Post by nvarsj »

meddlepal wrote: post #170
nvarsj wrote: post #131

Earlier quoted context omitted.

> Practically speaking: I worry this will become a parallel but subtlety different implementation of Kubernetes with it's own quirks. Hello, we're already there. Despite the CNCF certification process, there are many flavours of k8s, all with their own quirks. Being on the product side, it is quite painful to support.

I am aware of this problem (getambassador.io dev as well as infra engineer). The differences usually manifest at the edges rather than the internals right now. This has the potential to be even more disruptive problem. I agree the CNCF certification process isn't comprehensive enough.

Well, even the internals have issues.

Take Openshift as the most extreme example - it's a fork of K8s, with a lot of custom functionality like scc and routes, and also a lot of custom behavior like its admission controllers that manage pod security contexts.

It's almost like a different version of k8s - things that will run on GKE or vanilla k8s won't work on Openshift. Yet, it is CNCF compliant.

Post reply on HN