Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

microk8s.io

Post reply on HN
Print view View original

81–90 of 138 posts

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#81

Post by nullwarp »

regularfry wrote: post #69
fy20 wrote: post #60

I tried a few incarnations of self-hosted k8s a few years ago, and the biggest problem I had was persistent storage. If you are using a cloud service they will integrate k8s into whatever persistent storage they offer, but if you are self-hosting you are left on your own, it seems most people end up using something like nfs or hostPath - but that ends up being a single point of failure. Have there been any developmen…

I'm using longhorn, but it's been cpu-heavy.

I really liked longhorn but the CPU usage was ultimately too high for our use case.

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#82

Post by utf_8x »

Microk8s is great if you're lazy. I've recently built a small-ish 3 node cluster hosting internal apps for a few hundred users and pretty much the only setup I needed to do was: install it with Snap*, enable a few plugins (storage, traefik, coredns), run the join command on each node and set up a basic loadbalancer using haproxy** and keepalived***.

* I don't like Snap. Like, a lot. But unfortunately there aren't any other options at the moment.

** I have HAProxy load-balancing both the k8s API and the ingresses. Both on L4 so I can terminate TLS on the ingress controller and automatically provision Let's Encrypt certs using cert-manager[1].

*** Keepalived[2] juggles a single floating IP between all the nodes so you can just run HAProxy on the microk8s nodes instead of having dedicated external loadbalancers.

[1] https://cert-manager.io/docs/

[2] https://github.com/acassen/keepalived

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#83

Post by jayrwren »

fy20 wrote: post #60

I tried a few incarnations of self-hosted k8s a few years ago, and the biggest problem I had was persistent storage. If you are using a cloud service they will integrate k8s into whatever persistent storage they offer, but if you are self-hosting you are left on your own, it seems most people end up using something like nfs or hostPath - but that ends up being a single point of failure. Have there been any developmen…

seaweedfs seems pretty great for a cloud storage: http://seaweedfs.github.io

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#84

Post by rsmitty »

(Full disclaimer - I'm an engineer at Talos, but I believe it's pretty relevant here)

If folks are interested in this kind of K8s deployment, they might also be interested at what we're doing at Talos (https://talos.dev). We have full support for all of these same environments (we have a great community of k8s-at-home folks running with Raspberry Pis) and a bunch of tooling to make bare metal easier with Cluster API. You can also do the minikube type of thing by running Talos directly in Docker or QEMU with `talosctl`.

Talos works with an API instead of SSH/Bash, so there's some interesting things about ease of use when operating K8s that are baked in like built-in etcd backup/restore, k8s upgrades, etc.

We're also right in the middle of building out our next release that will have native Wireguard functionality and enable truly hybrid K8s clusters. This should be a big deal for edge deployments and we're super excited about it.

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#85

Post by sethammons »

arcticfox wrote: post #78
hardwaresofton wrote: post #5

Deploying k8s has gotten a lot easier these days -- some alternatives in this space: - https://docs.k0sproject.io ( https://github.com/k0sproject/k0s ) - https://k3s.io ( https://github.com/k3s-io/k3s/ ) k0s is my personal favorite and what I run, the decisions they have made align very well with how I want to run my clusters versus k3s which is similar but slightly different. Of course you also can't go wrong with k…

Is developing locally with one of these k8s implementations a good option? My current workflow is to develop locally with a combination of bare (non-containerized) servers and Docker containers, but all of my deployment is to a hosted k8s cluster. If developing locally with k8s would likely be a better workflow, are any of these options better than the others for that?

We deploy with k8s but few of us develop with it. Nearly our whole department uses docker compose to get out dependencies running and to manage our acceptance tests locally. Some people will leverage our staging k8s cluster via kubectl and others just leverage our build pipeline (buildkite + argo cd) that also takes you to stage as it will also take you into production.

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#86

Post by sethammons »

lvncelot wrote: post #66
fy20 wrote: post #60

I tried a few incarnations of self-hosted k8s a few years ago, and the biggest problem I had was persistent storage. If you are using a cloud service they will integrate k8s into whatever persistent storage they offer, but if you are self-hosting you are left on your own, it seems most people end up using something like nfs or hostPath - but that ends up being a single point of failure. Have there been any developmen…

I've had good experiences using the Rook operator for creating a CephFS cluster. I know that you can run it on k3s, but I don't know whether RaspberryPi nodes are sufficient. Maybe the high RAM Raspi 4 ones.

We do this at Twilio SendGrid

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#87

Post by stonemetal12 »

Jonnax wrote: post #11
kissgyorgy wrote: post #6

If anybody seriously believe that Kubernetes is good on Edge and IoT, our industry is in deep trouble. I usually like Canonical, but this is next level bullshit.

Care to explain why? And what industry you're talking about? Kubernetes has made efforts for nodes to work with poor network connectivity. The node requirements aren't that big. A lot of use cases are relatively easy to containerise. And edge / IoT devices are getting more powerful as well. They aren't talking about consumer IoT here as well. It has become a meme that Kubernetes is complicated. But it solves a lot of…

IDK about OP, but when I think IoT, I think small single purpose devices: a ring door bell, a "smart" thermostat or fire alarm, a security camera. There at most a couple of processes running, what is there to orchestrate on an IoT doorbell?

On the backend where you have services being fed, processing, and presenting all that data sure Kubernetes that part up, but that doesn't seem to need a special distro of Kubernetes.

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#88

Post by phumberdroz »

arcticfox wrote: post #78
hardwaresofton wrote: post #5

Deploying k8s has gotten a lot easier these days -- some alternatives in this space: - https://docs.k0sproject.io ( https://github.com/k0sproject/k0s ) - https://k3s.io ( https://github.com/k3s-io/k3s/ ) k0s is my personal favorite and what I run, the decisions they have made align very well with how I want to run my clusters versus k3s which is similar but slightly different. Of course you also can't go wrong with k…

Is developing locally with one of these k8s implementations a good option? My current workflow is to develop locally with a combination of bare (non-containerized) servers and Docker containers, but all of my deployment is to a hosted k8s cluster. If developing locally with k8s would likely be a better workflow, are any of these options better than the others for that?

The best time I had so far was with dockertest[0] in go it allows you to spin up containers as part of your test suite which then allows you to test against them. So we have one go pkg that has all our containers that we need regularly.

The biggest benefit there is no need to have a docker compose or have other resources running locally you just can run the test cases if you have docker installed. [0] https://github.com/ory/dockertest

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#89

Post by kiney »

traspler wrote: post #50
kiney wrote: post #37

How is persistent storage handled on microk8s (or k3s)?

If you want more than just local disk (directory/path on the node the pods currently reside on) you will have to deploy your own storage provider to handle PVCs (on k3s at least). According to Microk8s docs, it's the same there.

ok. thanks. Do you have any recommendations for lightweight solutions? I ran a ceph cluster a few years ago but remember it being quite a bit of work.

Re: MicroK8s – Low-ops, minimal Kubernetes, for cloud, clusters, Edge and IoT

#90

Post by pas »

tasubotadas wrote: post #71

TBH, I am not that a big fan of microk8s. I have it deployed on VPS and it's far from stable. The server itself, is, probably, overprovision, but I still struggle with responsiveness, logging, and ingress/service management. What's also funny, that using the Ubuntu's ufw service is not that seamless together with microk8s. I am think of moving now to k3s. The only thing that's holding me back is that k3s doesn't use…

You can easily deploy the Nginx ingress controller on k3s.
Post reply on HN