I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
There is the k3s distro of kubernetes
Docker Swarm Cluster Complete Guide
11–20 of 41 posts
Re: Docker Swarm Cluster Complete Guide
#12I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
Kubernetes really isn't as difficult as you might think. Early on it was "hard" because most people were learning containers as well as Kubernetes.
Also how that we have managed services like EKS, GKE, AKS it's really straight forward to learn.
I would suggest start with a GKE cluster and deploy nginx.
Re: Docker Swarm Cluster Complete Guide
#13I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
Nomad + Consul.
Used it for years in prod - just awesome.
I’ve also used k8s extensively and nomad/consul is more ”open” and less opinionated.
If you also manage some form of legacy infrastructure and VMs, consul is hard to ignore.
Two go-binaries - you’re up in minutes for a test drive.
Re: Docker Swarm Cluster Complete Guide
#14Re: Docker Swarm Cluster Complete Guide
#15Docker Swarm is a marvel. It's super easy to use yet does the job very simply.
On the surface. Easy to setup a simple use case but long term has a similar learning curve to Kubernetes for a production deployment.
Re: Docker Swarm Cluster Complete Guide
#16I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
Nomad + Consul. Used it for years in prod - just awesome. I’ve also used k8s extensively and nomad/consul is more ”open” and less opinionated. If you also manage some form of legacy infrastructure and VMs, consul is hard to ignore. Two go-binaries - you’re up in minutes for a test drive.
I looked into Nomad, did not understand much. I am pretty OK with docker and docker-compose, me and my team use docker to develop data workflows in python locally. So out of Docker I am much less comfortable. For instance, would JupyterHub for Kubernetes run on Nomad?
https://zero-to-jupyterhub.readthedocs.io/en/latest/customiz...
Re: Docker Swarm Cluster Complete Guide
#17I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
Kubernetes really isn't as difficult as you might think. Early on it was "hard" because most people were learning containers as well as Kubernetes. Also how that we have managed services like EKS, GKE, AKS it's really straight forward to learn. I would suggest start with a GKE cluster and deploy nginx.
There are many managed services, which is great, but how does this work when your storage is somewhere else? For instance I might decide to put the storage (postgres and minio) on a cheap Hetzner dedicated server, how would this work with EKS? I am especially trying to understand doing analytics and ML with dask.distributed. I think running compute on managed Kubernetes with storage inside another server could (hybrid?) should be ok, but I haven't figured this part yet.
Re: Docker Swarm Cluster Complete Guide
#18Earlier quoted context omitted.
Kubernetes really isn't as difficult as you might think. Early on it was "hard" because most people were learning containers as well as Kubernetes. Also how that we have managed services like EKS, GKE, AKS it's really straight forward to learn. I would suggest start with a GKE cluster and deploy nginx.
There are many managed services, which is great, but how does this work when your storage is somewhere else? For instance I might decide to put the storage (postgres and minio) on a cheap Hetzner dedicated server, how would this work with EKS? I am especially trying to understand doing analytics and ML with dask.distributed. I think running compute on managed Kubernetes with storage inside another server could (hybri…
You can use a persistent volume claim in multiple containers.
I would suggest using a Google Disk or EBS volume (Depending on if you are using GKE or EKS. They can be mounted to multiple containers at a time (your consumers). Portable and durable.
Keep this volume updated with your data. Maybe a sync process from your other server.
Re: Docker Swarm Cluster Complete Guide
#19Earlier quoted context omitted.
Kubernetes really isn't as difficult as you might think. Early on it was "hard" because most people were learning containers as well as Kubernetes. Also how that we have managed services like EKS, GKE, AKS it's really straight forward to learn. I would suggest start with a GKE cluster and deploy nginx.
There are many managed services, which is great, but how does this work when your storage is somewhere else? For instance I might decide to put the storage (postgres and minio) on a cheap Hetzner dedicated server, how would this work with EKS? I am especially trying to understand doing analytics and ML with dask.distributed. I think running compute on managed Kubernetes with storage inside another server could (hybri…
Yeah, hybrid cloud and multi-cluster is still not a first class citizen. Kubernetes wasn't originally meant for this, and while I think there exist some solutions, they are quite early stage. Of course nothing prevents you to put your data somewhere else and load it, you just won't be able to manage Kubernetes resources across clusters. But you wouldn't be able to do this with an approach like Docker Swarm either. It has nothing to do with managed services, it's just that Kubernetes clusters, managed or not, are not meant to span regions.
Re: Docker Swarm Cluster Complete Guide
#20I am looking into this now. Basically it’s either this or the super hard Kubernetes right? I find it hard to believe that we don’t have a middle ground?
I don't understand why people think Kubernetes is "hard" - I find it quite straightforward and the documentation is excellent. I've been running my side projects on a single k8s clusters and rarely have issues.
Yes, there is a learning curve. It's not a Heroku where you enter one command and it just works. You'll need to spend a few days to become familiar with the basics concepts. And it may take weeks grasp the advanced concepts. But it's less complexity than learning something like a new programming language.
You'll probably get back the time you invested. Things like deploying production-ready application with a single command using e.g. Helm is quite powerful and can save you money over using managed services.