Docker, Kubernetes, Terraform, and AWS crash course series

Oh boy...

Guys and gals, learn the basics.

You don't need Docker, just a version control system (e.g. git).

You don't need overpriced cloud offerings in the first place.

First of all, pick the right language. For me, that's Go, because it has lots of performance for little resource usage. Docker is written in Go. Why would I put a Go proxy in front of my Go program to have nothing but what's essentially a zip file with some startup script and then have to fight the Go proxy, Docker, and its configuration. Learn about systemd. It's the defacto standard init system for most popular Linix distros nowadays. Terraform? Ansible if you must. If you're managing more than 1 host maybe you do, but it's likely you don't. Or pick something like NixOS if you just have to define your machine state. AWS (and GCP, etc)? A, one, dedicated server where you can host all the projects your heart desires for a fixed price with low costs for bandwidth/traffic.

Kubernetes, you do not need it. Go can handle about 500 million visitors a month on 12 year old hardware per 1 server. You will almost certainly not reach these numbers.

So you don't need much to live a stress free and cheap but efficient life. What this article tries to teach you is how to become a tool for public cloud providers.

Running k8s requires at least 9 servers. Managed k8s, like those cloud providers offer are expensive, because even if you think it isn't so, the devil is in the details. I see that guy posting of how easy it is. BS. Nothing is easier than a single host with a front proxy like nginx, which btw k8s also uses, they call it ingres proxy, a simple nginx config, some letsencrypt, your Go backend and the database on the same server. It doesn't get easier.

What this article tries to teach you is stuff you will only ever need if you expect over 5k concurrency (note that's not 5000 users). When do those situations occur? Eurovision Song Contest. Very popular chat app (but even there you'd not use http).

You don't need all this.

And you also don't need microservices. Monoliths are fine. When are microservices useful? When you need to scale particular components of your big program.

Is it worth the trouble? Hell no.

Especially because now you have to deal with a problem so huge with very little benefit to going the event driven microservice route, because that's the only route that matters regarding microservices. And that's the only, ONLY time k8s makes sense. That's also where you can afford to pay those ridiculous prices for cloud traffic and managed k8s. Those are multilateral enterprise levels of business size, ok?

Forget all that crap.

In these comments you'll find people trying to BS you to get hyped for their product, promising "there will be only one". It's all BS. Just learn the basics. It's not hard.

Disclaimer: truly failsafe setups should run on 3 servers (hn runs on 2 and has regular downtime). So if you have a money maker of course do that. But until then even a single server setup with regular backups will do just fine.

For training purposes, Azure has cost-limited subscriptions.

Frustratingly, the other clouds have dragged their feet and largely refused to implement their own equivalents. This comes up here semi-regularly as one of the #1 complaints about the public cloud, and it's always hand-waved away by some Silicon Valley FAANG employee making half a million a year and using their parent company cloud account.

I know of several ways it would be possible for a malicious actor to deliberately bankrupt a solo dev limited to a normal credit card budget.

And as you've said, it's very fiddly and complicated to implement custom "stop" scripts, and this is futile anyway because the billing metrics in all clouds have huge delays on them.

My $0.02 is try to learn using Azure first because there's a true safety net, and then use customer accounts to continue upskilling.

1 It depends. For larger scale gluster or ceph. But quiet a bit more work.

Longhorn you can get up and working quickly.

Though if you are on a cloud provider just use their storage system.

2 K8 doesn't magically solve replication unfortunately.

Though there are helm charts that will automatically set up a replicated setup for you.

I still need to solve backups.

Once again if you are on a cloud provide. Just use their Db offering.

3 K8 doesn't have a default out of the box.

The repo shows you how to setup traefik to handle this.

On cloud providers they have normally integrated it with their lb already.

For me the large advantages are reproducibility and no vendor lock in.

Also give redundancy and quiet a bit of automation once set up.

Auto scaling is always tricky.

Lastly if you have the skills it can be far cheaper to run your own in metal.

If you don't the the time most likely would be better spent actually coding.

Depending where you are in the world and the relevant pay scales.

I'll try to answer, but keep in mind that I'm newbie myself.

> 1. Do i still need gluster/ceph or do i use the longhorn thing ?

Kubernetes does not care about storage implementation. It contains some abstract ways to request a storage (PersistentVolumeClaim). And then some particular Kubernetes installation will fulfil this request with PersistentVolume. So basically it comes down to your Kubernetes provider. It should have some instructions about volume classes that you can use.

If you're installing your Kubernetes on bare metal, you need to think about this aspect yourself, of course. Both ceph and glusterfs are popular options and there're good Kubernetes drivers for it. You can also just use local storage, like docker does, but of course it'll not survive server outage, so it limits your availability.

I'm installing my cluster on OpenStack. There's Cinder CSI plugin for Kubernetes, so it provides me storage when I ask for it. My provider uses SAN for one type of volume and Ceph for other type of volume.

I think that simplest solution is some kind of NFS server. Kubernetes can consume it as well.

> 2. Db replication? Do i use the usual solutions of master-slaves and clusters or does multiple k-nodes take that over ?

Basically Kubernetes does not care about your particular configuration. It runs containers and provides those containers with storage, DNS, network, etc. So it's up to you. You can configure database replication with your own tools and scripts if you like. I have had good experience with CloudNativePG. It's so-called Kubernetes Operator. Basically it's a thing that configures database clusters for (postgres) given abstract definition. It can configure master-slaves cluster and it allows for easy backup configuration to S3 storage. There're other operators as well, for almost any popular database. So probably it's better to use those, unless you're very good at database operations.

And in the true clouds it might be a good idea to use managed database and not to think about it at all.

> 3. Webserver LB with failover ? Do i use LB from hosting vendor, haproxy or does kubernetes have its own thing ??

You need some kind of external load balancer to deliver packets in a high-available way. My hoster provides that as part of its OpenStack package. I guess that every cloud hoster provides it. If you're using bare metal, you need some kind of haproxy and keepalived setup (or some kind of hardware load balancer, I have no idea).

This external load balancer have to deliver TCP packets to your worker nodes. Like 1.2.3.4:80 -> 10.1.1.1:30080, 10.11.1.2:30080, 10.11.1.3:30080. And once Kubernetes receives those packets in a high-available way, it routes them as needed. Usually you have ingress controller which provides HTTPS, and then uses HTTP host and path to route request to some pod which serves it in the end. Once request reached Kubernetes, it'll make sure to route it the right way. If your pods deployed in 2+ replicas, it'll be high-available. If your pod deployed in 1 replica and server is died, Kubernetes eventually will reschedule that pod to another server, but there'll be service interruption for a few minutes. So everything high-available should be deployed in 2+ replicas.

> From what I can tell as a kubernetes-noob the value is: 1) Reproducibility 2) Reliability via redundancy 3) AutoScaling.

Here's my take on Kubernetes value.

First is it introduces a language connecting developers and operations. It is important. You don't need developers to hand-waive which ports they need to expose, which services they need to consume and which HTTP routes they need to receive for that particular server. They've got language to express how their service should be used.

Second is it provides high-available cluster. And in my testing it's quite stable. My only issue is that it takes too much time to reschedule pods from dead server. I expected it to take few seconds, but it took few minutes. I don't yet know why is that.

Third and one I didn't really expect when started to learn it: it provides high-quality solutions for some hard problems. I mentioned database clustering and database backup. I can deploy database with single 50-lines YAML mostly copy&pasted from the example. It'll start master and slave pods and it'll provide continuous backup to S3 using barman. I don't have skills to configure that kind of setup and I expect that I'd need a week at least to come to that setup. Another problem is letsencrypt. Well, it's not that hard, but I've spent many hours debugging some convoluted nginx/caddy/whatever net of docker containers trying to figure out what letsencrypt does not work there. With Kubernetes cert-manager it's just works. All configuration is centralized, all services write ingress and they've got their TLS certificate automagically, whether it's HTTP-01 solver, DNS-01 solver, it's just abstracted away.

It has some steep learning curve, that's for sure. Even more so, if you want to deploy it yourself rather than using managed one. I suggest to use managed one if you can. I have some circumstances which prevent me to using managed Kubernetes, but I plan to migrate to managed as soon as I can. It's not that hard, but it takes time and managed Kubernetes is cheap enough. If you can't use managed Kubernetes, try to find a provider with OpenStack API. It'll help with load balancers and storage provisioning.

Autoscaling - that part I didn't solve yet. It's not easy if you're not using managed Kubernetes. But if you're using managed Kubernetes, it should be as easy as ticking a checkbox somewhere.

> All of the above has to some degree a previous/current solution , so which do i give up/replace with kubernetes-tool ?

Well, right now we're using three dedicated servers with docker-composes scattered all over, made with ad-hoc scripts and whatnot, partially working backups, no observability. Chaos. Kubernetes for us looks like a very promising way to throw away that chaos and rebuild operations correctly.

Author here. I tried to answer your question in the first two paragraphs. But to add some context, given the nature of my work, I hear from developers on a nearly daily basis who are struggling to get started with the technologies mentioned in this blog post series, which include not only Kubernetes, but also Docker, AWS, and Terraform. In part, they are struggling because they are too scared to ask for help, and comments like yours only make that worse: you seem to be implying that the materials out there for Kubernetes are so good, that if you don't get it, there must be something wrong with you. And yet, there are thousands of devs who don't get it, so maybe for different people, there are different ways to learn?

In discussions like this, I'm a fan of what Steve Yegge wrote about blogging [1]:

> This is an important thing to keep in mind when you're blogging. Each person in your audience is on a different clock, and all of them are ahead of you in some ways and behind you in others. The point of blogging is that we all agree to share where we're at, and not poke fun at people who seem to be behind us, because they may know other things that we won't truly understand for years, if ever.

That's why I write: to share what I know, from my particular perspective. Hopefully, that's useful to some people out there. If it's not useful to you, no problem!

And for the record, I agree the Kubernetes docs are great, including those interactive tutorials: if you read the series, you'd see I actually recommend those exact docs at the end of the post [2].

[1] https://sites.google.com/site/steveyegge2/you-should-write-b... [2] https://blog.gruntwork.io/a-crash-course-on-kubernetes-a96c3...