Live data from Hacker News

Docker, Kubernetes, Terraform, and AWS crash course series

blog.gruntwork.io

71–80 of 84 posts

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#71

Kubernetes must probably be one of the most tutorialized pieces of SW out there. You want a "crash course" on the platform of your choice? You got it. You want it for the price of your choice? You got it. The docs are great [1] and offer even interactive parts [2]. There are a ton of good materials on youtube. There are Coursera and EdX courses, e.g. [3] from Google and [4] from Red Hat. There are a ton of Udemy cour…

To be fair. Just last night i was googling kubernetes and k3s tutorials. All of them on page 1 of SERPS are super simple and brilliant and ONLY show you how to add nodes to clusters. Ok now what ?? Ive drank the k-Koolaid signup for the newsletters but how do i get value from this 'thing' ??? When i code a webapp how do setup/config the db ? How do I do a regular LAMP app ? Lol i never thought i say this but i need a…

(disclaimer: I work for a Bunnyshell) Honestly the most benefit would be gained from making Kubernetes invisible and speeding up your development process. Tools exist now to deploy short lived preview environments into your cluster for every PR. This is where the Kubernetes values sits. The whole “shift left” idea. Test before merge / identical short lived environments, etc.

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#72
post #8

I think that for the average developer, these tools will soon be obsolete. For frontend applications, hosting services such as Vercel and Netlify gives a far superior DX. For backend, Vercel does the same, or other BaaS' like Supabase that gives you most you would need to create a fully fledged app with easy to use API's. The only downside I can tell, is the pricing, but compared to dev hours saved, it's probably a n…

The difficult with backend is dealing with state. It's easy enough to provide a simple experience when dealing with stateless frontends, backends are a very different story. You're right that we need better tools. I'm the founder of Encore [1] which is all about bringing the simplicity of Vercel/Netlify to backend development. Not by substituting the backend for a BaaS but by building a developer experience hand-craf…

can it be used without creating account? also does your open source version has telemetry enabled?

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#73
post #32

Oh boy... Guys and gals, learn the basics. You don't need Docker, just a version control system (e.g. git). You don't need overpriced cloud offerings in the first place. First of all, pick the right language. For me, that's Go, because it has lots of performance for little resource usage. Docker is written in Go. Why would I put a Go proxy in front of my Go program to have nothing but what's essentially a zip file wi…

"You don't need X because I don't need X and everyone else must have exactly the same requirements and constraints as me" is one of my least favorite takes.

> You don't need Docker, just a version control system (e.g. git).

I've worked where we had to run a lot of systems that we didn't write ourselves and they came in a variety of languages, frameworks, etc. Some Python, some Go, some Ruby, some PHP, some Java, some Fortran, some C++, etc. We did that before Docker and when Docker came out it was a massive simplification for us to be able to just run standard containers where we know there's an entrypoint, some ports, environment variables, etc. rather than have to figure out how to configure all of those things for every kind of environment that we might run into. We could use the same basic build/deploy pipeline for all of them, run them on the same shared servers and know that they weren't conflicting with each other and updates to one wouldn't blow up the .so files that a different one used, etc. Docker doesn't really compete with a version control system like git; if anything, it's closer to an application package format like .deb or .rpm, but with a bit of a different focus.

> You don't need overpriced cloud offerings in the first place.

If you've done the cost analysis and the cloud offerings are cheaper than running your own datacenter, then they're not really "overpriced" and maybe you do need them. If you're just thinking "rent a server from Hetzner or a colo or run a workstation under your desk" and that those options are always cheaper than the cloud then you've clearly never worked somewhere that needed regulatory compliance (HIPAA, PCI, SOC2, etc).

> First of all, pick the right language.

If you're on a greenfield project and have that luxury, great. A lot of developers have to work with existing codebases and products though.

> Kubernetes, you do not need it. Go can handle about 500 million visitors a month on 12 year old hardware per 1 server. You will almost certainly not reach these numbers.

It must be wonderful to live in a world where all requests scale the same. What if each visitor is using your service to transcode 2GB video files? Or running massive climate modeling simulations? Or training an ML algorithm on a few TB of data? 500 million of those per month on one 12 year old server? Awesome. Sign me up.

I've done all of these things with and without tools like Docker and Kubernetes. Yes, there are workloads and environments where they'd be overkill and add more complexity than they're worth. But there's also very good reasons beyond hype that those tools became well known and widely used.

Or, you can just keep assuming that all those developers out there are just idiots compared to you. I doubt you'll change your mind, so I'll just hope I never have to work with you.

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#74
post #32

Oh boy... Guys and gals, learn the basics. You don't need Docker, just a version control system (e.g. git). You don't need overpriced cloud offerings in the first place. First of all, pick the right language. For me, that's Go, because it has lots of performance for little resource usage. Docker is written in Go. Why would I put a Go proxy in front of my Go program to have nothing but what's essentially a zip file wi…

Counterpoint: there is a niche in the industry where going with containers would be easier than following the best practices without, given how likely people are to slip into doing things the lazy way, or due to how you might have limited control over the applications that you might have to run. Containers at least limit the fallout from various mistakes and nudges people towards certain architectural choices - e.g. 12 Factor Apps, so you don't cripple your app into being a "singleton app" which can only have 1 instance running, instead of just a "monolith".

It doesn't matter whether you're running Podman, Docker, Docker Compose, Docker Swarm, Hashicorp Nomad, K3s/K0s/minikube or a full K8s distro like RKE, the OCI standard is a hill that I'll die on because unless you're the person who's shipping and managing their code 100% of the time, someone else will have to figure out how to setup and run your Java/.NET/Node/Python/PHP app and moreso, what to do when one needs JDK 8, another needs JDK 11 and there are inevitable conflicts in some other system/runtime dependencies. With containers, you can limit the the impact of relatively badly written and sometimes even somewhat insecure applications, by not exposing the rest of the underlying host to them, especially when running rootless or with host remapping.

Sure, in practice it would be exceedingly nice to just deploy a single packaged executable to the server. It would be really nice to even have the choice of picking a technology that plays nicely with such simple setups, like Go, as opposed to having Java shoved down your throat, especially in combination with a separately managed Tomcat instance or something. It would also be nice to have well written applications that don't misbehave and don't trash the CPU/RAM and don't cause the server to have to OOM kill processes, and also that have proper port configuration/remapping available and also applications that have proper health checks built in, or even just well written systemd services and Ansible to manage the configuration.

But in the real world you don't always get that. So unless you like moving around different companies a lot, you need ways to manage the fallout of sub-optimal circumstances (especially legacy code that still needs to run) and containers are one way to achieve that. I wrote down some of my experiences with the subject (albeit with details changed) in my blog post "My journey from ad hoc chaos to order (a tale of legacy code, services and containers)": https://blog.kronis.dev/articles/my-journey-from-ad-hoc-chao...

That said, developing applications poorly will also mean running them poorly on Kubernetes, or not being able to set the environments up at all. Thus, I believe that it's a slippery slope with options on all ends of it being bad, be it running things directly on the server and running into the aforementioned issues, or trying to go full steam ahead with Kubernetes and failing.

The sweet spot in my experience is using containers, but in a simple setup - currently that is Docker Compose or Docker Swarm, but might become Nomad in the future. Maybe K3s in 5-10 years (single node cluster worked fine, except for RPM distros where cgroups v2 support had problems and the Traefik proxy had bad documentation for wildcard SSL/TLS certificates). Then you no longer need to even care that much about the specifics of the technologies inside of the containers either.

Edit: As an example, here's all that I (or anyone else) would need to deploy my homepage, as long as the server can run OCI images or Compose/Swarm stacks:

  version: '3.7'
  services:
    homepage:
      image: REFERENCE_TO_MY_CONTAINER_IMAGE
      restart: "unless-stopped"
      environment:
        # All of the configuration goes here, for example:
        ANALYTICS_URL: LINK_TO_MY_MATOMO_INSTANCE_GOES_HERE
        ANALYTICS_ID: SOME_IDENTIFIER_SET_UP_IN_MATOMO
      volumes:
        - SOME_DIRECTORY/homepage/app/storage:/app/storage
        - SOME_DIRECTORY/homepage/app/db/files:/app/db/files
      networks:
        - prod_ingress_network
      deploy:
        mode: replicated
        replicas: 2
        update_config:
          parallelism: 1
          delay: 10s
          monitor: 10s
        placement:
          constraints:
            - node.hostname == SERVER_HOSTNAME_GOES_HERE
        resources:
          limits:
            cpus: "0.5"
            memory: "512M"

  networks:
    prod_ingress_network:
      driver: overlay
      attachable: true
      external: true
And I no longer have to worry about systemd slices, configuration files, file permissions (for the most part), Ruby or Rails versions, or install any other stuff on the server itself directly. I can just run the image and if it doesn't work, I can just let the developers know what to fix (though in this case I am the developer myself). And if it goes rogue and eats too much resources? The runtime will cap the container without it affecting the rest of the system outside of resource limits that I've set.

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#75
post #53
post #48

Earlier quoted context omitted.

Having deployed Dockerized services to ECS and Azure App Service, doing it from scratch twice is still easier than figuring out k8s.

If you don't know k8s, sure. But if you have someone who does, EKS makes more sense.

Sure, but -- if you're reading tutorials to learn yourself enough k8s to do damage, as provided in the link we're commenting on -- you don't know k8s. Parent is a reasonable question for this thread.

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#76

I think that for the average developer, these tools will soon be obsolete. For frontend applications, hosting services such as Vercel and Netlify gives a far superior DX. For backend, Vercel does the same, or other BaaS' like Supabase that gives you most you would need to create a fully fledged app with easy to use API's. The only downside I can tell, is the pricing, but compared to dev hours saved, it's probably a n…

Depends on what u consider avg developer, aren’t most developers employed by some company or enterprise? Who would definitely lock into big3?

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#77

Earlier quoted context omitted.

How does ClusterAPI compare to Helm charts (or just managing Kubernetes YAML manually or through Kustomize?) Why do you need clusterctl / clusterawsadm? Can't you just kubectl apply once you have your .kube/config set up for your AWS k8s cluster (or Azure or GCP or whatever)?

Cluster API is just CRDs to manage cluster infrastructure itself. Clusterctl and co is to manage controllers that reconcile said CRDs

> CRDs

CustomResourceDefinition

for anybody else following along

https://kubernetes.io/docs/concepts/extend-kubernetes/api-ex...

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#78

Earlier quoted context omitted.

How does ClusterAPI compare to Helm charts (or just managing Kubernetes YAML manually or through Kustomize?) Why do you need clusterctl / clusterawsadm? Can't you just kubectl apply once you have your .kube/config set up for your AWS k8s cluster (or Azure or GCP or whatever)?

Cluster API allows you to create clusters with helm charts. We use ClusterAPI + a centralized ArgoCD to push our clusters that’s other teams can use but with all the required add-ons that the business says we need like security tooling and policy management.

Can you elaborate a little please? Helm charts don't let you create a cluster? Wouldn't it be easier for Helm to add that functionality as opposed to introducing "yet another" tool/layer in the k8s stack?

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#79

Earlier quoted context omitted.

Cool thanks will have a look. Id be more interesting on where kubernetes (redundancy) fits into traditional redundancy ? Examples: 1. Do i still need gluster/ceph or do i use the longhorn thing ? 2. Db replication? Do i use the usual solutions of master-slaves and clusters or does multiple k-nodes take that over ? 3. Webserver LB with failover ? Do i use LB from hosting vendor, haproxy or does kubernetes have its own…

I'll try to answer, but keep in mind that I'm newbie myself. > 1. Do i still need gluster/ceph or do i use the longhorn thing ? Kubernetes does not care about storage implementation. It contains some abstract ways to request a storage (PersistentVolumeClaim). And then some particular Kubernetes installation will fulfil this request with PersistentVolume. So basically it comes down to your Kubernetes provider. It shou…

> I think that simplest solution is some kind of NFS server. Kubernetes can consume it as well.

Isn't this asking for trouble, running MySQL on top of NFS?

Also a k-nube.

Re: Docker, Kubernetes, Terraform, and AWS crash course series

#80
post #32

Oh boy... Guys and gals, learn the basics. You don't need Docker, just a version control system (e.g. git). You don't need overpriced cloud offerings in the first place. First of all, pick the right language. For me, that's Go, because it has lots of performance for little resource usage. Docker is written in Go. Why would I put a Go proxy in front of my Go program to have nothing but what's essentially a zip file wi…

Oh boy, another HN engineer who cannot imagine a technical situation other than the ones they're already familiar with telling everyone else what they don't need.

Yes, learn the basics. That way you'll know that Docker and `git` solve for almost entirely different domains, that Terraform and Ansible solve for almost entirely different domains, that "AWS" is a huge suite of services that makes no sense to compare with a single server, and that some of us do indeed work in distributed environments serving 10s or 100s (or more!) of thousands of queries per second.

Post reply on HN