I have built all our infrastructure on Docker Swarm before learning about its state. Currently waging migrating to Nomad out of fear for K8s complexity - I’ve worked with it in a previous job with more employees than now, and it still was a big hassle - but am afraid I’ll repeat the same mistake and should just bite the bullet. Does anyone have suggestions?
- Single-server and non-secure (no mTLS; no ACLs) clusters are super easy to set up and a great way to try out things before committing.
- ...However, enabling especially ACLs but also TLS on running clusters is going to be more hassle than simply setting up a fresh, properly bootstrapped cluster.
- Their minimum and recommended resource requirements are hugely inflated, you can generally get good mileage out of way less
- ...However: Do follow their advice on keeping nodes single-responsibility (ie don't run a consul server and nomad server on the same node; generally keep your servers dedicated and not running jobs)
- Consul and Vault integrations are generally rock solid.
- ...However: Nomad native service discovery is not yet (wrt consistency and template rewrites). Consul Connect may or may not have edge (it's some time since last we seriously tried it).
- learn.hashicorp.com has material for most things you want to do.
- Put all your ACL configuration in Terraform (or whatever else you have for the same purpose)