Live data from Hacker News

Running Nomad for a Home Server

mrkaran.dev

11–20 of 153 posts

Re: Running Nomad for a Home Server

#11
post #7

Earlier quoted context omitted.

Nomad's support for CNI and CSI is improving with just about every release, and they just proved you can run over 2 million containers in a globally distributed cluster. https://www.hashicorp.com/blog/hashicorp-nomad-meets-the-2-m... So what's an example of "enterprise" and "complex" problems you think it can't do?

K8s targets a long list of things that Nomad just doesn't want to deal with in the name of simplicity - and that's ok, if that suits your use case. Some are mentioned in the post but I can think of secret management, load balancing, config management, routing, orchestration beyond workloads (e.g. storage), rollbacks/rollouts and many more. Perhaps in few areas there are some support but it's not what Nomad intends to…

> Some are mentioned in the post but I can think of secret management

Hashicorp Vault seamless integration.

> In order to supply those needs in Nomad you'll need to spend time finding out how to integrate different solutions, keep them up-to-date, etc

Like how k8s secrets are secure out of the box ?

Re: Running Nomad for a Home Server

#12
post #8
post #3

The biggest problem with these posts is that authors seem to fail to see how K8s isn't targeting home servers or small companies. Sure, there's K3s which allows you to have a more minimalistic experience but it's still K8s. K8s is enterprise stuff and should be seen as such. It's complex because the problems that are being attempted to solve are complex and varied. That doesn't discount that Nomad is a great piece of…

> The biggest problem with these posts is that authors seem to fail to see how K8s isn't targeting home servers or small companies. Who would you say are in the target audience of Kubernetes? I doubt most medium to large companies I see implementing Kubernetes could be considered a good fit for Kubernetes. If you want to run on-prem / colo you are probably better of with something simpler like Nomad. If you want Kube…

>Who would you say are in the target audience of Kubernetes?

Everything else - I literally said "K8s is enterprise stuff". Now if we go to specifics then it depends of what the company does, maybe they'd do just fine with Nomad or a managed solution like ECS.

>If you want Kubernetes it's probably a better idea to use a hosted Kubernetes solution like Google's offering.

Well I agree? All of the big companies I've been in used EKS and before EKS was decent there was some maintenance overhead. It'd still be less than the maintenance overhead to maintain the full list of features that K8s provides with Nomad, as Nomad doesn't provide any of those and you'd need to seek solutions outside of the product and try to fit them in.

The same way you'd not buy a car if you're going to drive yourself quarter of a mile once a week, you'd not use such a complex solution to run a few dozen containers.

Re: Running Nomad for a Home Server

#13
post #7

Earlier quoted context omitted.

K8s targets a long list of things that Nomad just doesn't want to deal with in the name of simplicity - and that's ok, if that suits your use case. Some are mentioned in the post but I can think of secret management, load balancing, config management, routing, orchestration beyond workloads (e.g. storage), rollbacks/rollouts and many more. Perhaps in few areas there are some support but it's not what Nomad intends to…

> Some are mentioned in the post but I can think of secret management Hashicorp Vault seamless integration. > In order to supply those needs in Nomad you'll need to spend time finding out how to integrate different solutions, keep them up-to-date, etc Like how k8s secrets are secure out of the box ?

I can understand how they seamlessly integrate with their own products (and I like Vault a lot) - not sure how it'd work with other secret backends if you'd prefer them over Vault. It's also fine you picked an individual item of a long list to rebut what I said. But this is what I said:

>Perhaps in few areas there are some support but it's not what Nomad intends to do anyway

Look, I think we can agree that K8s have many features that Nomad don't and that's just how it is (not good or bad, just different). This comes with added complexity. If you wanted to have all these features in Nomad it'd cost you a lot to the point of being impractical. "I don't need all of those" then don't use K8s - but then don't complain when you bring a vast machine that does 1000 things to do 100 and complain it's too complex.

Re: Running Nomad for a Home Server

#14
post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

I just use JSON for all hashi config, you don't get comments but it is less mental overhead for me to keep track of.

Re: Running Nomad for a Home Server

#15
> Increase developer productivity by making it easier to deploy/onboard new services.

> Consistent experience of deployment by testing the deployments locally.

> (Not joking) You are tired of running Helm charts or writing large YAML manifests. The config syntax for Nomad jobs is human friendly and easy to grasp.

Seems like most of the ux issues stem from configuring kube easily. I wonder what the author would say about using a metaconfig language like cue or jsonnet to make is super easy to define new workloads.

Re: Running Nomad for a Home Server

#16
post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

I just use JSON for all hashi config, you don't get comments but it is less mental overhead for me to keep track of.

Do their tools not support json5? That seems like an omission to me, if so.

Re: Running Nomad for a Home Server

#17
post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

> Wish we’d all agree to one templated configuration format.

If you mean a standard configuration file format, that has almost never happened in the entire history of computing. There are standard data formats, sure, but to standardize a configuration file, all the applications need to be limited to the functionality expressed in a single config file format. Most applications hate that, because they want to add infinite features, and none want to wait to standardize some new version of a config file in order to release their features. (If they release before the config file changes happen, now you have a vendor-specific extension, which means waving goodbye to standardization)

The following are the only configuration file formats I am aware of that have been standardized:

  - bind zone configuration
  - inittab
Crontab isn't standard because different crons have different features. passwd, group, shadow, etc aren't standard configurations because they are actually databases. And anything else which goes over the wire instead of a file is a protocol, not a configuration.

Now, it's still not a bad idea to have standardization. The trick is to make it so they can change their configuration, but still be compatible with each other, and the general way to do that is to abstract the configuration elements via schemas. That way you can have the same basic functionality for each app, but they can define it however they want so that it can be re-mapped or interpreted by a different program.

However, that is in no way simple for a regular user to use. So to convince app developers to have a "standard configuration format", you need to reframe the value proposition. What's the value add? If you say it's just to a user can use one config file on many competing products, the developers won't care about what, because they want you to use their product, not be able to move to another one. If instead you reframe it as "extensible language for integrating and composing multiple independent pieces of software/functionality into one distributed system", then their ears might perk a bit. Basically, reformulate the proposed solution as a language to compose programs, like Unix pipes, but slightly more verbose/abstracted. The end result should be able to be read by any program supporting the config format, and "declaratively" (modern programmers love this ridiculous cargo cult) compose and execute distributed jobs across any system.

Re: Running Nomad for a Home Server

#18
post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

I find it particularly hard to manage Vault from the CLI. I deployed it some time ago and set up a bunch of backends, I don't recall the names of things or how I configured them and I want to backup that config to replicate it.

The auto complete will give you the top level command and that's it.

I haven't looked too hard because I got fed up, but i just want to dump the entire config so I can replicate it without having to remember each thing I configured.

Re: Running Nomad for a Home Server

#19

Nomad looks useful to me for use cases that have nothing to do with "a simple k8s" but more with distributed/HA systemd and cron. Just deployed a small 3 node cluster in prod last week for this: run some binaries with some parameters, make sure they restart on failure, have them move to another node if one fails or is rebooted, and don't waste resources by having a classical active/passive 2 node setup that doesn't s…

How easy is setting up nomad for this? Does this need a consul cluster too?

Re: Running Nomad for a Home Server

#20
post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

I myself would love to see more usage of HCL... after writing a lot of Terraform configurations, it feels so much nicer for me than any JSON or YAML/Helm configuration I have written to this day. We should agree on some kind of HCL-based industry standard and leave all these workarounds via JSON/YAML behind us... e.g. doing Helm logic with these Go templates just looks like a step backwards after writing Terraform HCL code for the last 2-3 years.

I understand JSON as a simple interchange format between systems, and is here to stay, but I don't understand all this YAML stuff, with all its quirks, from the K8s/DevOps people, when we have the much nicer HCL...

For anyone not used to HCL: https://github.com/hashicorp/hcl

Post reply on HN