Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Nomad for a Home Server

mrkaran.dev

Post reply on HN
Print view View original

81–90 of 153 posts

Re: Running Nomad for a Home Server

#81

Post by mahesh_rm »

marvinblum wrote: post #21

I did/do run both myself, Kubernetes and Nomad, and it was a million times easier to set up Nomad (including Consul) on bare metal than it was to set up Kubernetes. Kubernetes offers more features, but you most likely don't need them and the increase in complexity makes it a pain to maintain. I'm running a three-node cluster on Hetzner [0] for Pirsch [1] right now and haven't had any difficulties whatsoever when upgr…

>> I plan to write an article/kind-of-tutorial about the setup.

Please post it here when you do. :)

Re: Running Nomad for a Home Server

#82

Post by mr-karan »

aequitas wrote: post #6

Coincidentally my home server is also named Hydra, running Nomad on CoreOS/Flatcar. I've had this setup for years now running without issue (unless I managed to mess it up myself by, for example, changing the IP and not following proper procedures to restore the Raft cluster). Recently I deployed K3s on the same node to add some new workloads, but now that I want to move those workloads to Nomad to get rid of the cpu…

> Coincidentally my home server is also named Hydra, running Nomad on CoreOS/Flatcar

Author here. Ha, nice coincidence indeed :)

> But I'm missing a community repository of sorts for Nomad.

Indeed. I think it's time to build one! Indeed

Re: Running Nomad for a Home Server

#83

Post by rektide »

qbasic_forever wrote: post #77
rektide wrote: post #67

A whole lot of anti- Kubernetes "you most likely don't need them and the increase in complexity makes it a pain to maintain" and "Kubernetes is exceedingly complex" in this thread & somewhat in this article. I agree that you probably don't need Kubernetes, and perhaps yeah it could be considered complex. But I think it's the right fit for most developers/doers & over time most operators too. Kubernetes is not Kuberne…

Yeah if there were a one-click install _complete_ local distribution of Kubernetes with a GUI, and that just read and migrated all your docker-compose.yaml files I think it would see much less complaints and teeth-gnashing from new folks. The existing tools like minikube, kind, microk8s, are still too clunky and don't include everything you'd want for a good local setup (login/authentication, ingress, registry, easy…

I agree pretty roundly here.

Some what-color-do-we-paint-the-bikeshed comments on your particular tools:

* flux seems to be doing great. the ondr0p home-cloud repo i linked is built around it.

* tekton looked very promising when i was evaluating event-driven-architecture systems ~18 months ago, but since then, they've re-branded as a CI/CD tool. it's just branding, it's still generally useful, but i very much worry about drift, & using a product against-the-grain from how the community around it uses it. i think there is a really epically sad story here, that this is a huge mistake for Tekton, which is much more promising/useful than "CI/CD" alone allows. talked about this some two weeks ago[1].

* knative was on my todo list. it's resource requirements are fairly daunting. i'm trying to pack a lot of work on to my 3GB k3s VPS and knative seems right out. it's weird to me that requirements are so high. serving seems a bit on the complex side but useful abstractions, they make sense. eventing is very high in my interests, and i would prefer having an abstraction layer over my provider, give myself freedom, but again the cost seems very high.

* need to try some skaffold. i don't know where it would fit in my world yet, kind of forget it some.

* k8s, tekton, knative, skaffold are all somewhat from the googleverse. honestly i'm hoping we see some competing takes for some of these ideas, see different ideas & strategies & implementations. kubernetes is such great material for innovation, for better ways of composing systems. let's try some stuff out! please kindly think of those who don't have a lot of memory too.

[1] https://news.ycombinator.com/item?id=25993294

Re: Running Nomad for a Home Server

#84

Post by gen220 »

parhamn wrote: post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

Even better, a standard API that you can create dynamic config against. :)

My best experience writing config files has been with Aurora config files for mesos deployments in pystachio-Python.

Once you’re used to using a real programming language to write/generate config, you never want to go back to YML.

Re: Running Nomad for a Home Server

#85

Post by GordonS »

marvinblum wrote: post #21

I did/do run both myself, Kubernetes and Nomad, and it was a million times easier to set up Nomad (including Consul) on bare metal than it was to set up Kubernetes. Kubernetes offers more features, but you most likely don't need them and the increase in complexity makes it a pain to maintain. I'm running a three-node cluster on Hetzner [0] for Pirsch [1] right now and haven't had any difficulties whatsoever when upgr…

Are you using Nomad to schedule containers on those nodes? I'd also be really interested in a blog post or write-up about your setup!

Re: Running Nomad for a Home Server

#86

Post by mr-karan »

outime wrote: post #3

The biggest problem with these posts is that authors seem to fail to see how K8s isn't targeting home servers or small companies. Sure, there's K3s which allows you to have a more minimalistic experience but it's still K8s. K8s is enterprise stuff and should be seen as such. It's complex because the problems that are being attempted to solve are complex and varied. That doesn't discount that Nomad is a great piece of…

> authors seem to fail to see how K8s isn't targeting home servers or small companies

Well, I do run K8s in prod at my org. And my comparison was based off my experience with it running in prod.

> It's complex because the problems that are being attempted to solve are complex

Most people just want to deploy their workloads in a consistent, repeatable manner. That's why they look to run an orchestrator. K8s is the most popular choice but it's time we look at other tools which can also help us reach that goal with less headache.

Re: Running Nomad for a Home Server

#87

Post by marvinblum »

GordonS wrote: post #85
marvinblum wrote: post #21

I did/do run both myself, Kubernetes and Nomad, and it was a million times easier to set up Nomad (including Consul) on bare metal than it was to set up Kubernetes. Kubernetes offers more features, but you most likely don't need them and the increase in complexity makes it a pain to maintain. I'm running a three-node cluster on Hetzner [0] for Pirsch [1] right now and haven't had any difficulties whatsoever when upgr…

Are you using Nomad to schedule containers on those nodes? I'd also be really interested in a blog post or write-up about your setup!

Sure, that's what it is for :) We also run cron jobs, Postgres, and Traefik.

Re: Running Nomad for a Home Server

#88

Post by GordonS »

ForHackernews wrote: post #71
linux2647 wrote: post #45

Earlier quoted context omitted.

I’ve often thought that when developers saw Kubernetes, what they actually wanted was some form of PaaS

Are there any lightweight but production-ready PaaS offerings out there? I know there's Dokku, https://flynn.io/ looked super promising but I think it's basically dead now, same for Deis that is dead and forked to https://web.teamhephy.com/ .

Others I've came across on HN are Caprover, Flynn and Apollo.

Re: Running Nomad for a Home Server

#89

Post by dilyevsky »

throwaway894345 wrote: post #75
ForHackernews wrote: post #70

Earlier quoted context omitted.

I'd be really interested, too. Have you looked at k3s at all? We're considering trying to run https://github.com/rancher/k3os on bare metal servers.

Yeah, that's what I used. It comes with some providers out of the box, but they strike me as toys. For example, it gives you support for node-local volumes, but I don't really want to have to rely on my pods being scheduled on specific nodes (the nodes with the data). Even if you're okay with this, you still have to solve for data redundancy and/or backup yourself. The Rancher folks have a solution for this in the fo…

There’s metallb that lets you announce bgp to upstream routers. Another solution would be to just announce it via daemonset on every node and setup a nodeport. Or just add every frontend node IP into DNS. Obv all highly non-standard as it depends on your specific setup

Re: Running Nomad for a Home Server

#90

Post by viraptor »

ForHackernews wrote: post #71
linux2647 wrote: post #45

Earlier quoted context omitted.

I’ve often thought that when developers saw Kubernetes, what they actually wanted was some form of PaaS

Are there any lightweight but production-ready PaaS offerings out there? I know there's Dokku, https://flynn.io/ looked super promising but I think it's basically dead now, same for Deis that is dead and forked to https://web.teamhephy.com/ .

There seems to be a significant lack of open source PaaS as you mentioned. And Dokku is unfortunately a very leaky bucket of abstractions - fun if you're learning docker, annoying otherwise.

This kind of makes sense given how much work it is and how easy it is to monetize as a paid service.

Post reply on HN