Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Nomad for a Home Server

mrkaran.dev

Post reply on HN
Print view View original

111–120 of 153 posts

Re: Running Nomad for a Home Server

#111

Post by rektide »

chucky_z wrote: post #102
rektide wrote: post #67

A whole lot of anti- Kubernetes "you most likely don't need them and the increase in complexity makes it a pain to maintain" and "Kubernetes is exceedingly complex" in this thread & somewhat in this article. I agree that you probably don't need Kubernetes, and perhaps yeah it could be considered complex. But I think it's the right fit for most developers/doers & over time most operators too. Kubernetes is not Kuberne…

I agree with a lot of the points you make. Have you looked into nomad, consul and vault; along with everything they provide?

I really like all their offerings! Hashicorp writes incredibly easy to run & operate software. They have a wonderful software-design sense, identifying core problems & driving a well-defined approach forward. Rarely are companies so technically savvy in their offerings. It's clear from having run into all manners of Hashicorp people at conferences (back when that was a thing) that they have really great engineers too, talented & knowledgeable & engaged!

My Nomad familiarity is definitely a bit on the low side, & that's something I wouldn't mind changing. Consul & Vault I used to be able to operate & understood ok, but my knowledge has faded some.

Re: Running Nomad for a Home Server

#112

Post by dilyevsky »

throwaway894345 wrote: post #97
dilyevsky wrote: post #89

Earlier quoted context omitted.

There’s metallb that lets you announce bgp to upstream routers. Another solution would be to just announce it via daemonset on every node and setup a nodeport. Or just add every frontend node IP into DNS. Obv all highly non-standard as it depends on your specific setup

Yes, to be clear, these problems can be worked around (although many such workarounds have their own tradeoffs that must be considered in the context of the rest of your stack as well as your application requirements); I was observing that the defaults are not what I would consider to be production-ready.

I don’t think kubernetes ever promised to be a turnkey system at least outside of cloud. There are many commercial vendors though willing to fill that gap.

Re: Running Nomad for a Home Server

#113

Post by omneity »

sofixa wrote: post #73
omneity wrote: post #46

Earlier quoted context omitted.

Yeah, when making the decision it was quite harrowing to think of maintaining a cluster in production. Nomad had very little operational complexity compared to what we imagined. We've had two main outages in months: - Server disks were filling up and we hadn't set up monitoring properly at the time (ironic for the name of our company :) ). Not Nomad's fault. - A faulty healthcheck caused all the servers of a cluster…

Jaeger is pretty great for tracing, and can integrate with Traefik/Envoy ( or whatever you use for ingress/inter-service communication). We're running Loki for the logs ( via nomad log forwared/shipper and promtail) and so far it's going great. I'll have to do a write-up about the the whole thing.

Thank you for the pointers, very helpful. I'd love to see that write up too!

Re: Running Nomad for a Home Server

#114

Post by josegonzalez »

sweeneyrod wrote: post #96
ForHackernews wrote: post #71

Earlier quoted context omitted.

Are there any lightweight but production-ready PaaS offerings out there? I know there's Dokku, https://flynn.io/ looked super promising but I think it's basically dead now, same for Deis that is dead and forked to https://web.teamhephy.com/ .

Capnrover seems like the best option. I have the same perception of Flynn. Dokku is nice but I don't really see how you could use it in production since it's limited to a single server (there are definitely some cases where that's all you need but I can't imagine you'd need a PaaS for most of them).

Maintainer of Dokku here:

It isn't limited to a single system, and has a scheduler component for Kubernetes (https://github.com/dokku/dokku-scheduler-kubernetes) and Nomad (https://github.com/dokku/dokku-scheduler-nomad).

Re: Running Nomad for a Home Server

#115

Post by josegonzalez »

viraptor wrote: post #90
ForHackernews wrote: post #71

Earlier quoted context omitted.

Are there any lightweight but production-ready PaaS offerings out there? I know there's Dokku, https://flynn.io/ looked super promising but I think it's basically dead now, same for Deis that is dead and forked to https://web.teamhephy.com/ .

There seems to be a significant lack of open source PaaS as you mentioned. And Dokku is unfortunately a very leaky bucket of abstractions - fun if you're learning docker, annoying otherwise. This kind of makes sense given how much work it is and how easy it is to monetize as a paid service.

As the maintainer of Dokku, I'd love to hear more about what you think we can improve here. It is certainly an abstraction over basic Docker commands, but so are other schedulers and platforms.

That said, any feedback is more than welcome.

Re: Running Nomad for a Home Server

#116

Post by alephu5 »

parhamn wrote: post #9

Hashicorp has very cool stuff but I am not a fan of the config language they use on all their projects. It’s fine when I’m in an infrastructure type role and recency recall is fine but when Hashicorps tools are in my periphery it is a pain. Anyone share this? I guess the alternative is templated tomls/yamls/pseudo-jsons. Wish we’d all agree to one templated configuration format.

It's really frustrating for me that there's no way easy way to use YAML/JSON for configuring hashicorp products. It's supported at a low-level, but not from the CLIs for some reason...

I really like Dhall as a configuration language and use it across my whole system, so it's a shame I'm forced to use HCL or write a dhall backend for it.

Re: Running Nomad for a Home Server

#117

Post by ForHackernews »

josegonzalez wrote: post #114
sweeneyrod wrote: post #96

Earlier quoted context omitted.

Capnrover seems like the best option. I have the same perception of Flynn. Dokku is nice but I don't really see how you could use it in production since it's limited to a single server (there are definitely some cases where that's all you need but I can't imagine you'd need a PaaS for most of them).

Maintainer of Dokku here: It isn't limited to a single system, and has a scheduler component for Kubernetes ( https://github.com/dokku/dokku-scheduler-kubernetes ) and Nomad ( https://github.com/dokku/dokku-scheduler-nomad ).

I love the developer experience of Dokku!

That said, I would be very nervous to advocate using Dokku for any critical production systems. Are my fears ill founded? Or if not, is there any roadmap to making Dokku production-stable? How can people help?

Re: Running Nomad for a Home Server

#118

Post by 64mb »

GordonS wrote: post #91
3np wrote: post #52

Earlier quoted context omitted.

I'm doing exactly that, actually! Two bastion hosts/lbs sharing a virtual IP (keepalived), with two Traefik instances each (private and public). I actually schedule them through Nomad (on the host network interfaces) as well - since they solved the host networking issue I mentioned above it's properly set up with service checks. Super smooth to add and change services with consul catalog, and ACME-TLS included. Thing…

I recently setup Traefik 2.x to front a self-hosted Docker Registry, with automated Let's Encrypt renewals - I found the config to be really unintuitive and confusing! It feels like an awful lot of really finicky config for such a simply setup. Next time I'll try something else.

Traefik v1 is much simpler, v2 seemed to introduce so many extra layers which makes the simple stuff harder.

Re: Running Nomad for a Home Server

#119

Post by ipmb »

ForHackernews wrote: post #71
linux2647 wrote: post #45

Earlier quoted context omitted.

I’ve often thought that when developers saw Kubernetes, what they actually wanted was some form of PaaS

Are there any lightweight but production-ready PaaS offerings out there? I know there's Dokku, https://flynn.io/ looked super promising but I think it's basically dead now, same for Deis that is dead and forked to https://web.teamhephy.com/ .

If you're on AWS, you might be interested our product https://apppack.io.

Re: Running Nomad for a Home Server

#120

Post by Axsuul »

tetha wrote: post #59

We're pulling the trigger tomorrow to migrate the first productive system to nomad, and launch 1-2 new products on nomad first. It's quite exciting. We chose nomad there, because it's a business requirement to be able to self-host from an empty building due to the data we process - that's scary with K8. And K8 is essentially too much for the company. It's like 8 steps into the future from where most teams in product…

What did you migrate from?
Post reply on HN