Live data from Hacker News

How we use HashiCorp Nomad

blog.cloudflare.com

41–50 of 166 posts

Re: How we use HashiCorp Nomad

#41
I'm surprised Hashicorp hasn't repositioned this product.

Terraform was a huge breath of fresh air after Cloudformation. If you ask me, deploying apps via k8s is even better.

Everyone wants a free lunch and for me, momentum + cloud vendor support + ultimately the Nomad Enterprise features that come free with K8s made the choice easy.

Re: How we use HashiCorp Nomad

#42
post #37

I worked at a hedge fund that also used nomad. The problem, however, is not how well it scales or whatever, but the fact that all the accompanying deployment info and literature is for kubernetes, and k8s has far more features. I like the quality of products from HashiCorp, but k8s is far, far, ahead of where nomad is. What I really want is better integration for Terraform and kubernetes. The current TF k8s support l…

Crossplane https://crossplane.io might be what you’re looking for with its bunch of controllers and a nice composition API. One of the best features is that you can bundle a CR to request a MySQL database and it will be satisfied with whatever config is in your cluster so that app only declares the need but not care how it’s done. Disclaimer: I’m one of the maintainers of Crossplane.

Any chance you plan to integrate with Google's Config Connector? It's very similar to crossplane (but gcp specific).

https://cloud.google.com/config-connector/docs/overview

Re: How we use HashiCorp Nomad

#43
post #13

Off topic but... I've always wondered how Cloudflare can not charge for the bandwidth. Even the free plan is super generous (CDN, SSL, etc). How are they making a profit when AFAIK all other CDNs charge you for the bandwidth (and I assume they have to pay for to their providers)?

"2.8 Limitation on Serving Non-HTML Content

The Service is offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as a part of a Paid Service purchased by you, you agree to use the Service solely for the purpose of serving web pages as viewed through a web browser or other functionally equivalent applications and rendering Hypertext Markup Language (HTML) or other functional equivalents. Use of the Service for serving video (unless purchased separately as a Paid Service) or a disproportionate percentage of pictures, audio files, or other non-HTML content, is prohibited."

In other words, as soon as you start using significant CDN bandwidth for images, video, audio, etc. they will contact you and ask you to upgrade your account.

Re: How we use HashiCorp Nomad

#44

I worked at a hedge fund that also used nomad. The problem, however, is not how well it scales or whatever, but the fact that all the accompanying deployment info and literature is for kubernetes, and k8s has far more features. I like the quality of products from HashiCorp, but k8s is far, far, ahead of where nomad is. What I really want is better integration for Terraform and kubernetes. The current TF k8s support l…

"Kubernetes but with less stuff" is a valuable niche that I'm glad someone is targeting.

Re: How we use HashiCorp Nomad

#45
One point i never get for companies operating their own hardware: If your problem was having a number of well known servers for the internal management services and you then move to a nomad cluster or kubernetes to schedule them dynamically, you end up with the same problem as before to schedule the well known nomad servers or kubernetes masters. So is the only advantage here that the nomad server images update less often than the images of the management services?

Re: How we use HashiCorp Nomad

#47
post #13

Off topic but... I've always wondered how Cloudflare can not charge for the bandwidth. Even the free plan is super generous (CDN, SSL, etc). How are they making a profit when AFAIK all other CDNs charge you for the bandwidth (and I assume they have to pay for to their providers)?

> How are they making a profit

They aren't. In Q1 they made a loss of $33M on $91M in revenue.

Re: How we use HashiCorp Nomad

#48

I worked at a hedge fund that also used nomad. The problem, however, is not how well it scales or whatever, but the fact that all the accompanying deployment info and literature is for kubernetes, and k8s has far more features. I like the quality of products from HashiCorp, but k8s is far, far, ahead of where nomad is. What I really want is better integration for Terraform and kubernetes. The current TF k8s support l…

"Kubernetes but with less stuff" is a valuable niche that I'm glad someone is targeting.

That's a fair point, I guess it depends on your use case. The risk, however, is that the powers that be at HashiCorp one day decide to abandon Nomad once they realize it will never be a profit centre for them.

Re: How we use HashiCorp Nomad

#49
post #5

Cloudflare, make sure you upgrade to 0.11.3, the new scheduling behavior is awesome for large clusters. Also a massive warning to anyone wanting to use hard cpu limits and cgroups (they do work in nomad it’s just not trivial), they don’t work like anyone expects and need to be heavily tested.

are you serious? you've got a major company touting a technology and a key component of it is broken?

Yes, you know that small organization, Linux; who manages cgroups that everything else calls through with.

I have only tested it with Docker, again, that company Moby with this very little used software.

How dare their core systems have any systems that are not extremely simple to understand and have caveats.

Re: How we use HashiCorp Nomad

#50

Earlier quoted context omitted.

I haven’t tested hard CPU quotas with Nomad but I suspect the issue mentioned above is due to cgroups/CFS and is also applicable to Kubernetes. See these slides for more details: https://www.slideshare.net/mobile/try_except_/optimizing-kub...

I recall that there's been some improvements to CFS to fix this since 2018.

There's been a ton of improvements over time. That doesn't make it any less foot-gun-y, unfortunately.

I had the exact same problem with Kubernetes (and even just straight up containers), I just want to make sure folks are extremely aware of how big of a footgun it is, and to really, really, really test it well.

Post reply on HN