Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Mesos to be moved to Attic

lists.apache.org

Post reply on HN
Print view View original

81–90 of 139 posts

Re: Apache Mesos to be moved to Attic

#81

Post by tpetr »

relistan wrote: post #48

The real gem of the Mesos ecosystem was the much lesser known Singularity scheduler from HubSpot. https://github.com/HubSpot/Singularity I've run this at scale, in production since 2015 and it has been absolutely rock solid and does most of the production things you'd want. Unlike commercial products, it was written to run HubSpot's own infrastructure so it does what a production system needs. Really bummed to have t…

Thanks for the kind words! Singularity is still a core piece of infrastructure for running stateless apps at HubSpot (our status page currently reads ~13k Requests, ~22k Active Tasks, ~700 Agents). We're also heavily investing in Kubernetes, but Singularity is solid enough that our focus has been more towards reliably running and scaling stateful things like MySQL, ZK, Kafka, HBase, and ES.

Re: Apache Mesos to be moved to Attic

#83

Post by lovedswain »

lima wrote: post #75
dragonsh wrote: post #7

Earlier quoted context omitted.

For over 90% of workloads kubernetes is an overkill. Only when company is reaching google scale kubernetes make sense. A good alternative to kubernetes is LXD [1] or just stick with docker compose. Kubernetes except for managed services from cloud providers is more difficult than an average application to manage and a huge cost in itself to run and maintain. [1] https://www.linuxcontainers.org/

> For over 90% of workloads kubernetes is an overkill. Only when company is reaching google scale kubernetes make sense. I hear people repeating this truism all day, but from practical experience, it doesn't seem to be the case - Kubernetes is paying dividends even in small single-node setups.

K8s single noder and GKE user here, can confirm I wouldn't remotely even consider going back. Deploying an app takes 5-10 minutes at most first time around, new pushes I hated Kubernetes ops complexity at first, but there really isn't that much to it, and if it's too much, a service like GKE takes 70%+ of it away from you

Re: Apache Mesos to be moved to Attic

#84

Post by ForHackernews »

orf wrote: post #80
ForHackernews wrote: post #59

Earlier quoted context omitted.

Yes, yes, there's a million zillion Kubernetes ingress things, none of which are really good enough that anyone uses them without a cloud-provider LB in front of it. Also they only deal with HTTP/S traffic. Got other types of traffic you want to run? Too bad, tunnel it over HTTPS. If you want a picture of the future of computing, imagine everything-over-HTTPS-on-k8s-on-AWS stomping on a human face forever.

> Got other types of traffic you want to run? Too bad, tunnel it over HTTPS. Then you’d expose a service, not an ingress. You can do this in a variety of ways depending on your environment. I’m going to go out on a limb here and say you’ve never really used k8s and haven’t really grokked even the documentation. It’s complicated, some parts more than others, but if you’re still at the “wow guys secrets are not really…

> Then you’d expose a service, not an ingress.

That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s?

> if you’re still at the “wow guys secrets are not really secret!1!” level

I'm just pointing out one (of many) extremely obvious warts on k8s. You act like this is some misconception on my part, but it's not that silly to assume that a secret would be.

But to answer your smarm, yes, I've used Kubernetes in anger: my last company was all-in on k8s because it's so trendy, and it was (IMHO) an absolute nightmare. All kinds of Stockholm-syndrome engineers claiming that writing thousands of lines of YAML is a great experience, unable to use current features because even mighty Amazon can't safely upgrade a k8s cluster....

Re: Apache Mesos to be moved to Attic

#85

Post by orf »

ForHackernews wrote: post #84
orf wrote: post #80

Earlier quoted context omitted.

> Got other types of traffic you want to run? Too bad, tunnel it over HTTPS. Then you’d expose a service, not an ingress. You can do this in a variety of ways depending on your environment. I’m going to go out on a limb here and say you’ve never really used k8s and haven’t really grokked even the documentation. It’s complicated, some parts more than others, but if you’re still at the “wow guys secrets are not really…

> Then you’d expose a service, not an ingress. That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s? > if you’re still at the “wow guys secrets are not really secret!1!” level I'm just pointing out one (of many) extremely obvious warts on k8s. You act like this is some misconception on my part, but…

> That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s?

Quite the opposite. I’d re-read the docs[1]. Specifically this page[2]. If you’re on AWS you’d probably wire this up with a NLB and a bit of terraform if you’re allergic to YAML. Seems like a 5 minute job assuming you have Dovecot correctly containerized.

> I'm just pointing out one (of many) extremely obvious warts on k8s. You act like this is some misconception on my part

It’s hard not to point out misconceptions like the one above.

1. https://kubernetes.io/docs/concepts/services-networking/

2. https://kubernetes.io/docs/concepts/services-networking/serv...

Re: Apache Mesos to be moved to Attic

#86

Post by ForHackernews »

orf wrote: post #85
ForHackernews wrote: post #84

Earlier quoted context omitted.

> Then you’d expose a service, not an ingress. That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s? > if you’re still at the “wow guys secrets are not really secret!1!” level I'm just pointing out one (of many) extremely obvious warts on k8s. You act like this is some misconception on my part, but…

> That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s? Quite the opposite. I’d re-read the docs[1]. Specifically this page[2]. If you’re on AWS you’d probably wire this up with a NLB and a bit of terraform if you’re allergic to YAML. Seems like a 5 minute job assuming you have Dovecot correctly co…

From the docs you linked:

> ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.

Internal only.

> NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.

Nobody uses NodePort to expose external services directly, and I think you know that.

> LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.

As I mentioned above in this thread, requires cloud provider Load Balancer.

> ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up.

> Note: You need either kube-dns version 1.7 or CoreDNS version 0.0.8 or higher to use the ExternalName type.

This one's a new one to me, and apparently relies on some special new widgets.

Anyway, if you love k8s, I'm sure you'll have a profitable few years writing YAML. Enjoy.

Re: Apache Mesos to be moved to Attic

#87

Post by lars_francke »

Nothing is decided yet. This is a vote thread which _might_ end with someone interested in Mesos stepping in to pick the project up.

This hasn't happened in a while with other projects (a whole bunch of Apache projects have just retired in the last few months) but with Mesos there might be a chance.

Re: Apache Mesos to be moved to Attic

#88

Post by madhadron »

dharmab wrote: post #31
jeffbee wrote: post #26

Earlier quoted context omitted.

Are there examples of high-utilization, large-scale Mesos deployments? Mesos didn't even gain over-commit until 2015, so it seems like it was generally behind the state of the art.

Most famously, Siri (used to?) run on a very large scale Mesos deployment (10000s of nodes, much higher than Kubernetes can scale to). Unfortunately the original article is lost, but here's a summary: https://daringfireball.net/linked/2015/04/29/siri-apache-mes...

To be fair, Kubernetes right now only schedules relatively small clusters. But it turns out that the majority of the world is not Facebook or Google and only needs relatively small clusters.

Re: Apache Mesos to be moved to Attic

#89

Post by madhadron »

dragonsh wrote: post #7
fiberoptick wrote: post #2

End of an era... Are there any viable alternatives to Kubernetes and Nomad?

For over 90% of workloads kubernetes is an overkill. Only when company is reaching google scale kubernetes make sense. A good alternative to kubernetes is LXD [1] or just stick with docker compose. Kubernetes except for managed services from cloud providers is more difficult than an average application to manage and a huge cost in itself to run and maintain. [1] https://www.linuxcontainers.org/

> Only when company is reaching google scale kubernetes make sense.

Long before Google scale. Kubernetes won't handle clusters anywhere near that large.

Re: Apache Mesos to be moved to Attic

#90

Post by orf »

ForHackernews wrote: post #86
orf wrote: post #85

Earlier quoted context omitted.

> That only works to access the service from other pods inside k8s, it doesn't help you make that service accessible to the outside world. Tell me how you'd run Dovecot on k8s? Quite the opposite. I’d re-read the docs[1]. Specifically this page[2]. If you’re on AWS you’d probably wire this up with a NLB and a bit of terraform if you’re allergic to YAML. Seems like a 5 minute job assuming you have Dovecot correctly co…

From the docs you linked: > ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType. Internal only. > NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Se…

> This one's a new one to me, and apparently relies on some special new widgets.

ExternalName has been around since 2016.

> Nobody uses NodePort to expose external services directly, and I think you know that.

Sure they do. Anyone using a LoadBalancer does this implicitly. If you don’t want k8s to manage the allocated port or want to use something bespoke that k8s doesn’t offer out of the box then using a NodePort is perfectly fine. You can also create a custom resource type if you’ve got some bespoke setup that can be driven by an internal API.

The happy path is using a cloud load balancer, because that’s what you’d use if you are using a cloud provider and you’re comfortable with k8s wiring it all up for you.

Has your criticism of k8s evolved from “I’m unclear about services” to “well yes it supports everything I want out of the box but uhh nobody does it that way and therefore it can’t do it”?

Post reply on HN