Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do you really need Kubernetes?

blog.ekern.me

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Do you really need Kubernetes?

#51

Post by beeeeerp »

silverquiet wrote: post #40
beeeeerp wrote: post #34

Earlier quoted context omitted.

The biggest thing “the cloud” had gotten me is that I no longer have to call a sales rep and wait a week if I need a server. Yes this is a process problem, but for whatever reason it seems to stop being a problem (as much) with companies moving to the cloud. I have seen this 3+ times now. As an engineer, I’d much rather move off prem than fix bad management.

If you mean buying an actual server I order on menu regularly. If you mean a dedicated server in a DC then there are plenty of providers who can auto provision a bare metal instance for you. That was possible before AWS even opened up to public sales.

You’re not wrong, but generally the political friction of deploying a new service to k8s is lower than what you propose as well.

Re: Do you really need Kubernetes?

#52

Post by silverquiet »

beeeeerp wrote: post #51
silverquiet wrote: post #40

Earlier quoted context omitted.

If you mean buying an actual server I order on menu regularly. If you mean a dedicated server in a DC then there are plenty of providers who can auto provision a bare metal instance for you. That was possible before AWS even opened up to public sales.

You’re not wrong, but generally the political friction of deploying a new service to k8s is lower than what you propose as well.

Well yes, but I run container orchestration on owned servers (Swarm, not Kubernetes) so it's really easy to add a new service at a whim and only quite rarely do we need to add capacity, to which we can also add in a cloud provider. But I'm using off-lease servers which are sort of bonkers as to how much capacity you get for the price. I think they cost less than a months spend on an equivalent cloud instance.

Re: Do you really need Kubernetes?

#53

Post by beeeeerp »

silverquiet wrote: post #52
beeeeerp wrote: post #51

Earlier quoted context omitted.

You’re not wrong, but generally the political friction of deploying a new service to k8s is lower than what you propose as well.

Well yes, but I run container orchestration on owned servers (Swarm, not Kubernetes) so it's really easy to add a new service at a whim and only quite rarely do we need to add capacity, to which we can also add in a cloud provider. But I'm using off-lease servers which are sort of bonkers as to how much capacity you get for the price. I think they cost less than a months spend on an equivalent cloud instance.

That’s a good point. Preemptable instance can be insanely cheap. I would guess it’s more efficient for the data center overall to shuttle services around to different nodes as needed.

Re: Do you really need Kubernetes?

#54

Post by 1letterunixname »

Hey now, I made a killing in AWS consulting to convince megacorps to get rid of their own hardware and avoid going the OpenStack route.

The problems of pre-IaaS and pre-K8s were manageability, flexibility, and capacity utilization. These problems still haven't really been solved to a standardized, interoperable, and uniform manner because stacks continue to mushroom in complexity. Oxide appears to be on the right track but there is much that can be done to reduce the amount of tinkering, redundant abstractions, and avoiding conventional lifecycle management and cross-cutting concerns that people don't want to think about whenever another new way comes along.

Re: Do you really need Kubernetes?

#55

Post by 1letterunixname »

nova22033 wrote: post #13
zer00eyz wrote: post #2

There is a corollary to this: Do you really need cloud infrastructure? Cattle not pets right? Well, no. Have you seen amazons AWS margins? Its 30%. After amazon buys hardware, pays people to run it, it still makes 30%. Not having hardware is someone else's profit. That isnt cattle, its contract poultry farming. Learn capacity planing. Learn to write cachable, scalable apps. Track your hardware spend per customer. Lea…

Learn to write cachable, scalable apps. I'm expected to write a service like s3?

That's not what they meant. And, S3-compatible object stores already exist. Ceph, for one.

Cachable, scalable apps that you write as part of a web-centric business are ones that use in-application level knowledge to coordinate with the web framework to use caching services at each level of depth of the stack, be it 3rd-party Cloudflare or DIY CDNs (varnish/squid), memcache/NoSQL/database-level.

Re: Do you really need Kubernetes?

#56

Post by 1letterunixname »

vrnvu wrote: post #3

I'd say 99% of companies using Kubernetes can't really explain why they chose it over Nomad.

I don't think most companies can say why they're not at least isolating workloads with something like Kata Containers and why they're using only glorified cgroup jails, have an inventory of all the services they're running and why, can point to which machines have authoritative copies of data, how they back it up without replicating corruption, and how they'd do disaster recovery/BCP on it.

Re: Do you really need Kubernetes?

#57

Post by 1letterunixname »

ceejayoz wrote: post #17
tredre3 wrote: post #4

Related: https://doineedkubernetes.com/

Does this site return something different if it detects a Google/Facebook/Amazon corporate IP visiting?

It wouldn't be hard to check by BGP AS.

Stack Overflow routinely didn't work at one MAANG because of egress from a narrow range of IPs appearing to be bot-like behavior, but it was really an enormous amount to traffic from ~10k's of users.

Re: Do you really need Kubernetes?

#58

Post by bcaxis »

bigyikes wrote: post #6
zer00eyz wrote: post #2

There is a corollary to this: Do you really need cloud infrastructure? Cattle not pets right? Well, no. Have you seen amazons AWS margins? Its 30%. After amazon buys hardware, pays people to run it, it still makes 30%. Not having hardware is someone else's profit. That isnt cattle, its contract poultry farming. Learn capacity planing. Learn to write cachable, scalable apps. Track your hardware spend per customer. Lea…

Cloud is an abstraction over hardware. lLike any good abstraction, it makes certain tradeoffs. Sometimes it makes sense to move to a lower level of abstraction, for performance, cost, or compatibility reasons. In this case, diving under the cloud and running your own servers could save 30% of your server costs. Cloud adoption has shown that many (most?) companies prefer the convenience over cost savings. Maybe optimi…

AWS spends tons of engineering time on features you don't need. There is a lot more than 30% on the table.

Re: Do you really need Kubernetes?

#59

Post by clintonb »

bhouston wrote: post #14

I found that just using CloudRun and similar technologies is simpler and easier to manage than kubernetes. You need auto scaling, fast startup, limit number of concurrent connections to each instance, and scale to zero functionality.

I agree that Cloud Run greatly simplifies deployments.

Unfortunately, it only auto-scales based on requests and, eventually, CPU. We are in the process of moving our Temporal workers from Cloud Run to GKE Autopilot, which is ~30% cheaper given we can use arm64 Scale-Out nodes.

Re: Do you really need Kubernetes?

#60

Post by bcaxis »

romanhn wrote: post #19
zer00eyz wrote: post #2

There is a corollary to this: Do you really need cloud infrastructure? Cattle not pets right? Well, no. Have you seen amazons AWS margins? Its 30%. After amazon buys hardware, pays people to run it, it still makes 30%. Not having hardware is someone else's profit. That isnt cattle, its contract poultry farming. Learn capacity planing. Learn to write cachable, scalable apps. Track your hardware spend per customer. Lea…

I fully agree with your last paragraph, but not sure what it has to do with cloud infrastructure specifically. AWS also leverages huge economies of scale into profit, it's not like you're going to realize equivalent margins by running your own little server in a colo somewhere. You certainly won't realize equivalent availability, scalability, security, support ecosystem, etc with it either. Cloud infra can make a lot…

> it's not like you're going to realize equivalent margins by running your own little server

I get even better than their margins. In the neighborhood of 100x cheaper. It's not rocket science.

> equivalent availability

AWS availability isn't all that super great. Most of their services are rated for only 99.95 before they offer pittance credits. That's not difficult to meet with a single computer...

> scalability

A service only needs good enough scalability. Auto scaling is also a bug, not just a feature. Remember that it is also tied to auto billing. I can't afford to have my wallet DDOSed.

> security

Security is always a problem that needs to be solved. You don't get an auto pass on security needs because you signed up for AWS.

> support

You have to pay for that.... You can get it from other vendors too if you have budget for it.

> ecosystem

This is flat out wrong. Compare AWS offerings to the breadth and depth of open source offerings and the latter comes out far ahead.

Post reply on HN