Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes is a red flag signalling premature optimisation

jeremybrown.tech

Post reply on HN
Print view View original

421–430 of 558 posts

Re: Kubernetes is a red flag signalling premature optimisation

#421

Post by jbmsf »

hotpotamus wrote: post #417
bearjaws wrote: post #415

Earlier quoted context omitted.

Most startups get basic security for networking and compute wrong, K8s just adds even more things to mess up. Odds are even if you use an out of the box solution, unless you have prior experience you will get it wrong. I will always recommend using whatever container / function as a service e.g. ECS, GCF, Lambda any day over K8s for a startup. With these services its back to more similar models of security such as ne…

So question then - is it possible to found a tech startup without paying rent to a FAANG? Before I get the answer that anything is possible, I should say is it feasible or advisable to start a company without paying rent to the big guys?

re: advisable

If and only if your business model depends on it. A startup's job is mostly to find product market fit; if being decoupled from AWS isn't part of your market, you are spending money on a non-problem.

Re: Kubernetes is a red flag signalling premature optimisation

#422

Post by iasay »

nprateem wrote: post #404
Noughmad wrote: post #388

Earlier quoted context omitted.

In what cases would I have to worry about that?

If it breaks. If you have an outage because of weird K8s networking issues (and I've seen them) you'll suddenly care very much.

Yes same. We had one which we had to recreate a whole EKS cluster because AWS couldn’t fix it too.

I don’t sleep these days.

Re: Kubernetes is a red flag signalling premature optimisation

#423

Post by gravypod »

okamiueru wrote: post #419
hotpotamus wrote: post #417

Earlier quoted context omitted.

So question then - is it possible to found a tech startup without paying rent to a FAANG? Before I get the answer that anything is possible, I should say is it feasible or advisable to start a company without paying rent to the big guys?

There is nothing stopping you from hosting your own OpenStack, managed k8s, and all that, on your own hardware. You would need a good reason to not let someone else deal with all of this though.

For a small enough company you could even just use use k3s + offsite backups. Once you grow large enough you can setup machines in 2-4 locations across the land mass where your users exist. If you have enough than a hardware fault in one isn't an emergency and you'd be able to fly out to fix things if needed.

Realistically, on all flash, you are very unlikely to need to maintain anything on a server for a few years after deployment.

Re: Kubernetes is a red flag signalling premature optimisation

#424

Post by Spooky23 »

hotpotamus wrote: post #417
bearjaws wrote: post #415

Earlier quoted context omitted.

Most startups get basic security for networking and compute wrong, K8s just adds even more things to mess up. Odds are even if you use an out of the box solution, unless you have prior experience you will get it wrong. I will always recommend using whatever container / function as a service e.g. ECS, GCF, Lambda any day over K8s for a startup. With these services its back to more similar models of security such as ne…

So question then - is it possible to found a tech startup without paying rent to a FAANG? Before I get the answer that anything is possible, I should say is it feasible or advisable to start a company without paying rent to the big guys?

Who would you prefer to pay rent to?

The reality is unless you’re some rich dude who can borrow dad’s datacenter (And that’s cool if so), you’re either going to be renting colo space, virtual servers, etc.

It’s always a challenge in business to avoid the trap of spending dollars to save pennies.

IMO, you’re better off working in AWS/GCP/Azure and engineering around the strengths of those platforms. That’s all about team and engineering discipline. I’m not in the startup world, but I’ve seen people lift and shift on-prem architecture and business process to cloud and set money on fire. Likewise, I’ve seen systems that reduced 5 year TCO by 80% by building to the platform strengths.

Re: Kubernetes is a red flag signalling premature optimisation

#425

Post by mattbillenstein »

digianarchist wrote: post #379
mattbillenstein wrote: post #3

In the limit, there are some startups that could run production on a single Linux host - I recently helped one get off Heroku and their spend went from ~$1k/mo to ~$50/mo and it made debugging and figuring out performance issues so much easier than what they were doing previously...

I've worked in DevOps for a while and if I could pay $950 to not run and maintain a server then I'd consider it money well spent. There's always 1-2 comments in these threads that advise using a Linode VM or Hetzner dedicated server in order to save money; but they are really skipping over the headaches that come with building and maintaining your own servers. - Are the server provisioning scripts source controlled?…

I think this all ignores the opaqueness of most PaaS providers - when everything is on a single box, you have infinite observability, standard Linux perf analysis tools, etc.

If you do it correctly, it is loads easier to reason about and understand than a complex k8s deployment.

Re: Kubernetes is a red flag signalling premature optimisation

#426

Post by mattbillenstein »

wronglyprepaid wrote: post #368
gjulianm wrote: post #75

Earlier quoted context omitted.

> I guess redundancy is not really a thing then? Most of the time it really doesn't need to be. In the end what you care about is uptime and cost. A redundant solution doesn't have a perfect uptime just because it's redundant, in fact sometimes it might have even less uptime because of failures in the redundancy mechanism. Of course if you need to be always up it might be worth it. But for a lot of situations some do…

> A redundant solution doesn't have a perfect uptime just because it's redundant, in fact sometimes it might have even less uptime because of failures in the redundancy mechanism I'm fairly sure that google does a better job keeping cloud spanner and cloud run working, and their redundancy mechanisms working, than whoever runs your single linux box will do. > most of the time it's better to be on a simpler setup that…

> > A redundant solution doesn't have a perfect uptime just because it's redundant, in fact sometimes it might have even less uptime because of failures in the redundancy mechanism > I'm fairly sure that google does a better job keeping cloud spanner and cloud run working, and their redundancy mechanisms working, than whoever runs your single linux box will do.

You'd be surprised - the major cloud providers have outages all the time.

Google in particular will have some random backing service firing 502's seemingly randomly while their dashboards say "all good".

Re: Kubernetes is a red flag signalling premature optimisation

#427

Post by jollybean »

hotpotamus wrote: post #417
bearjaws wrote: post #415

Earlier quoted context omitted.

Most startups get basic security for networking and compute wrong, K8s just adds even more things to mess up. Odds are even if you use an out of the box solution, unless you have prior experience you will get it wrong. I will always recommend using whatever container / function as a service e.g. ECS, GCF, Lambda any day over K8s for a startup. With these services its back to more similar models of security such as ne…

So question then - is it possible to found a tech startup without paying rent to a FAANG? Before I get the answer that anything is possible, I should say is it feasible or advisable to start a company without paying rent to the big guys?

If you're priorities are 'which companies do my values align with among generally very high integrity companies to begin with' - then you might want to reconsider.

Google is not evil. They're just big, and define some practices which we might think should change in the future.

Once you have the thing up and running, you can start to think about hosting your own.

Also, you don't need to use fancy services because most startups can run just fine on a single instance of whatever, meaning, there are a lot of cloud providers out there.

Re: Kubernetes is a red flag signalling premature optimisation

#428

Post by mattbillenstein »

regularfry wrote: post #148
gjulianm wrote: post #85

Earlier quoted context omitted.

> I'm very much with you on this, but I do understand that it's one of those things that is just not feasible when your team has no sysadmin/devops experience. But this applies to everything. You also need Heroku or Kubernetes or whatever experience to maintain those systems, right?

The question is how much you need to know. You need much less knowledge to run something in Heroku than in k8s; dramatically less in the "onboard a CRUD app" case. I'd argue that running k8s effectively is no less knowledge-intensive than running VMs.

> I'd argue that running k8s effectively is no less knowledge-intensive than running VMs.

LOL - are you f'ing kidding?

Running VMs is much closer to running your app locally than containers and k8s - it's not that hard to do badly, and only marginally harder to do well.

Re: Kubernetes is a red flag signalling premature optimisation

#429

Post by marcosdumay »

okamiueru wrote: post #106
pritambarhate wrote: post #84

Doesn't look like the author knows what he is talking about. His point about early stage startup should not use K8S is fine. But the next advice about not using a different language for frontend and backend is wrong. I think the most appropriate advice is to choose a stack which the founding team is most familiar with. If that means RoR then RoR is fine. If it means PHP then PHP is fine too. Another option is to use…

I don't think their advice about not using it in a startup is correct either. You just need to somewhat know what you're doing. I know of such a case, where a single engineer could leverage the helm chart open source community, and set up a scalable infrastructure, with prometheus, grafana, worker nodes that can scale independently of web service, a CI/CD pipeline that can spin up complete stacks with TLS automated t…

> and set up a scalable infrastructure

As a rule, for anything in an startup, adding that "scalable" adjective is a waste.

Of course, exceptions exist, but it's one of those problems that if you have them, you probably know it, and if you have them and don't know it, you won't succeed anyway. So any useful advice has the form of "don't".

Re: Kubernetes is a red flag signalling premature optimisation

#430

Post by mattbillenstein »

nprateem wrote: post #410
onion2k wrote: post #223

Earlier quoted context omitted.

Obviously not. You don't pay someone to just set it up. You pay to help do what you'd do if you had a dedicated devOps teams. You pay someone to set up the system with your team so they understand it , train your team to use it, write some documentation about it, script a rollback procedure, maybe help on developing playbooks, etc. Besides, there are people out there who offer on-call services for a small retainer.

This is optimistic to say the least. I've worked as an SRE for 5 years and apart from the others in the team the devs don't have nearly as much knowledge. There's no way I'd rely on them to fix an outage. And even on a small retainer you'd better hope they retained the knowledge of how all that stuff works if you're only calling on them every now and again.

The idea is devops as _culture_ - I come in as an expert and set it up, then show them how I did it, then run through various disaster recovery scenarios so they learn it and can handle the vast majority of problems.

And you'd be surprised how little problems you might have - I've had many VMs with literally years of uptime running without issue.

Most people focus on "devops" as a job - and they never bother to teach the rest of the team anything about how stuff works. Worse than that, modern clouds encourage you to build opaque and complicated systems which probably only someone working on them full-time has a hope to understand...

Post reply on HN