Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We're Leaving Kubernetes

gitpod.io

Post reply on HN
Print view View original

241–250 of 348 posts

Re: We're Leaving Kubernetes

#241

Post by liveoneggs »

justinclift wrote: post #157
czhu12 wrote: post #148

Earlier quoted context omitted.

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

> installing ~50ish microservices on their machine Ouch. Where they using macOS at the time with laptops having not-enough-ram? I've seen that go poorly on macOS with java based microservices. Largely due to java VMs wanting ram pre-assigned for each, which really chews though ram that mostly sits around unused. This was a few years ago though, at the tail end of Intel based mac's where 32GB ram in a mac laptop wasn'…

why would run multiple JVMs?

Re: We're Leaving Kubernetes

#242

Post by KronisLV »

rtpg wrote: post #227
pmarreck wrote: post #88

Earlier quoted context omitted.

In my last role as a director of engineering at a startup, I found that a project `flake.nix` file (coupled with simply asking people to use https://determinate.systems/posts/determinate-nix-installer/ to install Nix) led to the fastest "new-hire-to-able-to-contribute" time of anything I've seen. Unfortunately, after a few hires (hand-picked by me), this is what happened: 1) People didn't want to learn Nix, neither d…

I trialed for a job where the CTO was convinced of dev environments in kube as "the way to work". Everyone else was at least ambivalent. I joined, tried to make some changes that would let me run things locally. Every time I got pushback about using the dev environments instead. It took me a couple of days to get a supervisor-based setup working locally. I was the only person on the team who would run the backend and…

> I really feel like if you are in a position to determine tooling, it's so much more helpful to lean into whatever people on the ground want to use.

This might mean picking something that you think/know kind of sucks for the task, but that will be easier for most people to grok - while it might subjectively feel unfortunate, it's probably the right thing to do, for the sake of the majority of the team having an easier time.

Pushing your interests more strongly, or even in a top down fashion, might work, but that's more risky both in regards to letting everyone get things done, as well as team cohesion and turnover.

Re: We're Leaving Kubernetes

#243

Post by Cthulhu_ »

blitzar wrote: post #214
flimsypremise wrote: post #169

Earlier quoted context omitted.

If the microservice has dependencies on other services it is not a microservice.

Connecting to a messaging queue or database count as a dependency? Why not break a microservice into a series of microservices, its microservices all the way down.

Only if you cannot change one service without changing the other simultaneously. It's fine to have evolving messages on the queue but they have to be backwards compatible with any existing subscribers, because you cannot expect all subscribers to update at the same time. Unless you have a distributed monolith in a monorepo, but at least be honest about it.

Multiple services connecting to the same database has been considered a bad idea for a long time. I don't necessarily agree, but I have no experience in that department. It does mean more of your business logic lives in the database (rules, triggers, etc).

Re: We're Leaving Kubernetes

#244

Post by Cthulhu_ »

themanmaran wrote: post #165
justinclift wrote: post #157

Earlier quoted context omitted.

> installing ~50ish microservices on their machine Ouch. Where they using macOS at the time with laptops having not-enough-ram? I've seen that go poorly on macOS with java based microservices. Largely due to java VMs wanting ram pre-assigned for each, which really chews though ram that mostly sits around unused. This was a few years ago though, at the tail end of Intel based mac's where 32GB ram in a mac laptop wasn'…

I bet it's less of a RAM issue, and more of an orchestration problem. Making sure you have the latest version of every microservice and it's configuration. "Oh it's not running locally, you need to also run service_18_v2.js, and include the right env variables"

If you have to have the latest version etc of every microservice, it's a distributed monolith.

Re: We're Leaving Kubernetes

#245

Post by eloisant »

czhu12 wrote: post #148
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

In a past job I've had a good experience in this case with docker compose (well, something similar).

You would list the services you need (or service groups) in a config file, start a command and all services would start in containers. Sure, you need a lot of RAM with that but on 32Gb it was working fine.

Re: We're Leaving Kubernetes

#246

Post by linuxftw »

The article offers toward the end that now self-hosted customers can run their app on something other than k8s. I think this is a mistake. We're a k8s enterprise shop, and I don't want to support any more VMs. If it's not on k8s, I'm not running it. I don't want to be responsible for golden images, patching, and all the fun that comes with managing workloads outside of k8s. That's why I have k8s.

All the problems in the article also seem self-imposed. k8s can run stateful workloads just fine. Don't start and stop them. Figure out the math on how much it costs to run a container 24/7, add your margin, and pass that cost to the customer. Customer can decide to stop the containers to save $$, so the latency won't hurt, they'll accept it because they know they're saving money.

Re: We're Leaving Kubernetes

#247

Post by wlll »

czhu12 wrote: post #148
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

That's ~12 people per microservice. I understand that it might be hard to boot and coordinate all that on each developers machine, and that local docker images/vms/whatever might not suit you, but is it possible that this is a problem of your companies own creation?

Re: We're Leaving Kubernetes

#248

Post by jen20 »

czhu12 wrote: post #148
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

> Both FAANG companies I’ve worked at had remote dev environments that were built in house.

This is certainly not universal among FAANGs though.

Requiring 50 services to be up is absolutely nuts, but it’s actually pretty trivial using something like Nomad locally.

Re: We're Leaving Kubernetes

#249

Post by Kye »

gchamonlive wrote: post #238
czhu12 wrote: post #148

Earlier quoted context omitted.

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

You need an option to enable only the services they need to develop locally and automatically configure them to talk to the other services hosted elsewhere.

What ever happened to make? It seems like so much of what developers do now just duplicates what could be done with a single makefile that pokes around, figures out what you need, and makes a script to set it up.

Re: We're Leaving Kubernetes

#250

Post by davewritescode »

bhouston wrote: post #17
candiddevmike wrote: post #14

Earlier quoted context omitted.

You know that Cloud Run is effectively a Kubernetes PaaS, right?

> You know that Cloud Run is a Kubernetes PaaS, right? Yup. Isn't it Knative Serving or a home grown Google alternative to it? https://knative.dev/docs/serving/ The key is I am not managing Kubernetes and I am not paying for it - it is a fool's errand, and incredibly rarely needed. Who cares what is underneath the simple Cloud Run developer UX? What matters for me is cost, simplicity, speed and understandability. You…

I haven't looked at Cloud Run pricing but running Kubernetes in the cloud is pretty cheap these days and my experience with solutions like Cloud Run in the past is that they end up becoming expensive.

Kubernetes can be as complex or as expensive as you'd like but it's also fairly possible to run a pretty bulletproof simple Kube cluster.

Post reply on HN