Live data from Hacker News

We're Leaving Kubernetes

gitpod.io

221–230 of 348 posts

Re: We're Leaving Kubernetes

#221
post #200

For development, I made the switch to nix/flox and it’s been a game-changer.

How well does Flox work out of the box? I would really like to introduce Nix to the dev environments in my company but the struggle of maintaining nix files and flakes is too large. I've looked at DevBox and it looks quite accessible but Flox also looks like a nice way to sneak some of the Nix goodness into the company.

Re: We're Leaving Kubernetes

#222

Earlier quoted context omitted.

Invariably this is an ideal and does not match up in reality. I work at ~50 ish employee company and we have layers of dependencies between at least 6 or 7 various microservices. I can see this adding up in complexity as the product scales

If the microservice has dependencies on other services it is not a microservice.

According to whom? How do those microservices get anything done if they just live in their own isolated world where they can't depend on (call out to) any other microservice?

Re: We're Leaving Kubernetes

#223
post #148

Earlier quoted context omitted.

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

I may be misunderstanding, but wouldn't you want the particular microservice you are working on independent enough to develop locally, then deploy into the remote environment to test the integration? (I don't work at this scale)

This is how you get to "I wrote to the spec, it's your problem that clicking the button doesn't do the thing". Huge feedback loops. When you run it "locally" enough you can do your integration stuff before you even ask for review.

Re: We're Leaving Kubernetes

#224

Earlier quoted context omitted.

From my perspective, installing Nix seems pretty invasive. I can understand if someone doesn't want to mess with their system "unnecessarily" especially if the tool and it's workings are foreign. And I can't really remember the last time I had issues with non-deterministic dependencies either. Dependency versions are locked. Maybe I'm missing something?

> installing Nix seems pretty invasive I'm writing a test to check whether a tool I'm writing can work without Nix (it works with it perfectly, but I want it to also work without it because there are a lot of folks like you, and like me about 3 years ago, who still think they'd rather struggle with manually installing the right glibc that goes with the right python dependency installed with the right pip and venv ver…

Nix on MacOS makes a bunch of build users and has a bad habit of touching files that then get touched by MacOS updates, leading to a system that works... then doesn't work.

Re: We're Leaving Kubernetes

#225

Earlier quoted context omitted.

Invariably this is an ideal and does not match up in reality. I work at ~50 ish employee company and we have layers of dependencies between at least 6 or 7 various microservices. I can see this adding up in complexity as the product scales

So basically a distributed monolith :P

I'm sure companies with well designed and properly isolated services exist but... in my time spent at several companies, "microservices" invariably degenerate to distributed monoliths.

Re: We're Leaving Kubernetes

#226

Earlier quoted context omitted.

Invariably this is an ideal and does not match up in reality. I work at ~50 ish employee company and we have layers of dependencies between at least 6 or 7 various microservices. I can see this adding up in complexity as the product scales

> Invariably this is an ideal and does not match up in reality. No, this does indeed match reality. At least for those who work with microservices. This is microservices 101. It's baffling how this is even being argued. We have industry behemoths building their whole development experience around this fact. Look at Microsoft. They even went to the extents of supporting Connected Services in Visual Studio 2022. Why on…

You can't compare Microsoft to your run-of-the-mill small (or even large) software shop, though. Maybe on HN, most people work on these amazingly designed systems, but in my experience most tech out there is shit and has no proper design or architecture beyond "we're doing microservices because everyone is".

Re: We're Leaving Kubernetes

#227

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

In my last role as a director of engineering at a startup, I found that a project `flake.nix` file (coupled with simply asking people to use https://determinate.systems/posts/determinate-nix-installer/ to install Nix) led to the fastest "new-hire-to-able-to-contribute" time of anything I've seen. Unfortunately, after a few hires (hand-picked by me), this is what happened: 1) People didn't want to learn Nix, neither d…

I trialed for a job where the CTO was convinced of dev environments in kube as "the way to work". Everyone else was at least ambivalent. I joined, tried to make some changes that would let me run things locally. Every time I got pushback about using the dev environments instead.

It took me a couple of days to get a supervisor-based setup working locally. I was the only person on the team who would run the backend and frontend when trying things out, because nobody was actually using the dev environments fully anyways. There was no buy-in for the dev environment!

I really feel like if you are in a position to determine tooling, it's so much more helpful to lean into whatever people on the ground want to use. Obviously there are times when the people on the ground don't care, but if you're spending your sweat and tears to put the square peg into the square hole suddenly you're the person with superpowers, and not the person pushing their pet project.

And sometimes that's just "wrap my thing with your thing".

Re: We're Leaving Kubernetes

#228
post #76

Earlier quoted context omitted.

Most teams/products I have been involved in, the stack always grows to the point that a dev can no longer test it on their own machine, regardless of how big the machine is. And having a different development machine than production leads to completely predictable and unavoidable problems. Devs need to create the software tooling to make remote dev less painful. I mean, they're devs... making software is kind of thei…

We have a project which spawns around 80 Docker containers and runs pretty OK on a 5 year old Dell laptop with 16GB RAM. The fans run crazy and the laptop is always very hot but I haven't noticed considerable lags, even with IntelliJ running. Most services are written in Go though and are pretty lightweight.

> Most services are written in Go though and are pretty lightweight

That's probably the difference. Throw elasticsearch, kafka and a bunch of Java services in and you'll be easily exhausting your RAM (at least at startup).

Re: We're Leaving Kubernetes

#229
post #204

Earlier quoted context omitted.

> So the solution here is to not have that kind of "stack". Reminds me of my favorites debugging technique. It's super fast: Don't write any bugs!

What a boring trite reply. All of this is analogous to badly written spaghetti code. And yes, you can absolutely avoid all of this if you know what you're doing.

Not trite at all. Bad code and bad architectures are a reality. You can fix them in theory, but that takes a lot of time and needs to be done incrementally. In the meantime, you have to live with the problem at hand.

Re: We're Leaving Kubernetes

#230

Sounds more to me like they need a new CTO. And that they're desperate to tell customers that they've fixed their problems. Kubernetes is absolutely the wrong tool for this use case, and I argue that this should be obvious to someone in a CTO-level position, or their immediate advisors. Kubernetes excels as a microservices platform, running reasonably trustworthy workloads. The key features of Kubernetes are rollout…

I consider Kubernetes to be an excellent framework to build these kinds of applications. The difference here is Gitpod being stateful, which is notoriously hard on Kubernetes, though easier now than ever before! To clarify on one of your points, Kubernetes itself has nothing to do with actually setting the security boundaries. It only providers a schema to describe resources and policies, and then an underlying syste…

You make an excellent point, and it emphasizes the need to distinguish between a typical Kubernetes setup (containers, pod/service mesh, and so on), and what Kubernetes can do in the abstract. In the extreme, the API server is just an HTTP interface for a KV store with a bit of RBAC and validation-mutation extensions.

What Neon is doing is quite a feat: Live migration (of a VM) while preserving TCP connections. It also took a lot of customization to achieve that.

But I agree that Kubernetes can indeed be used this way.

If anything, it further cements my original point about the Gitpod leadership.

The problem was never Kubernetes, but the dimwitted notion of using containers.

And then blaming Kubernetes for it: We're leaving you.

Post reply on HN