Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We're Leaving Kubernetes

gitpod.io

Post reply on HN
Print view View original

331–340 of 348 posts

Re: We're Leaving Kubernetes

#331

Post by saghm »

pmarreck wrote: post #193
kstenerud wrote: post #152

Earlier quoted context omitted.

After my personal 2-year experiment with NixOS, I'd avoid anything Nix like the plague, and would be looking for a new job if anyone instituted a Nix-only policy. It's not the learning new things that's a problem, but rather the fact that every little issue turns into a 2-day marathon that's eventually solved with a 1-line fix. And that's because the feedback loop and general UX is just awful - I really started to fe…

Literally every Docker image you've ever downloaded comes from a build that only worked (essentially) "by accident." If Docker builds were as deterministic as Nix, then all that would need to be distributed would be Dockerfiles and perhaps a cache of base images somewhere. Looking at a build as a pure function where each dependency (including any compiler(s), plus the environment), are "input arguments" to it, was a…

"Working by accident" is still "working" though, so from the standpoint of the person using it, the reason doesn't matter. I'd also argue that the main point made above about docker wasn't that it's reliable, but that it gives developers more feedback when trying to work with it. For people who's goal isn't just to consume builds but write their own, existing Nix builds being deterministic does nothing to help them if trying to modify or borrow parts of them leads to issues they can't easily fix.

I think Nix fits a pattern that's happened in plenty of other domains where the technology that focused on doing things "right" failed to win out against a competitor doing things "wrong" but optimizing for a lower barrier of entry. The logic that a perfect solution is worth an up-front cost is compelling, since having an imperfect solution has a long-term cost that never goes away, but this misses the fact that pushing the cost until later has value of its own; making things easier today at the cost of tomorrow buys time to improve things before the cost is incurred.

At the risk of a convoluted metaphor, imagine that someone moves into a new house and calls two plumbers asks them to hook up the water in their bathroom. The first plumber says that they can get it done so the bathroom can be used today, but they'll need to come by again in a week or two since they might need to make additional adjustments. The second plumber says they've come to with a way to make sure that they never need to come back to make adjustments, but it will take them a full week to finish setting it up before anyone can use it.

For most people, it doesn't matter if the second plumber's solution will be better next week if they need to use the bathroom today, as long as the first plumber's solution can last long enough before it needs to be fixed.

Re: We're Leaving Kubernetes

#332

Post by eddyg »

belthesar wrote: post #121
rahen wrote: post #102

Kubernetes works great for stateless workloads. For anything stateful, monolithic, or that doesn't require autoscaling, I find LXC more appropriate: - it can be clusterized (LXD/Incus), like K8S but unlike Compose - it exposes some tooling to the data plane, especially a load balancer, like K8S - it offers system instances with a complete distribution and a init system, like a VM but unlike a Docker container - it ca…

I too am rallying quickly to the Incus way of doing things. Also of note, there's an effort to build a utility to write Compose manifests for Incus workloads that I'm following very closely. https://github.com/bketelsen/incus-compose

Thanks for pointing out `incus-compose`!

Re: We're Leaving Kubernetes

#333

Post by 0xCMP »

chipdart wrote: post #324
0xCMP wrote: post #254

Earlier quoted context omitted.

It's very likely not their decisions that lead to this, but their responsibility to improve velocity. Imagine the goal is to fix the problems (e.g. make it possible to run less of the services or something like that): How do you do that without first running all the services, making the proper changes, and then testing those changes? You need to be able to run all the services in that interim period. So, wouldn't it…

> How do you do that without first running all the services (...) Why do you need to run all services in isolation to be able to troubleshoot and isolate a problem?

I think it's a fair to assume OP would have tried to run only some of the services and has seen or experienced problems with doing that.

Re: We're Leaving Kubernetes

#334

Post by 0xCMP »

wlll wrote: post #256
0xCMP wrote: post #254

Earlier quoted context omitted.

It's very likely not their decisions that lead to this, but their responsibility to improve velocity. Imagine the goal is to fix the problems (e.g. make it possible to run less of the services or something like that): How do you do that without first running all the services, making the proper changes, and then testing those changes? You need to be able to run all the services in that interim period. So, wouldn't it…

> It's very likely not their decisions that lead to this Yeah, I get that, I was deliberate about the phrasing of "your company" rather than just "your". Obviously we don't know anything about the parent commenters company and situation, perhaps 12 people per microservice genuinely is the right solution for them, but it seems like it would be better not to get into this situation in the first place, though once there…

Agreed that it may be the right solution, but it smells bad. If I was there I would be trying to reduce the complexity where I could.

Re: We're Leaving Kubernetes

#335

Post by themanmaran »

consteval wrote: post #258
Cthulhu_ wrote: post #244

Earlier quoted context omitted.

If you have to have the latest version etc of every microservice, it's a distributed monolith.

Right, the crux here is bad architects are building distributed systems that are essentially monoliths. It's the worst of both worlds. You get none of the guarantees and visibility of a monolith, and you get all the friction of a distributed system.

This is the first time I've heard the term distributed monolith and it's just clicked that every microservice application I've worked on has been exactly that.

Re: We're Leaving Kubernetes

#336

Post by liveoneggs »

chipdart wrote: post #326
liveoneggs wrote: post #300

Earlier quoted context omitted.

This is a bad idea. It defeats a major feature of the jvm.

> This is a bad idea. Not really. It all depends on what are your needs. > It defeats a major feature of the jvm. You're confusing things. Just because Java addressed the deployability problem for Java applications before containerization was even a word, this does not mean that deploying a JVM per service is a bad idea. Just think about it for a second. Why do you need to deploy and scale services independently? Do…

We were talking about dev enviroments on desktops, not independently scalable components.

I did mention the jvm in my response. You quoted me doing so, in fact.

Re: We're Leaving Kubernetes

#337

Post by nixdev »

pxc wrote: post #311
sgarland wrote: post #135

Earlier quoted context omitted.

This is an unpopular take, but entirely true. Skilled at a programming language, other than maybe C, does not in any way translate to general skill with system administration, or even knowing how to correctly operate a computer. I once had to explain to a dev that their Mac was out of disk space because a. They had never removed dangling containers or old image versions b. They had never emptied the Trash.

Unpopular take? I thought it was common knowledge that computer operation and computer programming involve distinct (if overlapping) skillsets.

It's unpopular because developers want to have root access to their machines whilst being proudly ignorant to how fast the sensitive medical data or financial data they're working on can fly out of the machine.

Even when provided a means to instantiate virtual machines where they can have root access within the virtual machine, a lot of them will bitch.

Re: We're Leaving Kubernetes

#338

Post by alanwreath »

chipdart wrote: post #187
czhu12 wrote: post #148

Earlier quoted context omitted.

We tried this approach at a former company with ~600 engineers at the time. Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which…

> Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable. This is certainly one of the critical mistakes you did. No developer needs to launch half of the company's services to work on a local deployment. That's crazy, and awfully short-sighted. The only services a dev…

I’m sure there are legion ways to do this. For our company, it was Tilt + Telepresence. Locally, we ran our local service with Tilt. The service we ran would often need access to some other service(s), which would have been hard/impossible to bring up in conjunction with their own rabbit hole of dependencies. Thus, those dependency services were made available to the local k8s installation via Telepresence. That way, locally running services could access their dependent services as if they were running in the local environment (even though they were actually running in the cloud). In our case, security teams would only allow us stage environments, but that was enough for most everything.

Re: We're Leaving Kubernetes

#339

Post by trog »

tored wrote: post #220
trog wrote: post #116

Earlier quoted context omitted.

If you're doing this, there are many cases where you might as well just spin up a decent Linux server and give your developers accounts on that? With some pretty basic setup everyone can just run their own stuff within their own user account. You'll run into occasional issues (e.g. if everyone is trying to run default node.js on default port) but with some basic guardrails it feels like it should be OK? I'm rememberi…

The painful part of that setup is that all the tools you want to use on the source code must either run on the server itself, thus installed somehow, or some slow remote mounted filesystem, this severely limits the tools you may want to use.

What tools don't run on Linux? Modern tooling almost assumes Linux in most cases now. As a Windows user I feel like I hit this wall way more often than any other.

Re: We're Leaving Kubernetes

#340

Post by wutwutwat »

epgui wrote: post #147
wutwutwat wrote: post #143

Earlier quoted context omitted.

4th paragraph in if you read the article… > This is not a story of whether or not to use Kubernetes for production workloads that’s a whole separate conversation. As is the topic of how to build a comprehensive soup-to-nuts developer experience for shipping applications on Kubernetes. > This is the story of how (not) to build development environments in the cloud.

I'm not sure that this really answers their question.

> Can someone clarify if they mean development environments, or if they're talking about a service that they sell that's related to development environments.

their question isn't asking anything. It's both about development environments AND a service they sell, which is dev envs.

Even so

> This is the story of how (not) to build development environments in the cloud.

that is what the article is about. Their words. The person asked what the article is about, this it it, from the authors themselves.

read the damn article

Post reply on HN