Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We're Leaving Kubernetes

gitpod.io

Post reply on HN
Print view View original

111–120 of 348 posts

Re: We're Leaving Kubernetes

#111

Post by kalaksi »

pmarreck wrote: post #88
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

In my last role as a director of engineering at a startup, I found that a project `flake.nix` file (coupled with simply asking people to use https://determinate.systems/posts/determinate-nix-installer/ to install Nix) led to the fastest "new-hire-to-able-to-contribute" time of anything I've seen. Unfortunately, after a few hires (hand-picked by me), this is what happened: 1) People didn't want to learn Nix, neither d…

From my perspective, installing Nix seems pretty invasive. I can understand if someone doesn't want to mess with their system "unnecessarily" especially if the tool and it's workings are foreign. And I can't really remember the last time I had issues with non-deterministic dependencies either. Dependency versions are locked. Maybe I'm missing something?

Re: We're Leaving Kubernetes

#113

Post by ed_mercer »

concerndc1tizen wrote: post #81

Sounds more to me like they need a new CTO. And that they're desperate to tell customers that they've fixed their problems. Kubernetes is absolutely the wrong tool for this use case, and I argue that this should be obvious to someone in a CTO-level position, or their immediate advisors. Kubernetes excels as a microservices platform, running reasonably trustworthy workloads. The key features of Kubernetes are rollout…

Why would you say that performance is bad on public cloud infrastructure?

Re: We're Leaving Kubernetes

#114

Post by bamboozled »

pmarreck wrote: post #88
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

In my last role as a director of engineering at a startup, I found that a project `flake.nix` file (coupled with simply asking people to use https://determinate.systems/posts/determinate-nix-installer/ to install Nix) led to the fastest "new-hire-to-able-to-contribute" time of anything I've seen. Unfortunately, after a few hires (hand-picked by me), this is what happened: 1) People didn't want to learn Nix, neither d…

Try Devbox, you can basically ignore nix entirely and reap all the benefits.

Re: We're Leaving Kubernetes

#115

Post by riiii »

> development environments

Kubernetes has never ever struck me as a good idea for a development environment. I'm surprised it took the author this long to figure out.

K8s can be a lifesaver for production, staging, testing, ... depending on your requirements and infrastructure.

Re: We're Leaving Kubernetes

#116

Post by trog »

lotharcable wrote: post #51
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

I strongly recommend just switching the Dev environment over to Linux and taking advantage of tools like "distrobox" and "toolbx". https://github.com/89luca89/distrobox https://containertoolbx.org/ It is sorta like Vagrant, but instead of using virtualbox virtual machines you use podman containers. This way you get to use OCI images for your "dev environment" that integrates directly into your desktop. https://podman…

If you're doing this, there are many cases where you might as well just spin up a decent Linux server and give your developers accounts on that? With some pretty basic setup everyone can just run their own stuff within their own user account.

You'll run into occasional issues (e.g. if everyone is trying to run default node.js on default port) but with some basic guardrails it feels like it should be OK?

I'm remembering back to when my old company ran a lot of PHP projects. Each user just had their own development environment and their own Apache vhost. They wrote their code and tested it in their own vhost. Then we'd merge to a single separate vhost for further testing.

I am trying to remember anything about what was painful about it but it all basically Just Worked. Everyone had remote access via VPN; the worst case scenario for them was they'd have to work from home with a bit of extra latency.

Re: We're Leaving Kubernetes

#118

Post by pjmlp »

hosh wrote: post #73
0xbadcafebee wrote: post #57

Earlier quoted context omitted.

Most teams/products I have been involved in, the stack always grows to the point that a dev can no longer test it on their own machine, regardless of how big the machine is. And having a different development machine than production leads to completely predictable and unavoidable problems. Devs need to create the software tooling to make remote dev less painful. I mean, they're devs... making software is kind of thei…

I have used remote dev machines just fine, but my workflow vastly differs from many of my coworkers: terminal-only spacemacs + tmux + mosh. I have a lot of CLI and TUI tools, and I do not use VScode at all. The main GUI app I run is a browser, and that runs locally. I have worked on developing VMs for other developers that rely on a local IDE such. The main sticking point is syncing and schlepping source code (someth…

Me as well, specially in the days that there was only a UNIX dev server for everyone.

It was never an issue to use X Windows on them, with hummingbird on my Windows thin client.

I guess a new generation has to learn the ways of timesharing development.

Re: We're Leaving Kubernetes

#119

Post by geoctl »

abofh wrote: post #110
geoctl wrote: post #109

Earlier quoted context omitted.

Firecracker is more comparable to container runtimes than to orchestrators such as K8s. You still need an orchestrator to schedule, manage and garbage-collect all your uVMs on top of your infrastructure exactly like you would do with containers via k8s. In other words, you will probably have to either use k8s or build your own k8s to run "supervisor" containers/processes that launch uVMs which in turn launch the cust…

For sure, but that's the point - containers aren't really good for an adversarial CI solution. You can run that shit in house on kubernetes on a VM in a simulated VR if you want. But if you have adversarial builds, you have a) builds that may well need close to root, and b) customers who may well want to break your shit. Containers are not the right solution for that, VM's get you mostly there, and the right answer i…

I would argue that dev containers are more complicated than CI even though they share many of the challenges (e.g. devcontainers might need to load 10s or 100s of GBs to start and are write heavy). I would also argue that userns/rootless containers provide "enough" isolation when it comes to isolating CPU/memory/networking as well as access to the host's syscalls if you're careful enough; however when it comes to storage (e.g. max disk size that a container can use and write to, max opened files, completely hiding the host's fs from the container's, etc...), it's unfortunately still extremely limited,fs-dependent for some features, even though modern solutions (e.g. vDPA and ublk) can be used to fix that and virtualize the storage for containers.

Re: We're Leaving Kubernetes

#120

Post by arp242 »

0xbadcafebee wrote: post #57
horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

Most teams/products I have been involved in, the stack always grows to the point that a dev can no longer test it on their own machine, regardless of how big the machine is. And having a different development machine than production leads to completely predictable and unavoidable problems. Devs need to create the software tooling to make remote dev less painful. I mean, they're devs... making software is kind of thei…

> the stack always grows to the point that a dev can no longer test it on their own machine

So the solution here is to not have that kind of "stack".

I mean, if it's all so big and complex that it can't be run on a laptop then you almost certainly got a lot of problems regardless. What typically happens is tons of interconnected services without clear abstractions or interfaces, and no one really understands this spaghetti mess, and people just keep piling crap on top of it.

This leads to all sorts of problems. Everywhere I've seen this happen they had real problems running stuff in production too, because it was a complex spaghetti mess. The abstracted "easy" dev-env (in whatever form that came) is then also incredibly complex, finicky, and brittle. Never mind running tests, which is typically even worse. It's not uncommon for it all to be broken for every other new person who joins because changes somewhere broke the setup steps which are only run for new people. Everyone else is afraid to do anything with their machine "because it now works'.

There are some exceptions where you really need a big beefy machine for a dev env and tests, maybe, but they're few and far between.

Post reply on HN