Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We're Leaving Kubernetes

gitpod.io

Post reply on HN
Print view View original

101–110 of 348 posts

Re: We're Leaving Kubernetes

#101

Post by fastasucan »

zo1 wrote: post #91
haolez wrote: post #64

Earlier quoted context omitted.

Yeah, I know. But then you have to make sure that your IDE is using the correct environment, that the notebook is using the correct environment, that the debugger is using the correct environment. It's trivial to setup a venv, but sometimes it's just not worth it for me.

This is one of the main reasons I tell people not to use VSCode. The people most likely to use it are juniors and people new to python specifically, and they're the most likely to fall victim to 'but my "IDE" says it's running 3.8 with everything installed, but when I run it from my terminal it's a different python 3.8' I watched it last week. With 4 (I hope junior) Devs in a "pair programming" session that forced me…

Weird, in my experience vscode makes it very clear by making you explicitly choose a .venv when running or debugging.

When it comes to opening a new terminal, you would have the exact same problem by... running commands in a terminal, cant see how vscode related that is.

Re: We're Leaving Kubernetes

#102

Post by rahen »

Kubernetes works great for stateless workloads.

For anything stateful, monolithic, or that doesn't require autoscaling, I find LXC more appropriate:

- it can be clusterized (LXD/Incus), like K8S but unlike Compose

- it exposes some tooling to the data plane, especially a load balancer, like K8S

- it offers system instances with a complete distribution and a init system, like a VM but unlike a Docker container

- it can orchestrate both VMs (including Windows VMs) and LXC containers at the same time in the same cluster

- LXC containers have the same performance as Docker containers unlike a VM

- it uses a declarative syntax

- it can be used as a foundation layer for anything stateful or stateless, including the Kubernetes cluster

LXD/Incus sits somewhere between Docker Swarm and a vCenter cluster, which makes it one of the most versatile platform. Nomad is also a nice contender, it cannot orchestrate LXC containers but can autoscale a variety of workloads, including Java apps and qemu VMs.

Re: We're Leaving Kubernetes

#103

Post by brunoborges »

horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

> Personally - just let the developer own the machine they use for development.

I wonder if Microsoft's approach for Dev Box is the right one.

Re: We're Leaving Kubernetes

#104

Post by deepsun »

> SSD RAID 0

> A simpler version of this setup is to use a single SSD attached to the node. This approach provides lower IOPS and bandwidth, and still binds the data to individual nodes.

Are you sure SSD is that slow? NVMe devices are so fast that I hardly believe there's any need for RAID 0.

Re: We're Leaving Kubernetes

#105

Post by preommr »

lolinder wrote: post #3

> This is not a story of whether or not to use Kubernetes for production workloads that’s a whole separate conversation. As is the topic of how to build a comprehensive soup-to-nuts developer experience for shipping applications on Kubernetes. > This is the story of how (not) to build development environments in the cloud. I'd like to request that the comment thread not turn into a bunch of generic k8s complaints. Th…

Can someone clarify if they mean development environments, or if they're talking about a service that they sell that's related to development environments.

Because I don't understand most of the article if it's the former. How are things like performance are a concern for internal development environments? And why are so many things stateful - ideally there should be some kind of configuration/secret management solution so that deployments are consistent.

If it's the latter, then this is incredibly niche and maybe interesting, but unlikely to be applicable to anyone else.

Re: We're Leaving Kubernetes

#106

Post by to11mtm »

horsawlarway wrote: post #33

Personally - just let the developer own the machine they use for development. If you really need consistency for the environment - Let them own the machine, and then give them a stable base VM image, and pay for decent virtualization tooling that they run... on their own machine. I have seen several attempts to move dev environments to a remote host. They invariably suck. Yes - that means you need to pay for decent h…

laughs in "Here's a VDI with 2vCPUs and 32GB of RAM but the cluster is overloaded, also you get to budget which IDEs you have installed because you have only a couple hundred GB of storage for everything including what we install on the base image that you will never use"

Re: We're Leaving Kubernetes

#107

Post by abofh »

I feel like anyone who was building a CI solution to sell to others and chose kubernetes didn't really understand the problem.

You're running hot pods for crypto miners and against people who really want to see the rest of the code that box has ever seen. You should be isolating with something purpose built like firecracker, and do your own dispatch & shred for security.

Re: We're Leaving Kubernetes

#108

Post by sangnoir »

horsawlarway wrote: post #85
jt2190 wrote: post #78

Earlier quoted context omitted.

I’m not sure we should leap from: > I have seen several attempts to move dev environments to a remote host. They invariably suck. To “therefore they will always suck and have no benefits and nobody should ever use them ever”. Apologies for the hyperbole but I’m making a point that comments like these tend to shut down interesting explorations of the state of the art of remote computing and what the pros/cons are. Edi…

But they don't suck because of lack of effort - they suck because there are real physical constraints. Ex - even on a VERY good connection, RTT on the network is going to exceed your frame latency for a computer sitting in front of you (before we even get into the latency of the actual frame rendering of that remote computer). There's just not a solution for "make the light go faster". Then we get into the issues the…

> But they don't suck because of lack of effort - they suck because there are real physical constraints.

They do suck due to lack of effort or investment. FANG companies have remote dev experiences that are decent - or even great - because they invest obscene amounts into dev tooling.

There physical constraints on the flipside: especially for gigantic codebases or datasets that don't fit on dev laptops or have need lower latencies to other services in the DC.

Added bonus: smaller attack surface area for adversaries who want to gain access to your code.

Re: We're Leaving Kubernetes

#109

Post by geoctl »

abofh wrote: post #107

I feel like anyone who was building a CI solution to sell to others and chose kubernetes didn't really understand the problem. You're running hot pods for crypto miners and against people who really want to see the rest of the code that box has ever seen. You should be isolating with something purpose built like firecracker, and do your own dispatch & shred for security.

Firecracker is more comparable to container runtimes than to orchestrators such as K8s. You still need an orchestrator to schedule, manage and garbage-collect all your uVMs on top of your infrastructure exactly like you would do with containers via k8s. In other words, you will probably have to either use k8s or build your own k8s to run "supervisor" containers/processes that launch uVMs which in turn launch the customer dev containers.

Re: We're Leaving Kubernetes

#110

Post by abofh »

geoctl wrote: post #109
abofh wrote: post #107

I feel like anyone who was building a CI solution to sell to others and chose kubernetes didn't really understand the problem. You're running hot pods for crypto miners and against people who really want to see the rest of the code that box has ever seen. You should be isolating with something purpose built like firecracker, and do your own dispatch & shred for security.

Firecracker is more comparable to container runtimes than to orchestrators such as K8s. You still need an orchestrator to schedule, manage and garbage-collect all your uVMs on top of your infrastructure exactly like you would do with containers via k8s. In other words, you will probably have to either use k8s or build your own k8s to run "supervisor" containers/processes that launch uVMs which in turn launch the cust…

For sure, but that's the point - containers aren't really good for an adversarial CI solution. You can run that shit in house on kubernetes on a VM in a simulated VR if you want. But if you have adversarial builds, you have a) builds that may well need close to root, and b) customers who may well want to break your shit. Containers are not the right solution for that, VM's get you mostly there, and the right answer is burning bare metal instances with fire after every change-of-tenant - but nobody does that (anymore), because VM's are close enough and it's faster to zero out a virtual disk than a real one.

So if you started with kubernetes and fought the whole process of why it's not a great solution to the problem, I have to assume you didn't understand the problem. I :heart: kubernetes, its complexity pays my bills - but it's barely a good CI solution when you trust everyone involved, it's definitely not a good one where you're trying to be general-purpose to everyone with a makefile.

Post reply on HN