Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We were wrong about GPUs

fly.io

Post reply on HN
Print view View original

411–420 of 604 posts

Re: We were wrong about GPUs

#411

Post by remram »

spratzt wrote: post #351
remram wrote: post #200

Earlier quoted context omitted.

I keep seeing this opinion and I don't understand it. For various reasons, I recently transitioned from a dev role to running a 60+ node, 14+ PB bare metal cluster. 3 years in, and the only thing ever giving me trouble is Ceph. Kubernetes is etcd, apiserver, and controllers. That's exactly as many components as your average MVC app. The control-loop thing is interesting, and there are a few "kinds" of resources to ge…

Genuinely curious about what sort of business stores and processes 14 PB on a 60 node cluster.

Research institution.

The department saw more need for storage than Kubernetes compute so that's what we're growing. Nowadays you can get storage machines with 1 PB in them.

Re: We were wrong about GPUs

#412

Post by remram »

ffsm8 wrote: post #266
remram wrote: post #252

Earlier quoted context omitted.

And system calls and filesystems and sockets and LVM and... Sure at some point there are too many layers to count but I wouldn't say any of this is "Kubernetes". What people tend to be hung about is the difficulty of Kubernetes compared to `docker run` or `docker compose up`. That is what I am surprised about. I never had any issue with kubelet, or kube-proxy, or CSI plugins, or CNI plugins. That is after years of ru…

But you just said that you had issues with ceph? How is that not a CSI problem? And CNI problems are extremely normal. Pretty much anyone that didn't just use weavenet and called it a day has had to spend quiet a bit of time to figure it out. If you already know networking by heart it's obviously going to be easier, but few devs do.

Never had a problem with the CSI plugin, I had problems with the Ceph cluster itself. No, I wouldn't call Ceph part of Kubernetes.

You definitely can run Kubernetes without running Ceph or any storage system, and you already rely on a distributed storage system if you use the cloud whether you use Kubernetes or not. So I wouldn't count this as added complexity from Kubernetes.

Re: We were wrong about GPUs

#413

Post by ripped_britches »

freedomben wrote: post #66

> The biggest problem: developers don’t want GPUs. They don’t even want AI/ML models. They want LLMs. System engineers may have smart, fussy opinions on how to get their models loaded with CUDA, and what the best GPU is. But software developers don’t care about any of that. When a software developer shipping an app comes looking for a way for their app to deliver prompts to an LLM, you can’t just give them a GPU. I'm…

The way I think about it is this: any individual engineer (or any individual team) has a limited complexity budget (in other words, how much can you fit in your meat brain). How you spend it is a strategic decision. Depending on your project, you may not want to waste it on infra so you can fit a lot of business logic complexity.

Re: We were wrong about GPUs

#414

Post by remram »

cuu508 wrote: post #299
freedomben wrote: post #269

Earlier quoted context omitted.

Very fair, although with managed services which are increasingly available, you don't typically need to think about CSI or CNI.

Hence > Kubernetes is not the first thing that comes to mind when I think of "understanding where their code is running and what it's doing"...

CSI and CNI do about as much magic as `docker volume` and `docker network`.

People act like their web framework and SQL connection pooler and stuff are so simple, while Kubernetes is complex and totally inscrutable for mortals, and I don't get it. It has a couple of moving parts, but it is probably simpler overall than SystemD.

Re: We were wrong about GPUs

#415

Post by chromanoid »

sgarland wrote: post #184
SkyPuncher wrote: post #94

Earlier quoted context omitted.

I enjoy the details, but I don’t get paid to tell my executives how we’re running things. I get paid to ship customer facing value. Particularly at startups, it’s almost always more cost effective to hit that “scale up” button from our hosting provider than do any sort of actual system engineering. Eventually, someone goes “hey we could save $$$$ by doing XYZ” so we send someone on a systems engineering journey for a…

Wild idea: maybe if more devs had good fundamental knowledge to begin with, the good systems engineering could be done along the way.

Premature optimization may hit them hard. Overengineering is imo usually the bigger technical debt and a huge upfront cost as well. Well-thought out plans tend to become a sunken cost fallacy. Making room for changes is hard enough in XP like ways of working. When you have to tell your manager that half a year of careful plans and engineering can be thrown away, because of the new requirements, which emerge from late entry to market, you look like a clown. Plans and complexity usually introduce more risk than less.

Re: We were wrong about GPUs

#416

Post by pier25 »

"We started this company building a Javascript runtime for edge computing."

Wait... what?

I've been a Fly customer for years and it's the first time I hear about this.

Re: We were wrong about GPUs

#417

Post by dahart »

lifeisstillgood wrote: post #83

I have a timeline that I am still trying to work through but it goes like this : 2012 - moores law basically ends - nand gates do t get smaller just more cleverly wrapped. Single threaded execution more or less stops at 2 GHz and has remained there. 2012-2022 - no one notices single threaded is stalled because everything moves to VMs in the cloud - the excess parallel compute from each generation is just shared out i…

I’m surprised nobody has yet (as of this writing) pointed out that Moore’s Law never claimed anything about single threaded execution or clock rates. Moore’s Law is that the number of transistors doubles every two years, and that trend has continued since 2012.

It looks like maybe the slope changed slightly starting around 2006, but it’s funny because this comment ends complaining that Moore’s Law is too good after claiming it’s dead. Yes, software needs to deal with the transistor count. Yes, parallel architectures fit Moore’s law. The need to go to more parallel and more parallel because of Moore’s Law was predicted, even before 2006. It was a talking point in my undergrad classes in the 90s.

https://upload.wikimedia.org/wikipedia/commons/0/00/Moore%27...

Re: We were wrong about GPUs

#418

Post by trashtester »

apitman wrote: post #404
dzikimarian wrote: post #398

Earlier quoted context omitted.

One may think Kubernetes is complex (I agree), but I haven't seen alternative that simultaneously allows to: * Host hundreds or thousands of interacting containers across multiple teams in sane manner * Let's you manage and understand how is it done in the full extent. Of course there are tons of organizations that can (and should) easily resign from one of these, but if you need both, there isn't better choice right…

But how many orgs need that scale?

A very small percentage of orgs, a not-as-small percentage of developers, and at the higher end of the value scale, the percentage is not small at all.

Re: We were wrong about GPUs

#419

Post by remram »

allarm wrote: post #352
motorest wrote: post #317

Earlier quoted context omitted.

> …and containerd and csi plugins and kubelet and cni plugins (...) Do you understand you're referring to optional components and add-ons? > and kubectl You mean the command line interface that you optionally use if you choose to do so? > and kube-proxy and ingresses and load balancers… Do you understand you're referring to whole classes of applications you run on top of Kubernetes? I get it that you're trying to mak…

How’s kubelet and cni are “optional components”? What do you mean by that?

CNI is optional, you can have workloads bind ports on the host rather than use an overlay network (though CNI plugins and kube-proxy are extremely simple and reliable in my experience, they use VXLAN and iptables which are built into the kernel and that you already use in any organization who might run a cluster, or the basic building blocks of your cloud provider).

CSI is optional, you can just not use persistent storage (use the S3 API or whatever) or declare persistentvolumes that are bound to a single or group of machines (shared NFS mount or whatever).

I don't know how GP thinks you could run without the other bits though. You do need kubelet and a container runtime.

Re: We were wrong about GPUs

#420

Post by dartos »

trashtester wrote: post #418
apitman wrote: post #404

Earlier quoted context omitted.

But how many orgs need that scale?

A very small percentage of orgs, a not-as-small percentage of developers, and at the higher end of the value scale, the percentage is not small at all.

I think the developers who care about knowing how their code works tend to not want hyperscale setups anyway.

If they understood their system, odds are they’d realize that horizontal scaling with few, larger services is plenty scalable.

At those large orgs, the individual developer doesn’t matter at all and the EMs will opt for faster release cycles and rely on internal platform teams to manage k8s and things like it.

Post reply on HN