Live data from Hacker News

We were wrong about GPUs

fly.io

521–530 of 604 posts

Re: We were wrong about GPUs

#521
post #367

Earlier quoted context omitted.

> Great opportunity for someone ballsy to write a book about kubernetes internals for the general engineering population. What would be the interest of it? Think about it: - kubernetes is an interface and not a specific implementation, - the bulk of the industry standardized on managed services, which means you actually have no idea what are the actual internals driving your services, - so you read up on the exact fu…

I don't really care about the standardized interface. I just want to know how you'd implement something that would load your services and dependencies from a config file, bind them altogether, distribute the load through several local VMs and make it still work if I kill the service or increase the load. In less than 1000 lines.

> I don't really care about the standardized interface.

Then you seem to be confused, because you're saying Kubernetes but what you're actually talking about is implementing a toy container orchestrator.

Re: We were wrong about GPUs

#522
post #417

I have a timeline that I am still trying to work through but it goes like this : 2012 - moores law basically ends - nand gates do t get smaller just more cleverly wrapped. Single threaded execution more or less stops at 2 GHz and has remained there. 2012-2022 - no one notices single threaded is stalled because everything moves to VMs in the cloud - the excess parallel compute from each generation is just shared out i…

I’m surprised nobody has yet (as of this writing) pointed out that Moore’s Law never claimed anything about single threaded execution or clock rates. Moore’s Law is that the number of transistors doubles every two years, and that trend has continued since 2012. It looks like maybe the slope changed slightly starting around 2006, but it’s funny because this comment ends complaining that Moore’s Law is too good after c…

Sadly, Moore’s law is often cited, but never read. It’s only 3 pages.

http://cva.stanford.edu/classes/cs99s/papers/moore-crammingm...

But to further needle, the law is

Re: We were wrong about GPUs

#523

Earlier quoted context omitted.

> There's an (increasingly small) group of software developers who don't like "magic" and want to understand where their code is running and what it's doing. These developers gravitate toward open source solutions like Kubernetes Kubernetes is not the first thing that comes to mind when I think of "understanding where their code is running and what it's doing"...

Based on the following I think they also meant _how_ the code is running: > The other group (increasingly large) just wants to `git push` and be done with it, and they're willing to spend a lot of (usually their employer's) money to have that experience. They don't want to have to understand DNS, linux, or anything else beyond whatever framework they are using. I'm a "full full-stack" developer because I understand w…

> I can attest that this type of thinking is becoming increasingly rare as our industry continues to specialize.

This (and a few similar upthread comments) sum the problem up really concisely and nicely: pervasive, cross-stack understanding of how things actually work and why A in layer 3 has a ripple effect on B in layer 9 has become increasingly rare, and those who do know it are the true unicorns in the modern world.

Big part of the problem is the lack of succession / continuity at the university level. I have been closely working with very bright, fresh graduates/interns (data science, AI/ML, software engineering – a wide selection of very different specialisations) in the last few years, and I have even hired a few of them due to being that good.

Talking to them has given me interesting insights into what and how universities teach today. My own conclusion is that the reputable universities teach very well, but what they teach to is highly compartmentalised and typically there is little to no intersection across areas of study (unless the prospective student hits the pot of luck and enrolls in elective studies that go across the areas of knowledge). For example, students who study game programming (yes, it is a thing) do not get taught the CPU architectures or low-level programming in assembly; they have no idea what a pointer is. Freshly graduated software engineers have no idea what a netmask is and how it helps in reading a routing table; they do not know what a route is, either.

So modern ways of teaching are one problem. The second (and I think a big one) is the problem that the computing hardware has become heavily commoditised and appliance-like, in general. Yes, there are a select few who still assemble their own racks of PC servers at home or tinker with Raspberry Pi and other trinkets, but it is no longer an en masse experience. Gone are the days when signing up with an ISP also required building your own network at home. This had an important side effect of acquiring the cross-stack knowledge, which can only be gained today by willingfully taking up a dedicated uni course.

With all of that disappearing into oblivion, the worrying question that I have is: who is going to support all this «low level» stuff in a matter of 20 years without a clear plan for the cross-stack knowledge to succeed the current (and the last?) generation of unicorns?

So those who are drumming up the flexibility of k8s and alike miss out on one important aspect: with the lack of cross-stack knowledge succession, k8s is a risk for any mid- to large-sized organisation due to being heavily reliant on the unicorns and rockstar DevOps engineers who are few and far between. It is much easier to palm the infrastructure off to a cloud platform where supporting it will become someone else's headache whenever there is a problem. But the cloud infrastructure usually just works.

Re: We were wrong about GPUs

#524

Earlier quoted context omitted.

Based on the following I think they also meant _how_ the code is running: > The other group (increasingly large) just wants to `git push` and be done with it, and they're willing to spend a lot of (usually their employer's) money to have that experience. They don't want to have to understand DNS, linux, or anything else beyond whatever framework they are using. I'm a "full full-stack" developer because I understand w…

> I can attest that this type of thinking is becoming increasingly rare as our industry continues to specialize. This (and a few similar upthread comments) sum the problem up really concisely and nicely: pervasive, cross-stack understanding of how things actually work and why A in layer 3 has a ripple effect on B in layer 9 has become increasingly rare, and those who do know it are the true unicorns in the modern wor…

> For example, students who study game programming (yes, it is a thing) do not get taught the CPU architectures or low-level programming in assembly; they have no idea what a pointer is. Freshly graduated software engineers have no idea what a netmask is and how it helps in reading a routing table; they do not know what a route is, either.

> So modern ways of teaching are one problem.

IME school is for academic discovery and learning theory. 90% of what I actually do on the job comes from self-directed learning. From what I gather this is the case for lots of other fields too. That being said I've now had multiple people tell me that they graduated with CS degrees without having to write anything except Python so now I'm starting to question what's actually being taught in modern CS curricula. How can one claim to have a B.Sc. in our field without understanding how a microprocessor works? If it's in deference to more practical coursework like software design and such then maybe it's a good thing...

Re: We were wrong about GPUs

#525

Earlier quoted context omitted.

> I can attest that this type of thinking is becoming increasingly rare as our industry continues to specialize. This (and a few similar upthread comments) sum the problem up really concisely and nicely: pervasive, cross-stack understanding of how things actually work and why A in layer 3 has a ripple effect on B in layer 9 has become increasingly rare, and those who do know it are the true unicorns in the modern wor…

> For example, students who study game programming (yes, it is a thing) do not get taught the CPU architectures or low-level programming in assembly; they have no idea what a pointer is. Freshly graduated software engineers have no idea what a netmask is and how it helps in reading a routing table; they do not know what a route is, either. > So modern ways of teaching are one problem. IME school is for academic disco…

> […] self-directed learning.

And this is whom I ended up hiring – young engineers with curious minds, who are willing to self-learn and are continuously engaged in the self-learning process. I also continuously suggest interesting, prospective, and relevant new things to take a look into, and they seem to be very happy to go away, pick the subject of study apart, and, if they find it useful, incorporate it into their daily work. We have also made a deal with each other that they can ask me absolutely any question, and I will explain and/or give them further directions of where to go next. So far, such an approach has worked very well – they get to learn arcane (it is arcane today, anyway) stuff from me, they get full autonomy, they learn how to make their own informed decisions, and I get a chance to share and disseminate the vasts of knowledge I have accumulated over the years.

> How can one claim to have a B.Sc. in our field without understanding how […]

Because of how universities are run today. A modern uni is a commercial enterprise, with its own CEO, COO, CO. They rely on revenue streams (a previously unheard-of concept for a university), they rely on financial forecasts, and, most important of all, they have to turn profits. So, a modern university is basically a slot machine – outcomes to yield depend entirely on how much cash one is willing to feed it. And, because of that, there is no incentive to teach across the areas of study as it does not yield higher profits or is a net negative.

Re: We were wrong about GPUs

#526

Earlier quoted context omitted.

I asked chatgpt the other day to explain to me Kubernetes. I still don't understand it. Can you share with me what clicked with you, or resources that helped you?

Controller in charge of a specific type of object watches a database table representing the object type. Database table represents the desired state of things. When entries to the table are CRUD-ed, that represents a change to the desired state of things. Controller interacts with the larger system to bring the state of things into alignment with the new desired state of things. "The larger system" is more controller…

Is there only a single controller ? What happens if goes down?

If multiple controllers, how do they coordinate ?

Re: We were wrong about GPUs

#527

Earlier quoted context omitted.

For $200/month, I can have 2 ALBs, 2 ECS services, 2 CloudWatch log groups, and 2 RDS instances on AWS (one each for dev and prod) and a GitHub Team account with enough included runner minutes to cover most deployments. A colo is going to be more hassle, and I'll have to monitor more things (like system upgrades and intrusion attempts). I'd also have to amortize parts and labor as part of the cost, which is going to…

A Postgres `db.m6g.large` (the cheapest non-burstable instance) runs $114/month for a single AZ, and that's not counting storage or bandwidth. A `db.t4g.medium` runs $47/month, again, not counting storage or bandwidth. An ALB that somehow only consumed a single LCU per month would run $22. The rest of the mentioned items will vary wildly depending on the application, but between those and bandwidth - not to mention G…

> A Postgres `db.m6g.large` (the cheapest non-burstable instance) runs $114/month for a single AZ, and that's not counting storage or bandwidth. A `db.t4g.medium` runs $47/month, again, not counting storage or bandwidth.

This is why numbers do not stack up in the calculations – the premise that the DB has to be provisioned is not the correct one to start off with.

The right way of cooking RDS in AWS is to go serverless from the start and configure the number of DCU's, e.g. 1 to N. That way it will be even cheaper than the originally quoted $200.

Generally speaking, there is absolutely no need for anything to be provisioned at a fixed compute capacity in AWS unless there is a very specific or an edge case that, likewise, warrants a provisioned instance of something.

Re: We were wrong about GPUs

#528
post #295

Earlier quoted context omitted.

It’s not a false dichotomy at all. You only have so many hours in a day. At a startup, it’s very unlikely (certainly not impossible!) that your differentiation will come from very cheap system orchestration - your time is likely better spent on building your product. Minimum knowledge is one thing; minimum time to apply it is another.

If you had to spend time / VC money learning all of this stuff before you could begin to apply it, I absolutely agree, it's a waste of time. That's not my point. My point is people (by people, I mean "someone interested in tech and is likely to pursue it as a career") can and should learn these things earlier in life such that it's trivial once they're in the workforce. I could go from servers sitting on the ground t…

Yeah this is what I meant as well. Though I'd also argue that on the job returning is essential too. Which should come through multiple avenues. Mentorship from seniors to junior as well as allowing for time to learn on the job. I can tell you from having been an aerospace engineer you'd be given this time. And from what I hear, in the old days you'd naturally get this time any time you hit compile.

There's a bunch of sayings from tradesmen that I think are relevant here. And it's usually said by people who take pride in their work and won't do shoddy craftsmanship

  - measure twice, cut once 
  - there's never time to do it right, but there's always time to do it twice
  - if you don't have time to do it right when will you have time to do it again?
I think the advantage these guys have is that when they do a shit job it's more noticeable. Not only to the builders but anyone else. Unfortunately we work with high abstractions but high skill is the main reason we get big bucks. Unfortunately I think this makes it harder for managers to differentiate high quality from shit. So they'd rather get shit fast than quality a tad slower because all they can differentiate is time. But they don't see the how this is so costly since everything has to be done at least thrice

Re: We were wrong about GPUs

#529
post #421

Earlier quoted context omitted.

You can run single-node k3s on a VM with 512MB of RAM and deploy your app with a hundred lines of JSON, and it inherits a ton of useful features that are managed in one place and can grow with your app if/as needed. These discussions always go in circles between Haters and Advocates: * H: "kubernetes [at planetary scale] is too complex" * A: "you can run it on a toaster and it's simpler to reason about than systemd +…

The sad thing is there probably is a toaster out there somewhere with 512MB of RAM.

It's not sad until it becomes self-aware.

Re: We were wrong about GPUs

#530

Earlier quoted context omitted.

> We can wish everyone were good at everything, or we can try to actually get things done. False dichotomy. There's no reason we can't have both. I want to be clear, there's no perfect code or a perfect understanding or any of that. But the complaint here about not knowing /enough/ fundamentals is valid. There is some threshold which we should recognize as a minimum. The disagreement is about where this threshold is,…

> there's no perfect code or a perfect understanding or any of that I'm unsure what those terms mean. What are qualities that perfect code or perfect understanding would have? Depending on your framing I may agree or disagree. Just to lob a softball, I'm sure there are/were people that have a perfect understanding of an older CPU architecture; or an entire system architecture's worth of perfect understanding that gav…

I'm making a not so subtle reference to "don't let perfection be the enemy of good". A saying often said to people who are saying things need to be better.

To answer your softball, no, I doubt there was anyone who understood everything except petty early on. But very few people understand the whole OS let alone do any specialized task like data analysis, HPC, programming languages, encryption, or anything else. But here's the thing, the extra knowledge never hurts. It almost always helps, but certain knowledge is more generally helpful than others. Especially if we're talking memory but things like caching, {S,M}I{S,M}D, some bash, and some assembly go A LONG way

Post reply on HN