Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We were wrong about GPUs

fly.io

Post reply on HN
Print view View original

581–590 of 604 posts

Re: We were wrong about GPUs

#581

Post by fmbb »

zsoltkacsandi wrote: post #499
fmbb wrote: post #442

Earlier quoted context omitted.

OK but the alternative if you think Kubernetes is too much magic when you want to operate hundreds of clusters with tens of thousands of nodes is? Some bash and Ansible and EC2? That is usually what Kubernetes haters suggest one does to simplify.

Hashicorp Nomad, Docker Swarm, Apache Mesos, AWS ECS? I love that the Kubernetes lovers tend to forget that Kubernetes is just one tool, and they believe that the only possible alternative to this coolness is that sweaty sysadmins writing bash scripts in a dark room.

I’m absolutely not a Kubernetes lover. Bash and Andible etc. is just a very common suggestion from haters.

I thought Mesos was kinda dead nowadays, good to hear it’s still kicking. Last time I used it it the networking was a bit annoying, not able to provide virtual network interfaces but only ports.

It seems like if you are going to operate these things, picking a solution with a huge community and in active development feels like the smart thing to do.

Nomad is very nice to use from a developer perspective, and it’s nice to hear infrastructure people preferring it. From outside the reason people pick Kubernetes seems to be the level of control of infra and security teams want over things like networking and disk.

Re: We were wrong about GPUs

#582

Post by lovich »

remram wrote: post #562
lovich wrote: post #537

Earlier quoted context omitted.

> So you're comparing Kubernetes to what? Not running services at all? Surprisingly there were hosted services on the internet prior to kubernetes existing. Hell, I even have reason to believe that the internet may possibly predate Docker

That is my point! If you think "just using SystemD services in a VM" is easy but "Kubernetes is hard", and you say "Kubernetes is hard" is because of Linux, cgroups, cloud storage, mount namespaces, ... Then I can't comprehend that argument, because those are things that exist in both solutions. Let's be clear on what we're comparing or we can't argue at all. Kubernetes is hard if you have never seen a computer befor…

ah I apologize for my snark then, I interpreted your sentence as _you_ believing that the only step simpler than using Kubernetes was to not have an application running

I see how you were asking the GP that question now

Re: We were wrong about GPUs

#583

Post by raymond_goo »

reilly3000 wrote: post #41

I shelled out for a 4090 when they came out thinking it would be the key factor for running local llms. It turns out that anything worth running takes way more than 24GB VRAM. I would have been better off with 2+ 3090s and a custom power supply. It’s a pity because I thought it would be a great solution for coding and a home assistant, but performance and quality isn’t there yet for small models (afaik). Perhaps DIGI…

no https://www.reddit.com/r/24gb/

Re: We were wrong about GPUs

#584

Post by amazingman »

DrFalkyn wrote: post #526
amazingman wrote: post #487

Earlier quoted context omitted.

Controller in charge of a specific type of object watches a database table representing the object type. Database table represents the desired state of things. When entries to the table are CRUD-ed, that represents a change to the desired state of things. Controller interacts with the larger system to bring the state of things into alignment with the new desired state of things. "The larger system" is more controller…

Is there only a single controller ? What happens if goes down? If multiple controllers, how do they coordinate ?

>Is there only a single controller ?

No, there are many controllers. Each is in charge of the object types it is in charge of.

>What happens if [it] goes down?

CRUD of the object types it manages have no effect until the controller returns to service.

>If multiple controllers, how do they coordinate ?

The database is the source of truth. If one controller needs to "coordinate" with another, it will CRUD entries of the object types those other controllers are responsible for. e.g. Deployments beget ReplicaSets beget Pods.

Re: We were wrong about GPUs

#585

Post by zsoltkacsandi »

fmbb wrote: post #581
zsoltkacsandi wrote: post #499

Earlier quoted context omitted.

Hashicorp Nomad, Docker Swarm, Apache Mesos, AWS ECS? I love that the Kubernetes lovers tend to forget that Kubernetes is just one tool, and they believe that the only possible alternative to this coolness is that sweaty sysadmins writing bash scripts in a dark room.

I’m absolutely not a Kubernetes lover. Bash and Andible etc. is just a very common suggestion from haters. I thought Mesos was kinda dead nowadays, good to hear it’s still kicking. Last time I used it it the networking was a bit annoying, not able to provide virtual network interfaces but only ports. It seems like if you are going to operate these things, picking a solution with a huge community and in active develop…

Can you describe who a Kubernetes hater is? Or show me an example. It's easy to stigmatise someone as a Kubernetes lover or hater. Then use it to invalidate their arguments.

I would argue against Kubernetes in particular situations, and even recommend Ansible in some cases, where it is a better fit in the given circumstances. Do you consider me as a Kubernetes hater?

Point is, Kubernetes is a great tool. In particular situations. Ansible is a great tool. In particular situations. Even bash is a great tool. In particular situations. But Kubernetes even could be the worst tool if you choose unwisely. And Kubernetes is not the ultimate infrastructure tool. There are alternatives, and there will be new ones.

Re: We were wrong about GPUs

#586

Post by david-gpu »

mandevil wrote: post #236
Uvix wrote: post #116

Earlier quoted context omitted.

The details matter because someone has to understand the details, and it's quicker and more cost-effective if it's the developer. At my job, a decade ago our developers understood how things worked, what was running on each server, where to look if there were problems, etc. Now the developers just put magic incantations given to them by the "DevOps team" into their config files. Most of them don't understand where th…

http://www.antipope.org/charlie/blog-static/2014/10/not-a-ma... This blog has a brilliant insight that I still remember more than a decade later: we live in a fantasy setting, not a Sci-fi one. Our modern computers are so unfathomable complex that they are demons, ancient magic that can be tamed and barely manipulated, but not engineered. Modern computing isn't Star Trek TNG, where Captain Picard and Geordi LaForge e…

> We live in a world where the simple cell phone in our pocket contains so much complexity that it is beyond any 10 human minds combined to fully understand how the hardware, the device drivers, the OS, the app layer, and the internet all interact between each other.

Try tens of thousands of people. A mobile phone is immensely more complicated than people realize.

Thank you for writing it so eloquently. I will steal it.

Re: We were wrong about GPUs

#587

Post by sgarland »

inkyoto wrote: post #577
sgarland wrote: post #566

Earlier quoted context omitted.

> The right way of cooking RDS in AWS is to go serverless from the start Nitpick, but there is no Serverless for RDS, only Aurora. The two are wildly different in their architecture and performance characteristics. Then there's RDS Multi-AZ Cluster, which is about as confusingly named as they could manage, but I digress. Let's take your stated Minimum ACU of 1 as an example. That gives you 2 GiB of RAM, with "CPU and…

Thanks for the correction, it is Aurora, indeed, although it can be found under the «RDS» console in AWS. Aurora is actually not a database but is a scalable storage layer that operates over the network and is decoupled from the query engine (compute). The architecture has been used to implement vastly different query engines on top of it (PgSQL, MySQL, DocumentDB – a MongoDB alternative, and Neptune – a property gra…

> Aurora is actually not a database but is a scalable storage layer that operates over the network and is decoupled from the query engine (compute).

They call it [0] a database engine, and go on to say "Aurora includes a high-performance storage subsystem.":

> "Amazon Aurora (Aurora) is a fully managed relational database engine that's compatible with MySQL and PostgreSQL."

To your point re: part of RDS, though, they do say that it's "part of RDS."

> The architecture has been used to implement vastly different query engines on top of it (PgSQL, MySQL, DocumentDB – a MongoDB alternative, and Neptune – a property graph database / triple store).

Do you have a source for this? That's new information to me.

> Aurora does not support RDS Proxy for PostgreSQL

Yes it does [1].

> I do not think it is correct to state that Aurora does not use the OS page cache – it does

It does not [2]:

> "Conversely, in Amazon Aurora PostgreSQL, the default value [for shared_buffers] is derived from the formula SUM(DBInstanceClassMemory/12038, -50003). This difference stems from the fact that Amazon Aurora PostgreSQL does not depend on the operating system for data caching." [emphasis mine]

Even without that explicit statement, you could infer it from the fact that the default value for `effective_cache_size` in Aurora Postgres is the same as that of `shared_buffers`, the formula given above.

> Switching to Aurora Serverless resulted in a 30% reduction in the monthly bill for the dev and uat environments right off the bat

Agreed, for lower-traffic clusters you can probably realize savings by doing this. However, it's also likely that for Dev/Stage/UAT environments, you could achieve the same or greater via an EventBridge rule that starts/stops the cluster such that it isn't running overnight (assuming the company doesn't have a globally distributed workforce).

What bothers me most about Aurora's pricing model is charging for I/O. And yes, I know they have an alternative pricing model that doesn't do so (but the baseline is of course higher); it's the principal of the thing. The amortized cost of wear to disks should be baked into the price. It would be difficult for a skilled DBA with plenty of Linux experience to accurately estimate how many I/O a given query might take. In a vacuum for a cold cache, it's not that bad: estimate or look up statistics for row sizes, determine if any predicates can use an index (and if so, the correlation of the column[s]), estimate index selectivity, if any, confirm expected disk block size vs. Postgres page size, and make an educated guess. If you add any concurrent queries that may be altering the tuples you're viewing, it's now much harder. If you then add a distributed storage layer, which I assume attempts to boxcar data blocks for transmission much like EBS does, it's nearly impossible. Now try doing that if you're a "cloud native" type who hasn't the faintest idea what blktrace [3] is.

[0]: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

[1]: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

[2]: https://aws.amazon.com/blogs/database/determining-the-optima...

[3]: https://linux.die.net/man/8/blktrace

Re: We were wrong about GPUs

#588

Post by godelski »

StefanBatory wrote: post #546
godelski wrote: post #531

Earlier quoted context omitted.

Are you wanting to argue or discuss? You can agree in part and disagree with another part. Doesn't have to be an argument. To your point I agree. I would argue that employers should be giving time for employees to better themselves. It's the nature of any job like this where innovation takes place. It's common among engineers, physicists, chemists, biologists, lawyers, pilots, and others to have time to learn. Doctor…

Sorry, discuss, not argue. I do agree with you, though. I have fears though on how much can that be a thing in reality - because I cannot disagree that this is a right approach.

Well here's the choice, we do that and build good things or we don't and build shit.

If you look around I think you'll notice it's mostly shit...

There's a flaw in markets though which allows shit to flourish. It's that before purchasing you can't tell the difference between products. So generally people then make the choice based on price. Of course, you get what you pay for. And many markets people are screaming for something different that isn't being currently met, but things are so entrenched that it's hard to even create that new market unless you're a huge player.

Here's a good example. Say you know your customers like fruit that is sweet. So all the farmers breed sweeter and sweeter strawberries. The customers are happy and sells go up. But at some point they don't want it any sweeter. But every farmer continues anyways and the customers have no choice but to buy too sweet strawberries. So strawberry sells decline. The farmers not having much signal from customers other than price and orders, what do they do? Well... they double down of course! It's what worked before.

The problem is that the people making decisions are so far removed from all this that they can't read the room. They don't know what the customer wants. Tbh, with tech, often the customer doesn't know what they want until they see it. (Which this is why so much innovation comes from open source because people are fixing things to make their own lives better and then a company goes "that's a good idea, let's scale this)

Re: We were wrong about GPUs

#589

Post by titmouse »

wolfgang42 wrote: post #571
titmouse wrote: post #395

Earlier quoted context omitted.

Could you elaborate? I may very well just be ignorant on the topic. I understand that if you write machine code and run it in your operating system, your operating system actually handles its execution (at least, I _think_ I understand that), but in what way does it have little to do with what the CPU is doing? For instance, couldn't you still run that same code on bare metal? Again, sorry if I'm misunderstanding som…

The quest for performance has turned modern CPUs into something that looks a lot more like a JITed bytecode interpreter rather than the straightforward “this opcode activates this bit of the hardware” model they once were. Things like branch prediction, speculative execution, out-of-order execution, hyperthreading, register renaming, L1/2/3 caches, µOp caches, TLB caches... all mean that even looking at the raw machi…

That makes sense, thank you!
Post reply on HN