101–108 of 108 posts
This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.
The efficiency doesn't matter as long as it's good enough, because you already have the computer.
Earlier quoted context omitted.
Hardware yes, weights? lol
Why not? Apple would have you download an updated set of weights with OSX / iOS updates. Nvidia could bundle them with driver updates. I don't think either would struggle to get people to keep their weights up to date.
Earlier quoted context omitted.
Yes, it's all about the amount of money invested right now. Justifying that was always going to be tricky, and it still is. The compute capacity, however, is here to stay. There will be a seemingly endless line of people queued up to buy it at pennies on the dollar if this whole thing blows up financially. And they will use it!
> And they will use it! Just out of curiosity: Use it for what?
Earlier quoted context omitted.
Whatever it is suited for that they can get a decent margin on. Or, you know, hobbies. The demand is there and the market will find pricing that works.
Do you really think any hobbyist is going to run a 200 kW liquid-cooled rack?
Or they'd be part of the customer base for leasing compute from someone else who bought the rack.
Although there is such a thing as hobbyists with a ton of money who want to experiment with large models on enterprise grade hardware. Probably not a lot of those, though, of course.
I don't think they are toast, I mean they will be in some trouble because all of them have fallen victim to fomo and started building out with so much debt for capacity that may or may not be needed nor achieve the returns that they want. I think there's a future where "personal software" meaning highly custom apps generated by an agent is a thing that doesn't mean everything will become that, same for local LLMs but…
Exactly. Seems naive at best for an investment consultant to look narrowly at current model capability and not consider the broader market. For example: 1. The hyperscalers are in a positive reinforcement loop. Despite any suggestions to the contrary they keep getting bigger. And can, er, “influence” government policy/officials and anything else needed to keep it that way. 2. The frontier labs and their investors. An…
Earlier quoted context omitted.
Why not? Apple would have you download an updated set of weights with OSX / iOS updates. Nvidia could bundle them with driver updates. I don't think either would struggle to get people to keep their weights up to date.
I’m sure it would be fine for simple routine tasks but it wouldn’t compete with OpenAI and Anthropic models
Earlier quoted context omitted.
> Hyperscalers don't run computing at some multiple more efficient than on prem. I'd disagree here. I see two avenues for an efficiency multiple, albeit a single-digit multiple: * Client aggregation allows a hyperscaler to average out demand spikes from uncorrelated clients, reducing the peak:average demand ratio and allowing better budgeting of compute. * Dynamic batching allows typical requests to run in batches of…
A hyperscaler can pack compute and that is it. It doesn't get you some multiple higher in efficiency. And it is a net negative when oversubscribed, which due to MBAs ruining all the clouds is how they operate. Customers pay more for less when using the cloud. Go get a job a hyperscaler, they want to smoke what you are smoking. I am not talking about diurnal cloud workloads, I am talking about the native efficiencies…
- buy in bulk, for lower prices and access to better hardware through big contracts
- build in bulk (i.e. spread out software improvements over a lot of data centres and customers)
- offer additional services such as edge caching and multi-region data redundancy
That doesn't mean that hyperscalers don't also do things wrong, but it seems very odd to pretend there's nothing to them.
Earlier quoted context omitted.
A hyperscaler can pack compute and that is it. It doesn't get you some multiple higher in efficiency. And it is a net negative when oversubscribed, which due to MBAs ruining all the clouds is how they operate. Customers pay more for less when using the cloud. Go get a job a hyperscaler, they want to smoke what you are smoking. I am not talking about diurnal cloud workloads, I am talking about the native efficiencies…
Surely they can: - buy in bulk, for lower prices and access to better hardware through big contracts - build in bulk (i.e. spread out software improvements over a lot of data centres and customers) - offer additional services such as edge caching and multi-region data redundancy That doesn't mean that hyperscalers don't also do things wrong, but it seems very odd to pretend there's nothing to them.