Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If this is true, the hyperscalers are toast

klementoninvesting.substack.com

Post reply on HN
Print view View original

51–60 of 108 posts

Re: If this is true, the hyperscalers are toast

#51

Post by js8 »

amelius wrote: post #26
philipallstar wrote: post #22

This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.

The logic seems mad to me because SLMs can simply not hold as much information as an LLM. Maybe if you combine an SLM with a database (as a tool) then it could work, but someone should first prove that.

The cost you pay is in additional reasoning the SLMs have to do. As I write elsewhere, LLM "remembers" that "Socrates is mortal", or other commonly useful deduction. SLM might need to derive it first by reasoning from the DB, which slows it down. (Or worse, it might miss the correct reasoning because it's just too much side quests to follow.) But the advantage is flexibility.

Re: If this is true, the hyperscalers are toast

#52

Post by palata »

root-parent wrote: post #4
palata wrote: post #2

"If", sure. How many developers here don't see a difference between the latest LLMs and SLMs they can run on their own computer? I tried running a smaller model locally, and it's not usable for me. I know people like to "predict" things, so that if they happen they can then say "I am a visionary, I predicted it" and start their blog posts with "as I predicted long ago (because I am a visionary), ...". > The research…

>> I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock. The lack of logic and risk management on this statement, is so strong, I hope humans are all quickly substituted by LLMs. Lets just do it and be done with it...

> I hope humans are all quickly substituted by LLMs

Why don't you go talk to your LLM instead of commenting here, then?

Re: If this is true, the hyperscalers are toast

#53

Post by root-parent »

beepbooptheory wrote: post #46
kyleblarson wrote: post #41

Given how often the "experts" on CNBC and Bloomberg TV use the term hyperscalers, my approach is to completely disregard anything written or spoken by any person who uses the term.

What would be a better term?

It has to be the Hyperspenders

Re: If this is true, the hyperscalers are toast

#54

Post by rapind »

embedding-shape wrote: post #27
ch_sm wrote: post #17

Earlier quoted context omitted.

> I tried running a smaller model locally, and it's not usable for me. If you have the hardware, a MacBook Pro for Qwen 3.6 35B A3B and Gemma 4 26B A4B for example, they are absolutely usable, both in terms of speed and quality. Anecdotally, I can use Qwen for day-to-day coding tasks in TS and Go, without hickups.

I'm unable to find a local model that comes close to the effectiveness of GPT models in Codex, and I have 96GB of VRAM available and tried every local model under the sun so far. Neither of those you mention I'd say are good enough for day to day software engineering for me, but I'm also really strict about code quality and iterate on what outputs agents give me a lot before I'm happy. With local models, this iterati…

> although I will say that the speed I'm able to get locally makes it more fun that any of the remote models.

This is becoming increasingly important to me. Super smart max reasoning frontier is fine if I leave it running overnight on some prepared set of clearly defined tasks, but when I want to work with the LLM, throughput really matters, and I'll go with a dumber model to get there.

At some point though, it's fast enough and any speed gains beyond that just makes me the bottleneck.

I also am seeing the smaller models gaining big strides lately, closing the gap on frontier models (still a decent sized gap though). I don't even run the small models like Qwen 3.8 27B locally. I just try them out in the cloud to see how they are progressing, and I'm definitely able to be productive.

Re: If this is true, the hyperscalers are toast

#55

Post by mcphage »

throwthrowuknow wrote: post #18

From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth. Also, the average consumer is not going to be running a local model until they a…

> Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights?

Apple or Nvidia, presumably.

Re: If this is true, the hyperscalers are toast

#56

Post by pu_pe »

The paper underlying this blog post is fundamentally flawed because of benchmark ceilings. If we define only simple tasks like asking what is the capital of France, all models will converge to 100%, obviously. But as bigger models get more capable we want them to replace more and more complex tasks, in as short time as possible.

Then of course there is the economics of it. Do people prefer to spend $5000 upfront to get things done 5x slower, or would they rather pay $20 a month for that?

Re: If this is true, the hyperscalers are toast

#57

Post by js8 »

amelius wrote: post #36
fph wrote: post #31

Earlier quoted context omitted.

But do you really need a model that has the complete Duran Duran discography memorized and preloaded in RAM at all time?

That's a different question. Probably not. But: 1. Training a large model with lots of information, then stripping the "useless" information from that model to obtain a small model => nobody has shown this. 2. Training a small model, letting it use a database tool so it scores the same as a large model without database => nobody has shown this.

I partly agree. Theory (of computation) shows it must be possible, however nobody has produced the small model (well, depends who you ask, what is small, article disputes that) and the database yet.

To go very small (thousands of rules) so that the reasoner can be understood by humans and proven sound - might be computationally quite difficult.

Re: If this is true, the hyperscalers are toast

#58

Post by throwthrowuknow »

mcphage wrote: post #55
throwthrowuknow wrote: post #18

From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth. Also, the average consumer is not going to be running a local model until they a…

> Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? Apple or Nvidia, presumably.

Hardware yes, weights? lol

Re: If this is true, the hyperscalers are toast

#59

Post by spinningslate »

aslkalska wrote: post #42

I don't think they are toast, I mean they will be in some trouble because all of them have fallen victim to fomo and started building out with so much debt for capacity that may or may not be needed nor achieve the returns that they want. I think there's a future where "personal software" meaning highly custom apps generated by an agent is a thing that doesn't mean everything will become that, same for local LLMs but…

Exactly. Seems naive at best for an investment consultant to look narrowly at current model capability and not consider the broader market. For example:

1. The hyperscalers are in a positive reinforcement loop. Despite any suggestions to the contrary they keep getting bigger. And can, er, “influence” government policy/officials and anything else needed to keep it that way.

2. The frontier labs and their investors. Another self-fulfilling reinforcement loop. Witness the circular gymnastics among OAI/Anthropic, Microsoft/Amazon and Nvidia

3. Data. No-one believes that Zuckerberg and co are going to say “great, we can just run the models on devices we don’t own and stop the surveillance economy because, y’know, privacy matters and we really care about mental health”.

And then there’s data centre locations and “yeah but jobs” even though your power bills are going up, and “why run your own data centre Mrs CTO, let us do it for you and save all that capex and those pesky employees you need to do it”.

Don’t get me wrong: I’m rooting for local, open weight/source models. But “hey look they benchmark well” is an unhelpfully narrow basis to forecast the demise of central hyperscaler hosting.

Re: If this is true, the hyperscalers are toast

#60

Post by slfnflctd »

regularfry wrote: post #32
philipallstar wrote: post #22

This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.

The valuations of the hyperscalars won't sustain just being more efficient than something you can run locally. There's a market there, but it's for margin on a commodity. They're priced for oligopoly on unique, premium products.

Yes, it's all about the amount of money invested right now. Justifying that was always going to be tricky, and it still is.

The compute capacity, however, is here to stay. There will be a seemingly endless line of people queued up to buy it at pennies on the dollar if this whole thing blows up financially. And they will use it!

Post reply on HN