Live data from Hacker News

The unbearable cheapness of open weight models

jamesoclaire.com

61–70 of 195 posts

Re: The unbearable cheapness of open weight models

#61

Earlier quoted context omitted.

If Anthropic announced AGI tomorrow, how much better would that model be than Fable 5? It's looking like the road to AGI is gradual and moat-less. Models seem capable of improving other models, and even without illegal distillations many are nipping at the heels of Anthropic.

Yeah, I think we're learning that we overestimated the relevance of recursive self-improvement in a singularity/intelligence takeoff scenario. We thought that once an AI could start improving itself, it would cause an exponential, self-reinforcing intelligence explosion. Turns out that scaling up compute is much more important and also limits the upper end of intelligence.

The bigger mistake is assuming it would be better at everything all at once.

Suppose it can do 80% of what the 20th percentile human can do. That's a huge advance and very useful, but it means there are still things it's not very good at. If any of those things is (or becomes) a bottleneck, you're not getting the hockey stick graph.

Re: The unbearable cheapness of open weight models

#63

The giants knew this was coming, and soon 95% of AI tasks will be able to be done by open models (coding, research, cowork style work). So why pay a premium? Why use them at all? This leaves the labs with two options: 1) push the frontier in a way only massive scale can, and cash in on it (mythos level cyber security, recursive training, frontier science work). There’s big money for never before possible capabilities…

You forgot

3. Try to get the government to "certify models" to cause regulatory capture which is what both Anthropic and OpenAI has been pushing. No certification no use in business.

Re: The unbearable cheapness of open weight models

#65
post #15

Earlier quoted context omitted.

I'm currently writing a blog post about data centres in orbit, and my current conclusion is that even though they can build one, they definitely can't put 1 million up there and would have better things to do if they could. AGI? Too loosely defined. They lack a lot of competences which humans recognise when we see them but find it hard to put into words; on the other hand what they can do they already do faster than…

Maybe you missed the part where starlink / orbiting datacenters don't really have to even make money as long as they partially fund rocket launch tests. Or maybe you don't take Elon seriously when he talks about Mars.

> Maybe you missed the part where starlink / orbiting datacenters don't really have to even make money as long as they partially fund rocket launch tests.

I am only dismissing the orbital data centres, I do see a future for Starlink. One with competition, but a future nonetheless.

I'm old enough to remember the dot.com bubble and "we lose money on each unit and make up for it in scale":

If they don't make sense, they don't help. Putting a single one in space, or even a handful, is physically possible! But even optimistic Alphabet researchers (and Alphabet owns more of SpaceX than the entire IPO) say this only makes sense at $200/kg, while early Starship launch costs while they sort out reusability be at best $400/kg and the researchers don't expect $200/kg until the mid-2030s even with a high launch rate:

  If the learning rate is sustained—which would require∼180 Starship launches/year—launch prices could fall to 
- section 2.4, https://arxiv.org/abs/2511.19468

At $200/kg, and using the payload estimates elsewhere in the paper (the learning rate is based on mass rather than launch count), they'd need to launch 370,000 tons (4.4 ibid); even at the "good enough" cost, $200/kg, they'd need to spend $200/kg * 3.7e8 kg = $7.4e10. That's a hell of an R&D spend for the next 10 years of a company whose lifetime revenue (not profit) is reportedly $4.6e10.

My current draft has a few thousand words of additional problems, plus a bunch of things which I mention only to say why they are not, and some more where I say the research has yet to be done.

> Or maybe you don't take Elon seriously when he talks about Mars.

Used to, not any more. Has been too slow with Starship even before the fact that iteration with hardware is necessarily slowed down by a 2-year gap between launch windows.

There's not even been any news about demonstration models of either Mars-rated or Starship-rated Sabatier processors, which would be an easy win and also win points for both environmentalism and energy independence viz. Iran/Hormuz.

Re: The unbearable cheapness of open weight models

#66
One of the purposes of open weight models is to create a moat. If there were no open models available, I think we'd see much more and better models coming from Europe by now. Right now, any startup wanting to build and sell a model needs to be substantially better than the open models, which has become increasingly difficult and expensive.

Re: The unbearable cheapness of open weight models

#67

Aren't these open models so cheap because they're (partially) chinese gov. sponsored, and because they're stealing and redistributing the IP that comes in?

Well I can't speak to the chinese gov part, but ALL the models are IP laundering systems. I'd rather IP get laundered into open source.

Re: The unbearable cheapness of open weight models

#68
post #9

Earlier quoted context omitted.

'Reach AGI', the same way SpaceX will put data centers in orbit. A pipe dream.

> will put data centers in orbit. A pipe dream. Cheap access to space was once a pipe dream. Reusable boosters were once a pipe dream. A new player beating Boeing to the ISS was once a pipe dream. LEO constellations were once a pipe dream. Launching thousands of satellites was once a pipe dream. You should know that a) they are already running "AI" chips on their current sats. and b) they are already producing kW of…

> You can watch Scott Manley's video on it, where he does some rough calculations and explains the overall architecture.

I'm currently writing a blog post, and there's one big thing everyone, including Scott Manley, missed.

Once I realised it, I wondered what took me so long to spot this issue.

Re: The unbearable cheapness of open weight models

#69
post #54

With cache hit rates being effectively free, harnesses like Reasonix have let me do a month of work for less than 2 dollars. It's not even the subsidies making it cheap, American providers like Digital Ocean or Cloudflare host the same model with similar pricing.

How does caching help here? How much repetition is there in queries?

Agent loops (particularly coding agents) have a huge amount of repetition, because the entire context is included in every model request. So long as it's at the start of the input and doesn't change, it will be able to hit the KV cache (assuming the model provider actually has the prefix in cache).

This only works because prompt caching is done by matching prefixes, not the entire input.

Re: The unbearable cheapness of open weight models

#70

Earlier quoted context omitted.

Google already owns the app layer, and hardware, and they are a frontier-level AI research firm. I don't see how Anthropic or OpenAI survives being eaten by DeepSeek et al from the bottom of the stack and Google from the top.

The only reason people use google apps is because they are cheap and reliable. The user experience is awful. Have you ever tried to find a document you had open yesterday in drive?

I used their enterprise chat the other week coz one of the clients used it

It is truly amazing how bad it is. Made me miss using MS Teams. No software should make anyone miss using MS Teams

Post reply on HN