Earlier quoted context omitted.
They're spending $10 million USD per engineer on LLMs?
Sorry, I meant per month, not per day.
40k per engineer per month? ~1.3k per day?
I love LLMs, I hate hype
191–200 of 340 posts
Re: I love LLMs, I hate hype
#192Earlier quoted context omitted.
If they haven't relaxed the classifier this makes little to no difference as the model is essentially unusable no matter the token quotas.
what are you working on? I only hit the guardrails twice after burning through two weeks of 20x max plan, both times on ML stuff; still more than I'd want to, but not unusable
I was downgraded to Opus after asking “how do fish get into mountain lakes”. Fable managed to get like half an answer through…
Re: I love LLMs, I hate hype
#193Earlier quoted context omitted.
They are raking in 10's of billions, growing revenues astronomically as we speak. That's not sign of commodity actor, just the opposite.
But, that's not what you said they'd do. I can switch to a different model with almost zero cost. That's the definition of a commodity.
This assumes the best models continue to be publicly available. There's some level of capability where it makes more sense to go in business for yourself.
Re: I love LLMs, I hate hype
#194Earlier quoted context omitted.
In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.
Apple Watch with 1TB of vram with the size of well.. a watch. Amazing story. If we make such leap in semiconductor field, it will be bigger than anything we have done till now. And all of that in 10years!
If it happens, that’ll be proof enough for me that LLM assisted science is giving economic returns!
Re: I love LLMs, I hate hype
#195Earlier quoted context omitted.
In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.
why would you need a local fable at that point? AGI will surely solve all the problems in the world at that point
I’m kidding but.. it’ll be great if in 10 years we look back on this and your joke has become a prophecy.
Re: I love LLMs, I hate hype
#196Earlier quoted context omitted.
In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.
You can’t seriously think an Apple Watch will have hundreds of gigs of vram in 10 years?
Maybe he thinks the model will be smaller.
Re: I love LLMs, I hate hype
#197This line: "this is my main argument against the valuation of frontier labs. It’s not that AI won’t create that much value, it’s that they won’t capture it." That is a very astute and concise way to explain everything about how the frontier labs are behaving and how they're trying to push more people to pay token rates for the best models. At the current subscription prices ($100 or $200 a month for a generous, thoug…
Yeah, I've started reaching for local models more. I'll use frontier models at the current cost for tasks that the local ones aren't great at, but when the rug pull inevitably comes, to me they're not worth 1000-2000 a month. And honestly, for my purposes I don't really need models to advance a lot. Like, I tried fable a couple of times and there just wasn't much there to justify its use to me. Opus did the same thin…
Yep, at those prices points I am most certainly just going to do whatever is in my power to run local. And to be honest, I think in the near future, we will get to a point where local models will be just good enough that it will not matter.
Re: I love LLMs, I hate hype
#198Earlier quoted context omitted.
No, you switched to another SOTA model. You didn't switch to 'Random Corner Store Token Seller' down the street, did you? There 2-3 top players, that is not commodity. Commodity is when there are enough that none of them have market power or can set prices. 'Commodity' means you buy your tokens from the Grocery Store on their loan plan. Like consumer credit is a commodity. FYI predict this is roughly the way it will…
> No, you switched to another SOTA model. You didn't switch to 'Random Corner Store Token Seller' down the street, did you? You're assuming that SOTA never hits a hard ceiling, letting local models catch up and achieve parity It seems unlike that the frontier labs are going to be keeping ahead forever, they'll hit some kind of ceiling eventually
It's a fair point about the 'ceilings' but I'm not quite making that assumption.
I think a few things are going to happen:
1) The Open Weights never really fully catch up, because there's too much Engineering and integration going now. It's way more than 'weights'
2) The Commodity Chinese models never quite catch up for the same reason every other product they make does not catch up - while they will shine in some areas, it won't land fully.
3) Horizontal integration, supply chains, availability, SLA, security, branding, regulatory requirements - all of this will add up to something competitively maintainable.
Can you name a product category that has truly hit a ceiling? Cars, computers, phones, airplanes ... always seems to be a way to nudge forward.
Good point though.
Re: I love LLMs, I hate hype
#199Earlier quoted context omitted.
The fact that you're even saying this it is probably an admission that you do think it's making you dumber. Most people I know, who are honest with themselves, have admitted to me that they feel like it's making them dumber or "zombifying" them. This is also well studied already, https://arxiv.org/abs/2506.08872 LLMs are poison for the brain, I'm almost certain of it, at least when used in the way most people are usi…
It's pretty easy to generalize this, but it doesn't match my perception. People who are using llms to do things they could have already done, but faster, probably have atrophying skill sets. People who are using these tools to accomplish significantly more difficult or complex work than they used to are absolutely finding new ways to push themselves. The problems are just much bigger. The average Joe can easily vibe…
This matches my experience as a statistician who used to begrudgingly write bad code when I had to. LLMs have opened up huge new possibilities for me.
No doubt there's some Gell-Mann amnesia going on, because I regularly have to correct them from doing stuff that's really dumb based on my expertise within my area of specialization. More than once I've managed to extract >3 orders of magnitude performance gains after asking them to justify why their code was so slow. Probably there's still some stupid stuff in there. But it's better than the code I would have written, and I never could have paid for a proper developer to write it.