141–150 of 154 posts
Earlier quoted context omitted.
The cost of a timeshare robot will come down first, so you'll be hiring a robot before you own one. But that might not take so long to occur as you imagine.
People have been talking about this for literally a hundred years at this point. It's not a thing and isn't gonna be one.
Earlier quoted context omitted.
Even if we agree with this take (and I do think it's a likely take that you're right on the long term), it doesn't change that it seems likely we're in a bubble, and it probably will pop. We see a similar paradigm with lots of revolutionary technology. The initial promise is high, people get very excited, lots of money pours in, and.... 15-30 years go by before we start seeing real impact across the economy at large.…
> So take robots: I can promise you that you won't see robotics in every household in the next decade (especially so if we exclude the current market of robot vacuums). People in 2016 were adamant that the current frontier LLM capabilities would not be achievable in the next decade. Modern home robots prototypes look awkward and mostly useless the same way GPT-2 was looking like a curious but mostly useless thing in…
I'll even bet < majority of the US
Earlier quoted context omitted.
> If the model consumes four times as many tokens to deliver a result, it’s not cheaper This is literally what the guy says in podcast. So the underlying point is not correct. He’s specifically talking about per task cost. There’s no “but actually they meant something else”. I don’t discount your point about overall cost increasing but that’s not relevant here. Let’s agree that the podcast is fundamentally wrong in t…
Here's the actual quote from the podcast you're presumably referring to. I guess everybody can make up their own mind about what they're saying and whether they are "fundamentally wrong." You gotta watch something, because harnesses have changed how you can look at the cost structure of these things. If you look at cost per million tokens, it might look lower, but if the model consumes four times as many tokens in or…
He literally says “but if it’s using twice as many tokens that’s not the same thing”. Why would he bring up tokens?
I genuinely don’t know how you can conclude that he still thinks overall price per fixed task reduced.
well if bubble pops, everyone will die EXCEPT google UNLESS openAI/etc actually succeeds in making AGI that never hallucinates and goes over the current LLM limitations as for google... well they own the web (+google has plenty of other revenue sources, so it can just pay out its AI survival) if you're a website owner, would you welcome chatgpt/etc's data-collection bots? but... as for google's bots... you need your…
Earlier quoted context omitted.
Here's the actual quote from the podcast you're presumably referring to. I guess everybody can make up their own mind about what they're saying and whether they are "fundamentally wrong." You gotta watch something, because harnesses have changed how you can look at the cost structure of these things. If you look at cost per million tokens, it might look lower, but if the model consumes four times as many tokens in or…
It clearly points to how the models use more tokens, thereby more price, to achieve same task. Even if API prices per token reduced. He literally says “but if it’s using twice as many tokens that’s not the same thing”. Why would he bring up tokens? I genuinely don’t know how you can conclude that he still thinks overall price per fixed task reduced.
If you continue reading, he explains it in the next sentence.
Earlier quoted context omitted.
It clearly points to how the models use more tokens, thereby more price, to achieve same task. Even if API prices per token reduced. He literally says “but if it’s using twice as many tokens that’s not the same thing”. Why would he bring up tokens? I genuinely don’t know how you can conclude that he still thinks overall price per fixed task reduced.
> Why would he bring up tokens? If you continue reading, he explains it in the next sentence.
Earlier quoted context omitted.
> Why would he bring up tokens? If you continue reading, he explains it in the next sentence.
I did. And he implies that cost per task increases. Otherwise there’s literally no reason to bring up tokens - that is an internal implementation detail.
I'm not sure if you're not aware of what he's referencing there, but the specific example he brings up is that Anthropic ostensibly kept pricing identical with new models, but changed the tokenizer, which made the model use more tokens for the same input and output text. So that's a case where, prima facie, the cost per task increased. Of course, this also depends on how verbose the model is and what harness you use, and so on.
Correct me if I'm wrong; I think you believe that he makes an argument like "OpenAI decreased API pricing, but this decrease was actually secretly an increase in cost." But he does not. He's saying that even though cost-per-token is going down overall, the actual cost of using LLMs is in many cases going up, because there isn't a direct causal relationship between cost-per-token and the total cost of using LLMs.
And he's entirely correct about that.
Earlier quoted context omitted.
Anthropic is almost purely a model company. They own close to no data centers. If models are becoming commodities, and the main bottleneck is actually serving them at scale, Anthropic does not appear particularly well positioned. If AWS had a ~60-80% margin for decades, I see no reason why inference can't have a ~60-80% margin for quite some time. The problem is, if costs continue to drop ~90% for the same level of q…
> If models are becoming commodities, and the main bottleneck is actually serving them at scale, Anthropic does not appear particularly well positioned. That has been my thesis ever since Dario went on about how difficult it is to forecast capacity on the Dwarkesh podcast right about the time Claude started having 9's comparable to GitHub's. Even as an outsider, seeing all the cloud providers bemoan their lack of cap…
To be fair, this mess up actually increases my respect for him, as he focused on making sure that they could continue to run the business under reasonable assumptions rather than YOLOing compute into existence like OpenAI.
Earlier quoted context omitted.
Now do Google
Advantages: Lowest inference cost due to scale, experience, advanced datacenters and custom chips. Huge customer base. AI automation will make it much easier to switch from AWS to GCP. Serious organization that doesn't let it's product unknowingly hack companies. Disadvantages: Top model is slightly behind the frontier for now. Bias against using their products because it's not cool/trendy.