Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

youtube.com

Post reply on HN
Print view View original

21–30 of 61 posts

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#21

Post by ACCount37 »

CuriouslyC wrote: post #17

Models like Cursor's Composer 2.5 show that you can get real work done without the crazy costs just by focusing on a domain. AGI is silly in part because models are spiky, in addition to making the model more expensive for all queries, you can't easily tell a priori what the model will be good at. The smaller focused model is cheaper to run and if you try to ask a coding question to a biology/chemistry model (or vice…

"Focusing on a domain" has a hard ceiling.

A model's capability is a function of model size, and you can only push a small overspecialized "idiot savant" model so far before its crippling size starts to bite you.

You can make a model like Composer 2.5. But Mythos 5 will beat it on capability, both at coding and at everything else. And the world is always hungry for more capabilities.

If you're running high on agentic AI and low on human oversight, paying x2 for going from 5% faults to 2% faults is a good deal.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#22

Post by jassyr »

Frankly these tools should be priced per query. One doesn't need mythos to get a french toast recipe or add a few numbers together, but the flat rate subscription rate hides the inefficiency. Maybe a routing filter at the beginning of the query that chooses the right model for the query?

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#24

Post by mystraline »

The underlying capitalist problem is that dumping is not just permitted, but expected as a business strategy. Except dumping is usually exporting to another country to kill their industry.

Instead, this dumping is exporting "thinking" to destroy humans' innate thoughts, get them hooked, then rugpull for 3x the cost. Cause just over 1 year of LLMs, takes a developer who could reverse engineer a thing, to now needing help to construct a for loop.

Thats why I run my own LLMs. Hard to rugpull what you own and control. And thats also why I focus on questions not of "do this", but "explain this". I seek to use LLMs to learn more effectively, so I end up needing it less and less.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#25

Post by vb-8448 »

simonw wrote: post #10
dataflow wrote: post #4

Earlier quoted context omitted.

Are you including capex when you say "cost"? Or are you just looking at inference costs?

It doesn't make sense to include the capex cost to train a model in this kind of discussion, because that cost is fixed. Consider a model that costs $100m to train. If the vendor then prices it such that each inference token has a margin of 10% over the variable costs to serve (power + server costs), whether or not they cover their costs is based entirely on how many tokens they can sell. If they sell less than $1bn…

Cost is fixed if you train a model once in several years, if you have to train 3/4 times per year to stay competitive training cost is a thing.

You have to include also failed training sessions and experiments in the math.

There are no official figures but given how fast new models are rolled out, I wouldn't be surprised if neither Anthropic nor OAI manage to cover the full models cost.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#26

Post by ACCount37 »

frotaur wrote: post #13
simonw wrote: post #10

Earlier quoted context omitted.

It doesn't make sense to include the capex cost to train a model in this kind of discussion, because that cost is fixed. Consider a model that costs $100m to train. If the vendor then prices it such that each inference token has a margin of 10% over the variable costs to serve (power + server costs), whether or not they cover their costs is based entirely on how many tokens they can sell. If they sell less than $1bn…

I think the capex being fixed assumes you can just stop training the next model. But its not clear that you can afford to do that and keep selling tokens. And if capabilities plateau such that training the next one is useless, then the margins will drop fast due to competition.

Model inference:training compute for frontier models is estimated to be over 10:1 now.

Driven mostly by just how much inference they sell nowadays - but also by things like base model reuse.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#27

Post by Matticus_Rex »

dannersy wrote: post #9
mpeg wrote: post #5

This is true for a lot of subscriptions though: some people use 100% of their mobile plan data, some people use 10% – the price accounts for this.

Uhhh, at this order of magnitude? No way.

You don't know the actual margins -- you only know the API rate. If their API rate has huge margins and the average subscriber isn't coming anywhere close to their limits, the subscription can be very profitable. If they're only near peak capacity in peak working hours (when API traffic is most active) and subscription 5h limits help them redistribute a lot of use outside peak hours when they've got spare capacity, that alone could make a massive difference in profitability.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#28

Post by Matticus_Rex »

mystraline wrote: post #24

The underlying capitalist problem is that dumping is not just permitted, but expected as a business strategy. Except dumping is usually exporting to another country to kill their industry. Instead, this dumping is exporting "thinking" to destroy humans' innate thoughts, get them hooked, then rugpull for 3x the cost. Cause just over 1 year of LLMs, takes a developer who could reverse engineer a thing, to now needing h…

Generating huge consumer surpluses as a business strategy? Awesome if true.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#29

Post by smallmancontrov »

spwa4 wrote: post #18

If only every government had competition departments who had essentially ONE job: prevent companies from getting away with this ... Oh wait.

The Reagan / Bork "Consumer Welfare Standard" intentionally crippled anti-trust 40 years ago. By legislating robber-baron talking points, it succeeded in transforming the US business sector into what you see today: big moats, high profits, low competition.

The good news is that, after 40 years of Democrats not prioritizing opposition to the Reagan/Bork CWS, the issue is back on the ballots. Lina Khan picked up the torch that Louis Brandeis picked up a century ago (the arguments are identical, in this aspect time is a circle). Unfortunately, her team lost the last election, but just remember for the next one: this issue is now on the ballots.

Re: The $15,000 AI Bill. Your $20 Subscription is a DELUSION [video]

#30

Post by TrackerFF »

I'm not yet dependent enough on any AI to shell out anything more than $15-$25/month.

If I lose it, it will not be the end of the world. I'll probably start digging into local models.

I suspect there are many like me. Far more than there are totally dependent users. I also suspect that the AI economy is some sort of "whale economy", where a minority is footing the bill, by paying outrageous amounts to Anthropic/Open AI/Google.

Post reply on HN