Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

ea.rna.nl

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#31

Post by tcp_handshaker »

And this is the reason, why the AI companies, are now preparing a bailout by the US government. We will be moving quickly from your 401K is their exit liquidity to US Treasuries are their exit liquidity...

Meeting next week at the White House by coincidence just before the SpaceX IPO. Message to investors will be dont worry the US has your back...

At which point the corruption is sooo big, that an Empire crumbles under its own stench?

"Trump to meet AI leaders to discuss US investment in their companies" - https://www.bbc.com/news/articles/c98r8r7dz5no

"Trump Officials Held Millions of Dollars of SpaceX Ahead of IPO" - https://finance.yahoo.com/markets/stocks/articles/trump-offi...

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#32

Post by brookst »

Retr0id wrote: post #22
amelius wrote: post #9

Earlier quoted context omitted.

Yes this is what I"m waiting for. I do hope these cards will not come with any kind of vendor locking though.

What kind of vendor locking would be possible?

IP-protected models manifested directly in silicon.

Everything we’re using now is the equivalent of building a GPU on an FPGA: the hardware is general purpose at one abstraction level, and that comes with inefficiency at the next layer up. Collapse the levels, gain efficiency at the cost of generality.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#33

Post by halJordan »

beering wrote: post #30

As usual, there’s no factual basis for the claims other than “I made it up” and author doesn’t seem to have technical experience with ML experience. A lot of weasel words doing all the heavy lifting here.

And yet the crazy thing is that this stuff is par for the course human-wise and yet HN continues to rise up with pitchforks in hand complaining that ai is going to replace whatever this is. Like we're losing something.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#34

Post by Retr0id »

brookst wrote: post #32
Retr0id wrote: post #22

Earlier quoted context omitted.

What kind of vendor locking would be possible?

IP-protected models manifested directly in silicon. Everything we’re using now is the equivalent of building a GPU on an FPGA: the hardware is general purpose at one abstraction level, and that comes with inefficiency at the next layer up. Collapse the levels, gain efficiency at the cost of generality.

The whole premise of what's being described here is to bake the weights into the silicon. That isn't what I'd describe as vendor lock-in, any more than I'd describe a CPU that can only execute ARM instructions as vendor locked.

To answer my own question, I bet they could figure out a way to still bill you per-token, if they wanted to.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#35

Post by verdverm »

topaz0 wrote: post #12
verdverm wrote: post #5

Earlier quoted context omitted.

I'm not sure I agree. Beyond the Anthropic world, we see 1. The GitHub Copilot pricing change that started a week ago already change buying decisions. 2. Small name open weight providers selling at what I assume, and hear through grape vine, is a profitable place. Claude is over priced for what you get, and if headline is true, expensive to run. I do wonder if their API pricing is profitable. That's the word on the s…

By "word on the street" do you mean "word that they are leaking to the press so they get favorable coverage, and based based on opaque and questionable accounting"?

The press is not the channel I hear things through. New media is fundamentally broken right now and you should not use it as a source for quality information.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#36

Post by jerf »

jodacola wrote: post #26
NotGMan wrote: post #2

If this is the true cost of AI then the future might be dedicated extension cards for computers that hardcode entire models + weights. Downside: you need to buy a new one for each model. Upside: insanely fast inference and zero subscription cost, only one time purchase cost. Once a certain open source model gets good enough this might become viable. Right now the landscape is still shifting too fast. State of the art…

I would shell out cash right now for something like Opus on silicon, like what Taalas [0] has built for Llama 3.1. Having lightning-speed, local inference of a super high-quality model would be incredible. If you haven't played with it, check out Taalas's demo [1]. Honestly, though - I have my doubts. Recurring revenue is just too nice to pass up; I'm sure AI companies wouldn't want me buying a dedicated Opus card an…

Recurring revenue does you no good if it is in fact a recurring loss because your subscription customers use up more than you're charging them.

Of course expecting the metaphorical Harvard Business School analysts to realize that is asking a lot. Subscriptions are Good and Goodness is Subscriptions, and like any other mass of people following trends the preconditions on when subscriptions are good for a business tend to get lost in the frenzy.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#37

Post by brookst »

Retr0id wrote: post #34
brookst wrote: post #32

Earlier quoted context omitted.

IP-protected models manifested directly in silicon. Everything we’re using now is the equivalent of building a GPU on an FPGA: the hardware is general purpose at one abstraction level, and that comes with inefficiency at the next layer up. Collapse the levels, gain efficiency at the cost of generality.

The whole premise of what's being described here is to bake the weights into the silicon. That isn't what I'd describe as vendor lock-in, any more than I'd describe a CPU that can only execute ARM instructions as vendor locked. To answer my own question, I bet they could figure out a way to still bill you per-token, if they wanted to.

Portability between x86 and ARM is not a form of vendor lock?

And of course they could bill per-token, same way cable PPV worked (the bits were already in your house). But the cost structure of weights in silicon means that competitors would be encouraged to compete on this per-token cost, as their marginal cost would be zero.

I don’t see that being a durable business model, but I guess the counter argument is it’s also similar to game consoles, where initial hardware is subsidized and the business model assumes ongoing payment for bits.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#38

Post by [deleted] »

verdverm wrote: post #5
jqpabc123 wrote: post #4

Hang on to your wallet, you ain't seen nothing yet. The true cost of AI won't be revealed until after a large portion of the customer base has become "hooked" on it.

I'm not sure I agree. Beyond the Anthropic world, we see 1. The GitHub Copilot pricing change that started a week ago already change buying decisions. 2. Small name open weight providers selling at what I assume, and hear through grape vine, is a profitable place. Claude is over priced for what you get, and if headline is true, expensive to run. I do wonder if their API pricing is profitable. That's the word on the s…

[deleted]

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#40

Post by huflungdung »

wongarsu wrote: post #14

The headline claim assumes that Anthropic is operating the API at cost, and losing massive amounts of money on subscriptions My own impression based on inference prices for deepseek or other "open" models in the 1T range (including providers like DeepInfra with no obvious reason to subsidize their API costs) is that Anthropic is offering subscriptions at cost (on average, power users are a bit more expensive, casual…

[dead]
Post reply on HN