Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

ea.rna.nl

Post reply on HN
Print view View original

81–89 of 89 posts

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#81

Post by tim-tday »

LaurensBER wrote: post #17
janim007 wrote: post #13

That’s probably because only a handful of companies manufacture GPUs, and they’re still expensive. I think that will change over time as competition increases. LLMs are also still in a relatively early stage. We’re already seeing models become both smaller and more capable—for example, GPT-4 compared to Qwen3-30B, which can outperform GPT-4 in many tasks while using significantly less compute. So if this trend contin…

Not to mention the rise of Chinese chips. GPUs that can run everything from Crysis to CUDA are a harder engineering problem to solve than creating a chip that's optimized for inference. Not to mention that inference is an excellent first step towards a full, competitive GPU as well.

There are no Chinese gpus. The one thing they announced is equivalent to middle of the range of consumer cards four generations old and it’s not even available yet.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#83

Post by pjjpo »

Yizahi wrote: post #79
beering wrote: post #30

As usual, there’s no factual basis for the claims other than “I made it up” and author doesn’t seem to have technical experience with ML experience. A lot of weasel words doing all the heavy lifting here.

I'm sorry, were you speaking about this topic post or about posts authored by Sam/Amodei and the like?

Can you clarify what you are saying here?

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#84

Post by bleuarff »

throwaway894345 wrote: post #18
jqpabc123 wrote: post #4

Hang on to your wallet, you ain't seen nothing yet. The true cost of AI won't be revealed until after a large portion of the customer base has become "hooked" on it.

Some countervailing forces off the top of my head: * Hardware improvements will reduce costs * Model training improvements (read: more efficient model training) will reduce costs * Better models will reduce costs (more inference for less hardware time while keeping quality constant) * Tooling and platform will stabilize—less need to dump money into applications and backend systems because they will become mature—also…

What's your time estimation for the last 2 points? Last I heard TSMC is not willing to commit dozens of billions to build new fabs for what might be a fad. Granted, they're not theonly foundry, but that's a signal nonetheless. Given the current craziness around hardware, I doubt the stabilization will come soon. Probably not before token costs soar.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#85

Post by throwaway894345 »

bleuarff wrote: post #84
throwaway894345 wrote: post #18

Earlier quoted context omitted.

Some countervailing forces off the top of my head: * Hardware improvements will reduce costs * Model training improvements (read: more efficient model training) will reduce costs * Better models will reduce costs (more inference for less hardware time while keeping quality constant) * Tooling and platform will stabilize—less need to dump money into applications and backend systems because they will become mature—also…

What's your time estimation for the last 2 points? Last I heard TSMC is not willing to commit dozens of billions to build new fabs for what might be a fad. Granted, they're not theonly foundry, but that's a signal nonetheless. Given the current craziness around hardware, I doubt the stabilization will come soon. Probably not before token costs soar.

I'm far from an expert, but I'm pretty sure the chips bottleneck at the moment is memory chips which has little to do with TSMC and the memory industry is _dumping_ capital into increasing manufacturing capacity. I'm not sure when energy prices will stabilize.

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#86

Post by Yizahi »

pjjpo wrote: post #83
Yizahi wrote: post #79

Earlier quoted context omitted.

I'm sorry, were you speaking about this topic post or about posts authored by Sam/Amodei and the like?

Can you clarify what you are saying here?

Well, since all LLM companies are very secretive about their finances, the only public information from them about their revenues or profits or liabilities are as the top commenter said are without any "factual basis for the claims other than “I made it up”" and their CEOs are seemingly not very well versed in ML judging by their public statements and usually use a lot of weasel words themselves. So my comment was a joke, that it is rather hard to distinguish who is making stuff up - LLM corporations defenders or critics :) .

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#87

Post by throwaway63467 »

LaurensBER wrote: post #80
throwaway63467 wrote: post #42

Earlier quoted context omitted.

Which chips are these? It seems the main challenge is data bus speed and memory capacity now and it seems no one can really compete with NVIDIA now? And i doubt NVIDIA is still optimizing for anything expect LLMs/AI, their last keynote had less than two minutes of gaming related content and they even canceled their new generation of gaming cards for now. And it seems all of these advanced chips rely on the most advan…

I'm not nearly as knowledgeable about chips as I would like to be but I'm seeing lots of hype around the new Huawei Ascend 910C-Chips. These are in no way competitive with Nvidia for training but they're cheap for inference and seems to be winning market share inside China. China doesn't access to any of the latest chips technology but Huawei seems to have a roadmap to work around this by focussing on "3D chips" (ver…

Interesting, always found it would be cool to have this stuff integrated e.g. into server hardware for specialized AI services

Re: Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them

#89

Post by gctwnl »

LiamPowell wrote: post #54
wongarsu wrote: post #14

The headline claim assumes that Anthropic is operating the API at cost, and losing massive amounts of money on subscriptions My own impression based on inference prices for deepseek or other "open" models in the 1T range (including providers like DeepInfra with no obvious reason to subsidize their API costs) is that Anthropic is offering subscriptions at cost (on average, power users are a bit more expensive, casual…

The assumptions are so much worse than that: > Methodology & assumptions: No caching This is absolutely absurd. Claude code is of course using the cache (and this can be verified by looking at the traffic). It would be an incredibly stupid design to resend the whole input without a cache for every input, every tool use, etc..

By the way, there is caching (cache TTL is hort but during sessions it is active). E.g. Settings Status Config Usage Stats

  Session
  
  Total cost:            $15.59
  Total duration (API):  18m 3s
  Total duration (wall): 2h 13m 16s
  Total code changes:    232 lines added, 80 lines removed
  Usage by model:
       claude-opus-4-6:  1.5k input, 41.8k output, 26.4m cache read, 212.6k cache write ($15.59)
Post reply on HN