Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

241–250 of 319 posts

Re: LLMs are cheap

#241

Post by cush »

xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

> You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models).

Regardless of maturity lifecycle, by definition loss-leaders are cheap. If I go to the grocery store and milk is $1, I don't think I'm being swindled. I know it's a loss-leader and I buy it because it's cheap.

We are currently in the early-Netflix-massive-library-for-five-dollars-a-month era of LLMs and I'm here for it. Take all you can grab right now because prices will 100x over the next two years.

Re: LLMs are cheap

#242

Post by gpm »

zambal wrote: post #89
barrkel wrote: post #77

Earlier quoted context omitted.

Oh contraire, I ask questions about recent things all the time, because the LLM will do a web search and read the web page - multiple pages - for me, and summarize it all. 4o will always do a web search for a pointedly current question, give references in the reply that can be checked, and if it didn't, you can tell it to search. o3 meanwhile will do many searches and look at the thing from multiple angles.

But in that case it's hard to argue that llm's are cheap in comparison to search (the premise of the article)

It seems like it shifts it from "using an LLM instead of a search engine is cheaper" to "using an LLM to query the search engine represents only a marginal increase in cost", no?

Re: LLMs are cheap

#244

Post by nine_k »

bee_rider wrote: post #123
JimDabell wrote: post #101

Earlier quoted context omitted.

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

> Yes, capex not opex. The cost of running inference is opex. This seems sort of interesting, maybe (I don’t know business, though). I agree that the cost of running inference is part of the opex, but saying that doesn’t rule out putting other stuff in the opex bucket. Currently these LLM companies train and models on rented Azure nodes in an attempt to stay at the head of the pack, to be well positioned for when LLM…

> when LLMs become really useful

It looks to me similar to the situation with that newly fashionable WWW thing in, say, 1998. Everybody tried to use it, in search of some magic advantage.

Take a look at the WWW heavyweights today: say, Amazon, Google, Facebook, TikTok, WeChat. Are the web technologies essential for their success? Very much so. But TCP/IP + HTML + CSS + JS are mere tools that enable their real technical and business advantages: logistics and cloud computing, ad targeting, the social graph, content curation for virality, strong vertical integration with financial and social systems, and other such non-trivial things.

So let's wait until a killer idea emerges for which LLMs are a key enabler, but not the centerpiece. Making an LLM the centerpiece is the same thinking that was trying to make catchy domain names the centerpiece, leading to the dot com crash.

Re: LLMs are cheap

#245

Post by aswegs8 »

raydev wrote: post #233
jsnell wrote: post #208

Earlier quoted context omitted.

Their spending is not a problem. It's quite low for a top-tier hard tech company that's also running a consumer service with 500M active users. They are making a loss because 95% of their users are on free accounts, and for now they're choosing not to monetize those users in any way (e.g. ads).

sama tweeted that the $200 tier was priced too low to cover costs a few months ago.

How credible are his PR statements, though?

Re: LLMs are cheap

#246

Post by AndyKelley »

what's with all this unnecessary math?

Cursor is $20/month

Zed is $20/month

and they both have "burn mode"/"max mode" since you can hit your limit in a matter of hours.

LLMs are NOT cheap

they'll be cheap when they run on consumer hardware.

Re: LLMs are cheap

#247

Post by prats226 »

What feels missing is cheap wrt what. You have to analyze value created for a service and value captured by company. Eg if you use LLM's to do multiplication of two numbers, its doing billions of computations to get one computation incorrectly predicted majority of the time.

Re: LLMs are cheap

#248

Post by otterley »

lazide wrote: post #219
otterley wrote: post #138

Earlier quoted context omitted.

That’s a little like saying you can compute the profitability of the energy market by looking only at the margins of gas stations. You can’t exclude all the outlays on actually acquiring the product to sell.

Sure - but is there any doubt in that example that gas stations are making a profit? And unlike gasoline, once models are trained there is no significant ongoing production cost.

Models aren't static. In order for them to remain relevant, they have to be constantly retrained with new data. Plus there's a model arms race going on and which will probably continue for the foreseeable future.

Re: LLMs are cheap

#249

Post by everforward »

burnte wrote: post #236
JimDabell wrote: post #101

Earlier quoted context omitted.

> you also don't have any evidence that they are profitable. Sure we do. Go to AWS or any other hosting provider and pay them for inference. You think AWS are going to subsidise your usage of somebody else’s models indefinitely? > All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achieve profitability yet. Yes, capex not opex. T…

Capex and opex are just accounting labels that help categorize costs and can improve planning ability. But at the end of the day a billion dollars is a billion dollars.

They’re significant here because opex impacts profits while capex sort of doesn’t. They have a path to profitability if revenue > opex, by quitting growth and slashing capex.

Lots of hand waving, but that’s the idea.

Re: LLMs are cheap

#250

Post by withinboredom »

hedayet wrote: post #195
pama wrote: post #24

Earlier quoted context omitted.

Please read the DeepSeek analysis of their API service (linked in this article): they have 500% profit margin and they are cheaper than any of the US companies serving the same model. It is conceivable that the API service of OpenAI or Anthropic have much higher profit margins yet. (GPUs are generally much more cost effective and energy efficient than CPU if the solution maps to both architectures. Anthropic certainl…

That claim actually gives me pause. It reminds me of an idea from Zero to One by Peter Thiel - that real monopolies like to appear as a small fish in a very big pond, while tiny players try to appear as a monopoly. So when I see a company bragging about "500% profitability," I can’t help but wonder if they’re even profitable at all.

I imagine pretty much none of them are profitable in the real accounting sense. However, if they all turned off their free plans -- they'd be insanely profitable.
Post reply on HN