Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

161–170 of 319 posts

Re: LLMs are cheap

#161

Post by username223 »

jsnell wrote: post #156
username223 wrote: post #141

Earlier quoted context omitted.

It's addressed poorly. > First, there's not that much motive to gain API market share with unsustainably cheap prices. Any gains would be temporary, since there's no long-term lock-in, What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. > Second, some of those models have been released with ope…

> What? If someone builds something on top of your API, they're tying themselves to it, and you can slowly raise prices while keeping each increase well below the switching cost. That's not really how the LLM API market works. The interfaces themselves are pretty trivial and have no real lock-in value, and there's plenty of adapters around anyway. (Often first-party, e.g. both Anthropic and Google provide OpenAI-comp…

> The market price of renting that compute on the market. That's fully loaded,

Sorry, I totally misread your post. Charging 80% on top of server rental isn't so bad, especially since I'm guessing there are significant markups on GPU rental given all the AI demand.

Re: LLMs are cheap

#162

Post by dietr1ch »

jstummbillig wrote: post #136
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

There is also a lot of different models at a lot of different price points (and LLMs are fairly hard to compare to begin with). In this theory of a likely loss-leader, must we assume that all of them, from all companies, are priced below cost...? If so, that seems like a fairly wild claim. What's Step 2 for all of these companies to get ahead of this, given how model development currently works? I think the far more…

I don't think it's that wild. Hardware will improve together with performance, but once the market stops expanding and behaviour gets stagnant the market shares will solidify, so you better aim to have a large portion to make the scale together with the improvements help reach profitability.

Re: LLMs are cheap

#163

Post by ac29 »

NewsaHackO wrote: post #81
andrew_lettuce wrote: post #37

Earlier quoted context omitted.

There are some big problems with this, mostly that openAI doesn't want to break even or be profitable, their entire setup is based on being wildly so. Building a Google sized business on ads is incredibly difficult. They need to be so much better than the competition that we have no choice but to use them, and that's not the case any more. More minor but still a major issue is the underlying IP rights. As users matur…

> mostly that openAI doesn't want to break even or be profitable, their entire setup is based on being wildly so. I’m sure you are going to provide some sort of evidence for this otherwise ridiculous claim, correct?

OpenAI is attempting to develop AGI. For most definitions of AGI, that would be a pretty wild success.

Theres another path where AI progress plateaus soon and OpenAI remains a profitably going concern of much more modest size, but that is not the goal.

Re: LLMs are cheap

#164

Post by JimDabell »

ceejayoz wrote: post #153
JimDabell wrote: post #152

Earlier quoted context omitted.

Most people don’t use an ad blocker. Even so, saying that the UX of the web is almost as good as the UX of an LLM after you take steps to work around the UX problems with the web isn’t really an argument.

> Most people don’t use an ad blocker. I mean, they should. Anyone on this site most certainly should. The LLM UX is going to rapidly converge with the search UX as soon as these companies run out of investor funds to burn. It's already starting; https://www.axios.com/2024/12/03/openai-ads-chatgpt . What then?

> I mean, they should.

Yes, they should. They don’t.

There’s really no point talking about how the web could have almost as good UX as LLMs if users did things that they do not do. Users are still getting shitty UX from the web.

> The LLM UX is going to rapidly converge with the search UX as soon as these companies run out of investor funds to burn.

The point of the article is that these companies can be profitable as-is. If chatbots screw up their UX, it’s not because they need it to survive.

And again, I’m judging based on what is actually the case today, not a speculative future.

I’m pointing out that LLMs have much better UX than the web. Repeatedly saying “but what if they didn’t?” to me is uninteresting.

Re: LLMs are cheap

#165

Post by otterley »

dist-epoch wrote: post #39
pmdr wrote: post #33

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable. They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't…

Stock price go up is another way a company is profitable. The amazon playbook for 10+ years.

Stock prices are (at least in theory, discounting speculation) a consequence of profits; they are not profits in and of themselves. Profits are at the bottom of the income statement.

Re: LLMs are cheap

#166

Post by tiagod »

10 years ago, we had nearly free ride-sharing and delivery. When a new company entered my market, I could usually get stuff cheaper through it than by walking to the shop they were picking it up from.

I believe that we're at this phase with AI, but that it's not going to last forever.

Re: LLMs are cheap

#167

Post by otterley »

andrew_lettuce wrote: post #51
dist-epoch wrote: post #39

Earlier quoted context omitted.

Stock price go up is another way a company is profitable. The amazon playbook for 10+ years.

Amazon made huge money as they captured more and more of the market and didn't return any of it. The company literally became worth more and more each year. Open AI continues to hemorrhage money.

Amazon hemorrhaged money for the first decade of its life. It was founded in 1994 and didn’t turn its first profit until 2004.

Re: LLMs are cheap

#168

Post by sdenton4 »

fkyoureadthedoc wrote: post #34
paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

Anecdotally, I'm a paying user and do a lot of super basic queries. What is this bug, rewrite this drivel into an email to my HOA, turn me into a gnome, what is the worst state and why is it west Virginia. This would probably increase 10x if one of the providers sold a family plan and my kids got paid access. Most of my heavy lifting is work related and goes through my employer's pockets.

Careful there: Once the machine turns you into a gnome, the price to turn back is quite hefty. A friend of mine gave up an eye, I only lost my most cherished memory. And most people ask the wrong question entirely and are never heard from again.

Re: LLMs are cheap

#169

Post by raincole »

mupuff1234 wrote: post #154
raincole wrote: post #148

Earlier quoted context omitted.

> an API that is likely a loss-leader to grab market share (hosted LLM cloud models) Everyone just repeats this but I never buy it. There is literally a service that allows you to switch models and service providers seamlessly (openrouter). There is just no lock-in. It doesn't make any financial sense to "grab market share". If you sell something with UI, like ChatGPT (the web interface) or Cursor, sure. But selling…

Except they most likely do have a plan to make it harder to switch.

Yeah, sure, please elaborate on how providers such as Fireworks, DeepInfra, Chutes are going to "make it harder to switch."

Re: LLMs are cheap

#170

Post by rco8786 »

JimDabell wrote: post #125
rco8786 wrote: post #104

Earlier quoted context omitted.

AWS isn’t doing the training on those models.

OpenAI spends less on training than inference, so the worst case scenario is less than double the cost after factoring in training. Inference is still cheap.

Inference is cheap. Training is cheaper. Then where's all the money going? OpenAI is reporting heavy losses, but you're saying the unit economics of inference are all good. What are they spending money on?
Post reply on HN