Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are cheap

snellman.net

Post reply on HN
Print view View original

31–40 of 319 posts

Re: LLMs are cheap

#31

Post by JimDabell »

eptcyka wrote: post #16
holoduke wrote: post #8

Earlier quoted context omitted.

There are more monetization ways than just a hard paying user. You can ask Google or Facebook. I dont think its super hard to get chatgpt to a. Profitable business. Its probably the most used service currently out there. And its use and effectiveness is immense.

I wonder how many more watts does producing an answer OpenAI use than answering a Google search query.

This is a good article on the subject. Make sure you read the linked articles as well.

https://andymasley.substack.com/p/reactions-to-mit-technolog...

It’s basically the same story as this article: people incorrectly believe they use a huge amount of energy (and water), but it’s actually pretty reasonable and not out of line with anything else we do.

Re: LLMs are cheap

#32

Post by iamnotagenius »

pama wrote: post #24
xxbondsxx wrote: post #5

You can't compare an API that is profitable (search) to an API that is likely a loss-leader to grab market share (hosted LLM cloud models). Sure there might not be any analysis that proves that they subsidized, but you also don't have any evidence that they are profitable. All the data points we have today show that companies are spending an insane amount of capex on gaining AI dominance without the revenue to achiev…

Please read the DeepSeek analysis of their API service (linked in this article): they have 500% profit margin and they are cheaper than any of the US companies serving the same model. It is conceivable that the API service of OpenAI or Anthropic have much higher profit margins yet. (GPUs are generally much more cost effective and energy efficient than CPU if the solution maps to both architectures. Anthropic certainl…

With all due respect to Deepseek, I would take their numbers with grain of salt, as they might as well be politically motivated.

Re: LLMs are cheap

#33

Post by pmdr »

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable.

They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't be turning a profit anytime soon.

Just look at YouTube: in business for 20 years, but it's still unclear whether it's profitable or not, as Alphabet chooses not to disclose YT's net income. I'd imagine any public company would do this, unless those numbers are in the red.

Re: LLMs are cheap

#34

Post by fkyoureadthedoc »

paxys wrote: post #20

The entire comparison hinges on people only making simple factual searches ("what is the capital of USA") on both search engines and LLMs. I'm going to say that's far enough from the standard use case for both these sets of APIs to be entirely meaningless. - If I'm using a search engine, I want to search the web. Yes these engines are increasingly providing answers rather than just search results, but that's a UI/pro…

Anecdotally, I'm a paying user and do a lot of super basic queries. What is this bug, rewrite this drivel into an email to my HOA, turn me into a gnome, what is the worst state and why is it west Virginia.

This would probably increase 10x if one of the providers sold a family plan and my kids got paid access.

Most of my heavy lifting is work related and goes through my employer's pockets.

Re: LLMs are cheap

#35

Post by bboygravity »

Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

The entire businessmodel may only work as long as inference takes up the physical space and cost of a small building.

Last time personal computing took up an entire building, we put the same compute power into a (portable) "personal computer" a few decades later.

Can't wait to send all my data and life to my own lil inference box, instead of big tech (and NSA etc).

Re: LLMs are cheap

#36

Post by dist-epoch »

exceptione wrote: post #26
theOGognf wrote: post #17

Some anecdotal data, but we recently estimated the cost of running a LLM at $WORK by looking at power usage over a bursty period of requests from our internal users and it was on the order of $10s/mil tokens. And we arent a big place, nor were our servers at max load, so I can see the cost being much lower at scale

This is only the power usage?

Hardware spend also need to be amortized (over 1 year? 2 years?) Unless you cloud rent.

Re: LLMs are cheap

#37

Post by andrew_lettuce »

jsnell wrote: post #21
Etheryte wrote: post #7

> OpenAI reportedly made a loss of $5B in 2024. They also reportedly have 500M MAUs. To reach break-even, they'd just need to monetize those free users for an average of $10/year, or $1/month. A $1 ARPU for a service like this would be pitifully low. This is a tangent to the rest of the article, but this "just" is doing more heavy lifting than Atlas holding up the skies. Taking a user from $0 to $1 is immeasurably ha…

Ok, I clearly should have made the wording more explict since this is the second comment I got in the same vein. I'm not saying you'd convert users to $1/month subscriptions. That would indeed be an absurd idea. I'm saying that good-enough LLMs are so cheap that they could easily be monetized with ads, and it's not even close. If you look at other companies with similar sized consumer-facing services monetized with a…

There are some big problems with this, mostly that openAI doesn't want to break even or be profitable, their entire setup is based on being wildly so. Building a Google sized business on ads is incredibly difficult. They need to be so much better than the competition that we have no choice but to use them, and that's not the case any more. More minor but still a major issue is the underlying IP rights. As users mature they will increasingly look for citations from LLMs, and if open AI is monetizing in this vein everyone is going to come for a piece.

Re: LLMs are cheap

#38

Post by theOGognf »

exceptione wrote: post #26
theOGognf wrote: post #17

Some anecdotal data, but we recently estimated the cost of running a LLM at $WORK by looking at power usage over a bursty period of requests from our internal users and it was on the order of $10s/mil tokens. And we arent a big place, nor were our servers at max load, so I can see the cost being much lower at scale

This is only the power usage?

Right, this is only power usage. Factoring in labor and all that would make it more expensive for sure. However, it’s not like it’s a complex system to maintain. We use a popular inference server and just run it with some modest rate limits . It’s been hands-off for close to a year at this point

Re: LLMs are cheap

#39

Post by dist-epoch »

pmdr wrote: post #33

I really doubt that, in an industry where chips are so hard to come by, draw so much power and are so terribly expensive, big players could at any time flip a switch and become profitable. They burn through insane amounts of cash and are, for some reason, still called startups. Sure, they'll be around for a long time until they figure something out, but unless hardware prices and power consumption go down, they won't…

Stock price go up is another way a company is profitable. The amazon playbook for 10+ years.
Post reply on HN